Exhibit 23

CONSENT OF INDEPENDENT REGISTERED PUBLIC ACCOUNTING FIRM

We consent to the incorporation by reference in Registration Statement Nos. 333-29157, 333-102383, 333-61640 and 333-108032 on Form S-8 of Bank of Granite Corporation of our reports dated March 2, 2006, relating to the financial statements of Bank of Granite Corporation and management’s report on the effectiveness of internal control over financial reporting, appearing in this Annual Report on Form 10-K of Bank of Granite Corporation for the year ended December 31, 2005.

/s/ Deloitte & Touche LLP

Hickory, North Carolina

March 14, 2006