Exhibit 23

CONSENT OF INDEPENDENT REGISTERED PUBLIC ACCOUNTING FIRM

The Board of Directors

Yadkin Valley Financial Corporation

Elkin, NC

We consent to the incorporation by reference in the Form S-8 Registration Statements Nos. 333-167861, 333-158982, 333-153339, 333-150190, 333-136970, 333-136969, 333-136968, and 333-136967 and the Form S-3D Registration Statement No. 333-136050 of Yadkin Valley Financial Corporation of our reports dated February 28, 2012 with respect to the consolidated financial statements of Yadkin Valley Financial Corporation and the effectiveness of internal control over financial reporting, which reports appear in the December 31, 2011 annual report on Form 10-K of Yadkin Valley Financial Corporation, Elkin, NC.

/s/ Dixon Hughes Goodman LLP

Charlotte, North Carolina

February 28, 2012