Exhibit 23

Consent of Independent Registered Public Accounting Firm

We consent to the incorporation by reference in the Registration Statement (Form S-3 No. 333-159145) of Hatteras Financial Corp. and in the related Prospectus of our reports dated February 23, 2012, with respect to the financial statements of Hatteras Financial Corp., and the effectiveness of internal control over financial reporting of Hatteras Financial Corp., included in this Annual Report (Form 10-K) for the year ended December 31, 2011.

/s/ Ernst & Young LLP

Raleigh, North Carolina

February 23, 2012