Exhibit 23.1

Consent of Independent Registered Public Accounting Firm

We consent to the reference to our firm under each of the captions “Experts” in the Registration Statement (Form S-3) of Capital One Financial Corporation, Capital One Capital II, Capital One Capital III, and Capital One Capital IV and each of the two Prospectuses contained therein for the registration of debt securities, preferred stock, depositary shares for preferred stock, common stock, purchase contracts, and units of Capital One Financial Corporation and trust preferred securities of Capital One Capital II, Capital One Capital III, and Capital One Capital IV and junior subordinated debt securities of Capital One Financial Corporation and to the incorporation by reference therein of our report dated February 11, 2005, except for Note 24, as to which the date is March 7, 2005, with respect to the consolidated financial statements of Capital One Financial Corporation, and our report dated February 11, 2005, with respect to Capital One Financial Corporation management’s assessment of the effectiveness of internal control over financial reporting, and the effectiveness of internal control over financial reporting of Capital One Financial Corporation, included in Capital One Financial Corporation’s Annual Report (Form 10-K) for the year ended December 31, 2004, filed with the Securities and Exchange Commission.

/s/ Ernst & Young LLP

McLean, Virginia

July 5, 2005

1