Exhibit 23.1

CONSENT OF INDEPENDENT AUDITORS

We consent to the incorporation by reference in the Registration Statements on Form S-3 (Nos. 333-41329, 333-61625, 333-71329 and 333-67964) and Form S-8 (Nos. 333-36085, 333-55907, 333-48332 and 333-97339) and related Prospectuses of IndyMac Bancorp, Inc. of our reports dated January 23, 2004 with respect to the consolidated financial statements of IndyMac Bancorp, Inc. and management’s assertion it maintained effective internal control over financial reporting. Both reports are included in this Annual Report (Form 10-K) for the year ended December 31, 2003.

/s/ ERNST & YOUNG LLP

Los Angeles, California
March 4, 2004