EXHIBIT 23

CONSENT OF INDEPENDENT REGISTERED PUBLIC ACCOUNTING FIRM

We consent to the incorporation by reference in Registration Statement No. 333-54710 on Form S-8 of our reports dated March 16, 2005, relating to the consolidated financial statements of Beverly Hills Bancorp Inc. and Subsidiaries and management’s report on the effectiveness of internal control over financial reporting, appearing in this Annual Report on Form 10-K of Beverly Hills Bancorp, Inc. and Subsidiaries for the year ended December 31, 2004.

/s/    Deloitte & Touche LLP

Los Angeles, California

March 16, 2005