Exhibit 23

Consent of Independent Registered Public Accounting Firm

The Board of Directors

PFF Bancorp, Inc.:

We consent to the incorporation by reference in the registration statements (Nos. 333-120113, 333-20337 and 333-65816) on Form S-8 of PFF Bancorp, Inc. of our report dated June 8, 2006, with respect to the consolidated balance sheets of PFF Bancorp, Inc. as of March 31, 2006 and 2005, and the related consolidated statements of earnings, comprehensive earnings, stockholders’ equity, and cash flows, for each of the years in the three-year period ended March 31, 2006, management’s assessment of the effectiveness of internal control over financial reporting as of March 31, 2006 and the effectiveness of internal control over financial reporting as of March 31, 2006, which reports appear in the March 31, 2006, annual report on Form 10-K of PFF Bancorp, Inc. Our report refers to changes in accounting for share based payments in 2006.

/s/    KPMG LLP

Los Angeles, California

June 8, 2006