Exhibit 23.3

CONSENT OF INDEPENDENT REGISTERED PUBLIC ACCOUNTING FIRM

We consent to the incorporation by reference in this Registration Statement on Form S-4 of First PacTrust Bancorp, Inc. of our report dated March 30, 2012, relating to the consolidated financial statements and effectiveness of internal control over financial reporting appearing in the Annual Report on Form 10-K of First PacTrust Bancorp, Inc. for the year ended December 31, 2011, and to the reference to us under the heading “Experts” in the prospectus.

/s/ Crowe Horwath LLP

Costa Mesa, California

January 3, 2013