Exhibit 23.2

CONSENT OF INDEPENDENT REGISTERED PUBLIC ACCOUNTING FIRM

Consent of Independent Registered Public Accounting Firm

The Board of Directors and Stockholders

Territorial Bancorp Inc.:

We consent to the use of our report dated March 31, 2010, with respect to the consolidated balance sheets of Territorial Bancorp Inc. as of December 31, 2009 and 2008, and the related consolidated statements of income, stockholders’ equity and comprehensive income, and cash flows for each of the years in the three-year period ended December 31, 2009, incorporated by reference herein, which report appears in the December 31, 2009 annual report on Form 10-K of Territorial Bancorp Inc.

/s/ KPMG LLP

Honolulu, Hawaii

November 12, 2010