Exhibit 23

Consent of Independent Registered Public Accounting Firm

We consent to the incorporation by reference in the Registration Statements on Form S-8 (No. 333-168839, No. 333-170579, No. 333-218723, and No. 333-237039) of Territorial Bancorp Inc. and Subsidiaries (the “Company”) of our report dated March 15, 2024, relating to the consolidated financial statements of the Company (which report expresses an unqualified opinion and includes an explanatory paragraph relating to the change in the method of accounting for credit losses on loans), appearing in this Annual Report on Form 10-K for the year ended December 31, 2023.

/s/ Moss Adams LLP

Portland, Oregon

March 15, 2024