Fair Value Measurement - Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis (Details) - USD ($) $ in Thousands	Dec. 31, 2019	Dec. 31, 2018
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Debt securities available-for-sale	$ 1,138,352	$ 808,031
Impaired loans	4,942	5,697
Recurring Basis
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Debt securities available-for-sale	1,138,352	808,031
Trading securities	11,222	8,968
Equity securities	250	237
Total	1,149,824	817,236
Recurring Basis | Level 1
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Debt securities available-for-sale	0	0
Trading securities	11,222	8,968
Equity securities	250	237
Total	11,472	9,205
Recurring Basis | Level 2
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Debt securities available-for-sale	1,138,352	808,031
Trading securities	0	0
Equity securities	0	0
Total	1,138,352	808,031
Recurring Basis | Level 3
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Debt securities available-for-sale	0	0
Trading securities	0	0
Equity securities	0	0
Total	0	0
Recurring Basis | Mortgage-backed securities:
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Debt securities available-for-sale	973,127	565,009
Recurring Basis | Mortgage-backed securities: | Level 1
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Debt securities available-for-sale	0	0
Recurring Basis | Mortgage-backed securities: | Level 2
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Debt securities available-for-sale	973,127	565,009
Recurring Basis | Mortgage-backed securities: | Level 3
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Debt securities available-for-sale	0	0
Recurring Basis | Mortgage-backed securities, Pass-through certificates, GSE
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Debt securities available-for-sale	329,407	314,788
Recurring Basis | Mortgage-backed securities, Pass-through certificates, GSE | Level 1
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Debt securities available-for-sale	0	0
Recurring Basis | Mortgage-backed securities, Pass-through certificates, GSE | Level 2
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Debt securities available-for-sale	329,407	314,788
Recurring Basis | Mortgage-backed securities, Pass-through certificates, GSE | Level 3
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Debt securities available-for-sale	0	0
Recurring Basis | Mortgage-backed securities, REMICs, GSE
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Debt securities available-for-sale	643,667	250,163
Recurring Basis | Mortgage-backed securities, REMICs, GSE | Level 1
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Debt securities available-for-sale	0
Recurring Basis | Mortgage-backed securities, REMICs, GSE | Level 2
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Debt securities available-for-sale	643,667	250,163
Recurring Basis | Mortgage-backed securities, REMICs, GSE | Level 3
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Debt securities available-for-sale	0	0
Recurring Basis | Mortgage-backed securities, REMICs, Non-GSE
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Debt securities available-for-sale	53	58
Recurring Basis | Mortgage-backed securities, REMICs, Non-GSE | Level 1
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Debt securities available-for-sale	0	0
Recurring Basis | Mortgage-backed securities, REMICs, Non-GSE | Level 2
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Debt securities available-for-sale	53	58
Recurring Basis | Mortgage-backed securities, REMICs, Non-GSE | Level 3
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Debt securities available-for-sale	0	0
Recurring Basis | Municipal bonds
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Debt securities available-for-sale	299	273
Recurring Basis | Municipal bonds | Level 1
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Debt securities available-for-sale	0	0
Recurring Basis | Municipal bonds | Level 2
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Debt securities available-for-sale	299	273
Recurring Basis | Municipal bonds | Level 3
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Debt securities available-for-sale		0
Recurring Basis | Corporate bonds
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Debt securities available-for-sale	164,926	242,749
Recurring Basis | Corporate bonds | Level 1
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Debt securities available-for-sale	0	0
Recurring Basis | Corporate bonds | Level 2
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Debt securities available-for-sale	164,926	242,749
Recurring Basis | Corporate bonds | Level 3
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Debt securities available-for-sale	0	0
Recurring Basis | Other debt obligations
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Debt securities available-for-sale	165,225	243,022
Recurring Basis | Other debt obligations | Level 1
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Debt securities available-for-sale	0	0
Recurring Basis | Other debt obligations | Level 2
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Debt securities available-for-sale	165,225	243,022
Recurring Basis | Other debt obligations | Level 3
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Debt securities available-for-sale	0	0
Nonrecurring
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Total	4,942	5,697
Nonrecurring | Level 1
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Total	0	0
Nonrecurring | Level 2
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Total	0	0
Nonrecurring | Level 3
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Total	4,942	5,697
Real Estate | Nonrecurring
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Impaired loans	4,927	5,679
Real Estate | Nonrecurring | Level 1
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Impaired loans	0	0
Real Estate | Nonrecurring | Level 2
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Impaired loans	0	0
Real Estate | Nonrecurring | Level 3
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Impaired loans	4,927	5,679
Real Estate | Nonrecurring | Commercial real estate
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Impaired loans	4,871	4,847
Real Estate | Nonrecurring | Commercial real estate | Level 1
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Impaired loans	0	0
Real Estate | Nonrecurring | Commercial real estate | Level 2
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Impaired loans	0	0
Real Estate | Nonrecurring | Commercial real estate | Level 3
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Impaired loans	4,871	4,847
Real Estate | Nonrecurring | One-to-four family residential mortgage
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Impaired loans		765
Real Estate | Nonrecurring | One-to-four family residential mortgage | Level 1
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Impaired loans		0
Real Estate | Nonrecurring | One-to-four family residential mortgage | Level 2
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Impaired loans		0
Real Estate | Nonrecurring | One-to-four family residential mortgage | Level 3
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Impaired loans		765
Real Estate | Nonrecurring | Multifamily
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Impaired loans	26	39
Real Estate | Nonrecurring | Multifamily | Level 1
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Impaired loans	0	0
Real Estate | Nonrecurring | Multifamily | Level 2
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Impaired loans	0	0
Real Estate | Nonrecurring | Multifamily | Level 3
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Impaired loans	26	39
Real Estate | Nonrecurring | Home equity and lines of credit
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Impaired loans	30	28
Real Estate | Nonrecurring | Home equity and lines of credit | Level 1
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Impaired loans	0	0
Real Estate | Nonrecurring | Home equity and lines of credit | Level 2
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Impaired loans	0	0
Real Estate | Nonrecurring | Home equity and lines of credit | Level 3
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Impaired loans	30	28
Commercial and Industrial | Nonrecurring
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Impaired loans	15	18
Commercial and Industrial | Nonrecurring | Level 1
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Impaired loans	0	0
Commercial and Industrial | Nonrecurring | Level 2
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Impaired loans	0	0
Commercial and Industrial | Nonrecurring | Level 3
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Impaired loans	$ 15	$ 18