Fair Value Measurements - Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis (Details) - USD ($) $ in Thousands	Mar. 31, 2020	Dec. 31, 2019
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Debt securities available-for-sale	$ 1,061,443	$ 1,138,352
Trading securities	8,388	11,222
Impaired loans	4,208	4,942
Measured on a recurring basis
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Debt securities available-for-sale	1,061,443	1,138,352
Trading securities	8,388	11,222
Equity securities	374	250
Total	1,070,205	1,149,824
Measured on a recurring basis | Mortgage-backed securities:
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Debt securities available-for-sale	934,498	973,127
Measured on a recurring basis | Pass-through certificates, GSE
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Debt securities available-for-sale	310,869	329,407
Measured on a recurring basis | REMICs, GSE
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Debt securities available-for-sale	623,580	643,667
Measured on a recurring basis | Non-GSE
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Debt securities available-for-sale	49	53
Measured on a recurring basis | Municipal bonds
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Debt securities available-for-sale	279	299
Measured on a recurring basis | Corporate bonds
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Debt securities available-for-sale	126,666	164,926
Measured on a recurring basis | Other debt obligations
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Debt securities available-for-sale	126,945	165,225
Measured on a recurring basis | Quoted Prices in Active Markets for Identical Assets (Level 1)
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Debt securities available-for-sale	0	0
Trading securities	8,388	11,222
Equity securities	374	250
Total	8,762	11,472
Measured on a recurring basis | Quoted Prices in Active Markets for Identical Assets (Level 1) | Mortgage-backed securities:
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Debt securities available-for-sale	0	0
Measured on a recurring basis | Quoted Prices in Active Markets for Identical Assets (Level 1) | Pass-through certificates, GSE
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Debt securities available-for-sale	0	0
Measured on a recurring basis | Quoted Prices in Active Markets for Identical Assets (Level 1) | REMICs, GSE
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Debt securities available-for-sale	0	0
Measured on a recurring basis | Quoted Prices in Active Markets for Identical Assets (Level 1) | Non-GSE
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Debt securities available-for-sale	0	0
Measured on a recurring basis | Quoted Prices in Active Markets for Identical Assets (Level 1) | Municipal bonds
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Debt securities available-for-sale	0	0
Measured on a recurring basis | Quoted Prices in Active Markets for Identical Assets (Level 1) | Corporate bonds
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Debt securities available-for-sale	0	0
Measured on a recurring basis | Quoted Prices in Active Markets for Identical Assets (Level 1) | Other debt obligations
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Debt securities available-for-sale	0	0
Measured on a recurring basis | Significant Other Observable Inputs (Level 2)
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Debt securities available-for-sale	1,061,443	1,138,352
Trading securities	0	0
Equity securities	0	0
Total	1,061,443	1,138,352
Measured on a recurring basis | Significant Other Observable Inputs (Level 2) | Mortgage-backed securities:
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Debt securities available-for-sale	934,498	973,127
Measured on a recurring basis | Significant Other Observable Inputs (Level 2) | Pass-through certificates, GSE
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Debt securities available-for-sale	310,869	329,407
Measured on a recurring basis | Significant Other Observable Inputs (Level 2) | REMICs, GSE
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Debt securities available-for-sale	623,580	643,667
Measured on a recurring basis | Significant Other Observable Inputs (Level 2) | Non-GSE
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Debt securities available-for-sale	49	53
Measured on a recurring basis | Significant Other Observable Inputs (Level 2) | Municipal bonds
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Debt securities available-for-sale	279	299
Measured on a recurring basis | Significant Other Observable Inputs (Level 2) | Corporate bonds
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Debt securities available-for-sale	126,666	164,926
Measured on a recurring basis | Significant Other Observable Inputs (Level 2) | Other debt obligations
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Debt securities available-for-sale	126,945	165,225
Measured on a recurring basis | Significant Unobservable Inputs (Level 3)
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Debt securities available-for-sale	0	0
Trading securities	0	0
Equity securities	0	0
Total	0	0
Measured on a recurring basis | Significant Unobservable Inputs (Level 3) | Mortgage-backed securities:
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Debt securities available-for-sale	0	0
Measured on a recurring basis | Significant Unobservable Inputs (Level 3) | Pass-through certificates, GSE
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Debt securities available-for-sale	0	0
Measured on a recurring basis | Significant Unobservable Inputs (Level 3) | REMICs, GSE
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Debt securities available-for-sale	0	0
Measured on a recurring basis | Significant Unobservable Inputs (Level 3) | Non-GSE
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Debt securities available-for-sale	0	0
Measured on a recurring basis | Significant Unobservable Inputs (Level 3) | Municipal bonds
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Debt securities available-for-sale	0	0
Measured on a recurring basis | Significant Unobservable Inputs (Level 3) | Corporate bonds
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Debt securities available-for-sale	0	0
Measured on a recurring basis | Significant Unobservable Inputs (Level 3) | Other debt obligations
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Debt securities available-for-sale	0	0
Measured on a non-recurring basis
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Total	4,208	4,942
Measured on a non-recurring basis | Real Estate
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Impaired loans	4,194	4,927
Measured on a non-recurring basis | Real Estate | Commercial real estate
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Impaired loans	4,143	4,871
Measured on a non-recurring basis | Real Estate | Multifamily
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Impaired loans	22	26
Measured on a non-recurring basis | Real Estate | Home equity and lines of credit
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Impaired loans	29	30
Measured on a non-recurring basis | Commercial and Industrial
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Impaired loans	14	15
Measured on a non-recurring basis | Quoted Prices in Active Markets for Identical Assets (Level 1)
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Total	0	0
Measured on a non-recurring basis | Quoted Prices in Active Markets for Identical Assets (Level 1) | Real Estate
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Impaired loans	0	0
Measured on a non-recurring basis | Quoted Prices in Active Markets for Identical Assets (Level 1) | Real Estate | Commercial real estate
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Impaired loans	0	0
Measured on a non-recurring basis | Quoted Prices in Active Markets for Identical Assets (Level 1) | Real Estate | Multifamily
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Impaired loans	0	0
Measured on a non-recurring basis | Quoted Prices in Active Markets for Identical Assets (Level 1) | Real Estate | Home equity and lines of credit
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Impaired loans	0	0
Measured on a non-recurring basis | Quoted Prices in Active Markets for Identical Assets (Level 1) | Commercial and Industrial
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Impaired loans	0	0
Measured on a non-recurring basis | Significant Other Observable Inputs (Level 2)
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Total	0	0
Measured on a non-recurring basis | Significant Other Observable Inputs (Level 2) | Real Estate
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Impaired loans	0	0
Measured on a non-recurring basis | Significant Other Observable Inputs (Level 2) | Real Estate | Commercial real estate
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Impaired loans	0	0
Measured on a non-recurring basis | Significant Other Observable Inputs (Level 2) | Real Estate | Multifamily
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Impaired loans	0	0
Measured on a non-recurring basis | Significant Other Observable Inputs (Level 2) | Real Estate | Home equity and lines of credit
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Impaired loans	0	0
Measured on a non-recurring basis | Significant Other Observable Inputs (Level 2) | Commercial and Industrial
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Impaired loans	0	0
Measured on a non-recurring basis | Significant Unobservable Inputs (Level 3)
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Total	4,208	4,942
Measured on a non-recurring basis | Significant Unobservable Inputs (Level 3) | Real Estate
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Impaired loans	4,194	4,927
Measured on a non-recurring basis | Significant Unobservable Inputs (Level 3) | Real Estate | Commercial real estate
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Impaired loans	4,143	4,871
Measured on a non-recurring basis | Significant Unobservable Inputs (Level 3) | Real Estate | Multifamily
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Impaired loans	22	26
Measured on a non-recurring basis | Significant Unobservable Inputs (Level 3) | Real Estate | Home equity and lines of credit
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Impaired loans	29	30
Measured on a non-recurring basis | Significant Unobservable Inputs (Level 3) | Commercial and Industrial
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Impaired loans	$ 14	$ 15