Loans Receivable and Allowance for Loan Losses - Narrative (Details)	3 Months Ended		6 Months Ended		12 Months Ended
	Jun. 30, 2020 USD ($) loan	Jun. 30, 2019 USD ($)	Jun. 30, 2020 USD ($) loan	Jun. 30, 2019 USD ($) loan	Dec. 31, 2019 USD ($) loan
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans held-for-sale	$ 9,600,000		$ 9,600,000		$ 0
Proceeds from sales of loans held-for-sale			103,992,000	$ 45,003,000
Loss on sale of loans held-for-investment			0
Purchases of loans receivable	0		0	29,885,000
Carrying value of servicing liability	592,800,000		592,800,000		$ 526,300,000
Proceeds from sales of loans receivable			$ 22,876,000	$ 2,483,000
Threshold period, past due status of financial receivables			30 days
Threshold period, past due for nonperforming status of financial receivables			90 days		90 days
Non-accrual	13,500,000		$ 13,500,000		$ 6,700,000
Financing receivable, 90 days or more past due, still accruing	0		0		0
Loans receivable	6,639,874,000		6,639,874,000		6,197,566,000
Real estate owned	$ 0		$ 0		$ 0
Mortgage loans in process of foreclosure, number of loans | loan	1		1		4
Mortgage loans in process of foreclosure, amount	$ 180,000		$ 180,000		$ 522,000
Specific allowance for loan losses attributable to impaired loans	2,129,000		2,129,000		1,621,000
Impaired loans for which there are no related allowance for loan losses	10,221,000		10,221,000		10,747,000
Financing receivable, modifications, recorded investment	31,300,000		$ 31,300,000		20,000,000.0
Number of loans | loan			1	1
Real estate loans
Accounts, Notes, Loans and Financing Receivable [Line Items]
Proceeds from sale of loans held-for-investment			$ 8,800,000
Gain on sale of loans held-for-investment		$ 0	82,000	$ 0
Purchases of loans receivable		24,900,000		24,900,000
Real estate loans | Home equity loans and advances
Accounts, Notes, Loans and Financing Receivable [Line Items]
Proceeds from sale of loans held-for-investment		2,500,000		$ 2,500,000
Real estate loans | Multifamily and commercial
Accounts, Notes, Loans and Financing Receivable [Line Items]
Specific allowance for loan losses attributable to impaired loans	584,000		584,000		2,000
Impaired loans for which there are no related allowance for loan losses	735,000		$ 735,000		1,494,000
Number of loans | loan			1	1
Real estate loans | Residential
Accounts, Notes, Loans and Financing Receivable [Line Items]
Proceeds from sales of loans held-for-sale	52,400,000	27,600,000	$ 104,000,000.0	$ 45,000,000.0
Gain on sale of loans held-for-sale	740,000	196,000	1,400,000	328,000
Loss on sale of loans held-for-sale	0	0	0	0
Purchases of loans receivable		2,600,000		5,000,000.0
Specific allowance for loan losses attributable to impaired loans	434,000		434,000		484,000
Impaired loans for which there are no related allowance for loan losses	4,418,000		4,418,000		4,314,000
Real estate loans | Construction
Accounts, Notes, Loans and Financing Receivable [Line Items]
Proceeds from sale of loans held-for-investment	6,700,000		6,700,000
Gain on sale of loans held-for-investment	$ 0		$ 0
Number of loans held-for-investment | loan	1		1
Commercial business loans
Accounts, Notes, Loans and Financing Receivable [Line Items]
Proceeds from sale of loans held-for-investment	$ 579,000		$ 7,300,000
Gain on sale of loans held-for-investment	55,000		55,000
Specific allowance for loan losses attributable to impaired loans	1,100,000		1,100,000		1,121,000
Impaired loans for which there are no related allowance for loan losses	4,016,000		4,016,000		3,859,000
Less Than 90 Days
Accounts, Notes, Loans and Financing Receivable [Line Items]
Non-accrual	4,200,000		$ 4,200,000		$ 1,500,000
Financial receivable, number of loans in nonaccrual status | loan			21		8
30-59 Days
Accounts, Notes, Loans and Financing Receivable [Line Items]
Financing receivable, modifications, recorded investment	603,000		$ 603,000		$ 660,000
Number of loans | loan			3		3
60-89 Days
Accounts, Notes, Loans and Financing Receivable [Line Items]
Financing receivable, modifications, recorded investment	364,000		$ 364,000
Number of loans | loan			2
90 Days or More
Accounts, Notes, Loans and Financing Receivable [Line Items]
Financing receivable, modifications, recorded investment	1,600,000		$ 1,600,000
Number of loans | loan			6		0
Freddie Mac
Accounts, Notes, Loans and Financing Receivable [Line Items]
Proceeds from sales of loans receivable	0	$ 15,600,000	$ 0	$ 21,600,000
Financial Asset Acquired with Credit Deterioration
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans receivable					$ 6,900,000
Stewardship Financial Corporation | Financial Asset Acquired with Credit Deterioration
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans receivable	6,900,000		6,900,000
Roselle Entities | Financial Asset Acquired with Credit Deterioration
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans receivable	226,000		226,000
Paycheck Protection Program | Commercial business loans
Accounts, Notes, Loans and Financing Receivable [Line Items]
Financing receivable, before allowance for credit loss	467,000,000.0		467,000,000.0		$ 0
Financing receivable, unamortized loan fee	$ 13,500,000		$ 13,500,000