Loans Receivable and Allowance for Loan Losses - Narrative (Details)	3 Months Ended		9 Months Ended		12 Months Ended
	Sep. 30, 2020 USD ($) loan	Sep. 30, 2019 USD ($) loan	Sep. 30, 2020 USD ($) loan	Sep. 30, 2019 USD ($) loan	Dec. 31, 2019 USD ($) loan
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans held-for-sale	$ 4,100,000		$ 4,100,000		$ 0
Proceeds from sales of loans held-for-sale			111,764,000	$ 94,834,000
Loss on sale of loans held-for-investment			0
Purchases of loans receivable	0		0	29,885,000
Proceeds from sales of loans receivable			28,923,000	$ 10,273,000
Carrying value of servicing liability	602,500,000		$ 602,500,000		$ 526,300,000
Threshold period, past due status of financial receivables			30 days
Threshold period, past due for nonperforming status of financial receivables			90 days		90 days
Non-accrual	10,900,000		$ 10,900,000		$ 6,700,000
Financing receivable, 90 days or more past due, still accruing	0		0		0
Loans receivable	6,468,845,000		6,468,845,000		6,197,566,000
Real estate owned	$ 0		$ 0		$ 0
Mortgage loans in process of foreclosure, number of loans | loan	1		1		4
Mortgage loans in process of foreclosure, amount	$ 180,000		$ 180,000		$ 522,000
Specific allowance for loan losses attributable to impaired loans	1,839,000		1,839,000		1,621,000
Impaired loans for which there are no related allowance for loan losses	26,499,000		26,499,000		10,747,000
Financing receivable, modifications, recorded investment	$ 44,100,000		$ 44,100,000		20,000,000.0
Number of loans | loan	5	0	6	1
Real estate loans
Accounts, Notes, Loans and Financing Receivable [Line Items]
Proceeds from sale of loans held-for-investment		$ 5,500,000		$ 5,500,000
Gain on sale of loans held-for-investment		0		0
Purchases of loans receivable				24,900,000
Real estate loans | Home equity loans and advances
Accounts, Notes, Loans and Financing Receivable [Line Items]
Proceeds from sale of loans held-for-investment		2,300,000		4,800,000
Gain on sale of loans held-for-investment		0		0
Real estate loans | Residential
Accounts, Notes, Loans and Financing Receivable [Line Items]
Proceeds from sales of loans held-for-sale	$ 7,800,000	49,100,000	$ 111,800,000	94,800,000
Gain on sale of loans held-for-sale	327,000	382,000	1,700,000	710,000
Loss on sale of loans held-for-sale	0	0	0	0
Proceeds from sale of loans held-for-investment			8,800,000
Gain on sale of loans held-for-investment			82,000
Purchases of loans receivable		$ 0		$ 5,000,000.0
Specific allowance for loan losses attributable to impaired loans	406,000		406,000		484,000
Impaired loans for which there are no related allowance for loan losses	$ 2,825,000		2,825,000		4,314,000
Real estate loans | Construction
Accounts, Notes, Loans and Financing Receivable [Line Items]
Number of loans held-for-investment | loan	1
Proceeds from sale of loans held-for-investment	$ 5,800,000		12,500,000
Gain on sale of loans held-for-investment	$ 0		0
Commercial business loans
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loss on sale of loans held-for-sale			0
Number of loans held-for-investment | loan	1
Proceeds from sale of loans held-for-investment	$ 274,000		7,600,000
Gain on sale of loans held-for-investment	23,000		78,000
Specific allowance for loan losses attributable to impaired loans	871,000		871,000		1,121,000
Impaired loans for which there are no related allowance for loan losses	$ 2,577,000		$ 2,577,000		3,859,000
Number of loans | loan	1	0	2	1
Less Than 90 Days
Accounts, Notes, Loans and Financing Receivable [Line Items]
Non-accrual	$ 3,900,000		$ 3,900,000		$ 1,500,000
Financial receivable, number of loans in nonaccrual status | loan			22		8
30-59 Days
Accounts, Notes, Loans and Financing Receivable [Line Items]
Financing receivable, modifications, recorded investment	420,000		$ 420,000		$ 660,000
Number of loans | loan			1		3
60-89 Days
Accounts, Notes, Loans and Financing Receivable [Line Items]
Financing receivable, modifications, recorded investment	262,000		$ 262,000
Number of loans | loan			1
90 Days or More
Accounts, Notes, Loans and Financing Receivable [Line Items]
Financing receivable, modifications, recorded investment	910,000		$ 910,000
Number of loans | loan			3		0
Freddie Mac
Accounts, Notes, Loans and Financing Receivable [Line Items]
Proceeds from sales of loans receivable	48,500,000		$ 48,500,000	$ 21,600,000
Gain on sale of loans receivable	1,500,000		1,500,000
Loss on sale of loans receivable	0		0
Proceeds from securitizations of loans held-for-investment		$ 0
Stewardship Financial Corporation | Financial Asset Acquired with Credit Deterioration
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans receivable	5,900,000		5,900,000		$ 6,900,000
Roselle Entities | Financial Asset Acquired with Credit Deterioration
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans receivable	179,000		179,000
Paycheck Protection Program | Commercial business loans
Accounts, Notes, Loans and Financing Receivable [Line Items]
Financing receivable, before allowance for credit loss	474,900,000		474,900,000		$ 0
Financing receivable, unamortized loan fee	$ 11,200,000		$ 11,200,000