Loans Receivable and Allowance for Loan Losses - Narrative (Details) - USD ($) $ in Thousands	9 Months Ended	12 Months Ended
	Sep. 30, 2010	Dec. 31, 2016	Dec. 31, 2015	Dec. 31, 2014	Dec. 31, 2013
Accounts, Notes, Loans and Financing Receivable [Line Items]
Recorded investment in residential real estate property		$ 73,400	$ 51,000
Accrued interest receivable on all loans		34,994	34,996
Loans classified as TDRs as a result of relief granted under Chapter 7 bankruptcy filing		4,400
Non-performing
Accounts, Notes, Loans and Financing Receivable [Line Items]
Interest income that would have been recorded if all non-accrual loans had been performing		6,300	5,400	$ 5,600
Actual payments recorded as interest income		2,800	3,400	3,600
Mortgage Loans
Accounts, Notes, Loans and Financing Receivable [Line Items]
Modifications in a TDR		15,215	18,556	$ 21,471
All loans
Accounts, Notes, Loans and Financing Receivable [Line Items]
Accrued interest receivable on all loans		$ 26,900	27,600
Residential
Accounts, Notes, Loans and Financing Receivable [Line Items]
Amortizing hybrid ARM fixed rate period one		5 years			3 years
Amortizing hybrid ARM fixed rate period two		7 years
Amortizing hybrid ARM fixed rate period three		10 years
Term of ARM loans after initial fixed rate period	1 year	1 year
Amortizing interest-only hybrid ARM fixed rate period two	5 years
Amortizing interest-only hybrid ARM fixed rate period three	7 years
Interest-only payment term of interest-only hybrid ARM loans		10 years
Residential | Interest-only | Mortgage Loans
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans		$ 45,400	388,000
Residential | Reduced documentation | Mortgage Loans
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans		731,800	885,200
Residential | Reduced documentation | Mortgage Loans | SISA (stated income, stated asset) loans
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans		$ 114,800	$ 135,700
Residential | Geographic concentration risk | Minimum
Accounts, Notes, Loans and Financing Receivable [Line Items]
Threshold for concentration risk, as a percent		5.00%
Interest rate modifications
Accounts, Notes, Loans and Financing Receivable [Line Items]
Modifications in a TDR		$ 8,300
Payment deferrals or term extensions
Accounts, Notes, Loans and Financing Receivable [Line Items]
Modifications in a TDR		$ 2,500