Loans Receivable and Allowance for Loan Losses - Balances of Residential Mortgage, Consumer, and Other Loan Receivables Credit Quality (Details) - USD ($) $ in Thousands	Dec. 31, 2015	Dec. 31, 2014
Recorded Investment
Total	$ 11,111,727	$ 11,909,722
Consumer and Other Loans
Recorded Investment
Total	252,693	249,368
Consumer and Other Loans | Home Equity and Other Consumer
Recorded Investment
Total	160,819	184,553
Consumer and Other Loans | Commercial and industrial
Recorded Investment
Total	91,874	64,815
Mortgage Loans
Recorded Investment
Total	10,859,034	11,660,354
Mortgage Loans | Residential Mortgage Loans
Recorded Investment
Total	6,015,415	6,873,536
Mortgage Loans | Residential Mortgage Loans | Full documentation interest-only
Recorded Investment
Total	432,288	854,182
Mortgage Loans | Residential Mortgage Loans | Full documentation amortizing
Recorded Investment
Total	4,697,966	5,003,693
Mortgage Loans | Residential Mortgage Loans | Reduced documentation interest-only
Recorded Investment
Total	303,231	611,008
Mortgage Loans | Residential Mortgage Loans | Reduced documentation amortizing
Recorded Investment
Total	581,930	404,653
Performing | Consumer and Other Loans | Home Equity and Other Consumer
Recorded Investment
Total	154,414	178,513
Performing | Consumer and Other Loans | Commercial and industrial
Recorded Investment
Total	91,171	64,815
Performing | Mortgage Loans | Residential Mortgage Loans | Full documentation interest-only
Recorded Investment
Total	413,913	826,155
Performing | Mortgage Loans | Residential Mortgage Loans | Full documentation amortizing
Recorded Investment
Total	4,653,437	4,981,880
Performing | Mortgage Loans | Residential Mortgage Loans | Reduced documentation interest-only
Recorded Investment
Total	275,659	568,424
Performing | Mortgage Loans | Residential Mortgage Loans | Reduced documentation amortizing
Recorded Investment
Total	552,070	396,922
Non-performing | Consumer and Other Loans | Current or past due less than 90 days | Home Equity and Other Consumer
Recorded Investment
Total	0	0
Non-performing | Consumer and Other Loans | Current or past due less than 90 days | Commercial and industrial
Recorded Investment
Total	0	0
Non-performing | Consumer and Other Loans | Past due 90 days or more | Home Equity and Other Consumer
Recorded Investment
Total	6,405	6,040
Non-performing | Consumer and Other Loans | Past due 90 days or more | Commercial and industrial
Recorded Investment
Total	703	0
Non-performing | Mortgage Loans | Residential Mortgage Loans | Current or past due less than 90 days | Full documentation interest-only
Recorded Investment
Total	7,016	16,525
Non-performing | Mortgage Loans | Residential Mortgage Loans | Current or past due less than 90 days | Full documentation amortizing
Recorded Investment
Total	11,662	7,458
Non-performing | Mortgage Loans | Residential Mortgage Loans | Current or past due less than 90 days | Reduced documentation interest-only
Recorded Investment
Total	12,287	26,295
Non-performing | Mortgage Loans | Residential Mortgage Loans | Current or past due less than 90 days | Reduced documentation amortizing
Recorded Investment
Total	15,538	4,888
Non-performing | Mortgage Loans | Residential Mortgage Loans | Past due 90 days or more | Full documentation interest-only
Recorded Investment
Total	11,359	11,502
Non-performing | Mortgage Loans | Residential Mortgage Loans | Past due 90 days or more | Full documentation amortizing
Recorded Investment
Total	32,867	14,355
Non-performing | Mortgage Loans | Residential Mortgage Loans | Past due 90 days or more | Reduced documentation interest-only
Recorded Investment
Total	15,285	16,289
Non-performing | Mortgage Loans | Residential Mortgage Loans | Past due 90 days or more | Reduced documentation amortizing
Recorded Investment
Total	$ 14,322	$ 2,843