Loans Receivable and Allowance for Loan Losses (Details 3) (USD $) In Thousands, unless otherwise specified	12 Months Ended
	Dec. 31, 2014	Dec. 31, 2013	Dec. 31, 2012
Loans receivable and allowance for loan losses
Balance at the beginning of the period	$ 139,000	$ 145,501	$ 157,185
Provision (credited) charged to operations	(9,469)	19,601	40,400
Charge-offs	(29,589)	(37,040)	(61,217)
Recoveries	11,658	10,938	9,133
Balance at the end of the period	111,600	139,000	145,501
Consumer and other loans
Loans receivable and allowance for loan losses
Balance at the beginning of the period	8,824	6,316	3,800
Provision (credited) charged to operations	1,709	3,604	4,538
Charge-offs	(2,073)	(1,916)	(2,541)
Recoveries	365	820	519
Balance at the end of the period	8,825	8,824	6,316
Mortgage loans (gross) | Residential mortgage loans
Loans receivable and allowance for loan losses
Balance at the beginning of the period	80,337	89,267	105,991
Provision (credited) charged to operations	(23,464)	9,368	24,663
Charge-offs	(19,868)	(26,644)	(49,794)
Recoveries	9,278	8,346	8,407
Balance at the end of the period	46,283	80,337	89,267
Mortgage loans (gross) | Multi-family
Loans receivable and allowance for loan losses
Balance at the beginning of the period	36,703	35,514	35,422
Provision (credited) charged to operations	5,337	4,684	6,161
Charge-offs	(4,365)	(4,732)	(6,275)
Recoveries	1,575	1,237	206
Balance at the end of the period	39,250	36,703	35,514
Mortgage loans (gross) | Commercial real estate
Loans receivable and allowance for loan losses
Balance at the beginning of the period	13,136	14,404	11,972
Provision (credited) charged to operations	6,949	1,945	5,038
Charge-offs	(3,283)	(3,748)	(2,607)
Recoveries	440	535	1
Balance at the end of the period	$ 17,242	$ 13,136	$ 14,404