Financing Receivables - Recorded investment in loans (Details) (Loans Receivable, USD $) In Thousands, unless otherwise specified	Mar. 31, 2014		Dec. 31, 2013
Accounts, Notes, Loans and Financing Receivable [Line Items]
Purchased loans	$ 110,587
Fair value discount	(1,968)
Purchased loans book value	108,619
Originated loans	227,989	[1]
Ending balance	336,608		341,215
One-to-four Family Residential
Accounts, Notes, Loans and Financing Receivable [Line Items]
Purchased loans	75,997
Fair value discount	(1,055)
Purchased loans book value	74,942
Originated loans	157,045	[1]
Ending balance	231,987		236,236
Multi-family Residential
Accounts, Notes, Loans and Financing Receivable [Line Items]
Purchased loans	6,115
Fair value discount	(159)
Purchased loans book value	5,956
Originated loans	14,054	[1]
Ending balance	20,010		22,805
Construction
Accounts, Notes, Loans and Financing Receivable [Line Items]
Purchased loans
Fair value discount
Purchased loans book value
Originated loans	7,040	[1],[2]
Ending balance	7,040		7,141
Commercial
Accounts, Notes, Loans and Financing Receivable [Line Items]
Purchased loans	26,783
Fair value discount	(520)
Purchased loans book value	26,263
Originated loans	49,064	[1]
Ending balance	75,327		72,755
Consumer
Accounts, Notes, Loans and Financing Receivable [Line Items]
Purchased loans	1,692
Fair value discount	(234)
Purchased loans book value	1,458
Originated loans	786	[1]
Ending balance	$ 2,244		$ 2,278

[1] Includes loans held for sale

[2] Before consideration of undisbursed Loans-in-process