Fair Value Measurements (Details 2) (USD $) In Thousands, unless otherwise specified	Dec. 31, 2013	Dec. 31, 2012
Carrying value of assets
Impaired loans	$ 384,522	$ 346,906
MSR, net	12,800	6,947
REO, net	42,636	28,523
Carrying Value
Carrying value of assets
Loans held-for-sale, net	7,375	76,306
MSR, net	12,800	6,947
Level 3
Carrying value of assets
Loans held-for-sale, net	7,436	78,486
MSR, net	12,804	6,948
Measured on a non-recurring basis | Level 3 | Carrying Value
Carrying value of assets
Impaired loans	271,408	282,723
MSR, net	12,800	6,947
REO, net	27,101	20,796
Total	312,100	314,347
Measured on a non-recurring basis | Level 3 | Carrying Value | Non-performing loans held-for-sale, net
Carrying value of assets
Loans held-for-sale, net	$ 791	$ 3,881