Fair Value (Details 3) (USD $) In Thousands, unless otherwise specified	Jun. 30, 2014	Dec. 31, 2013
Financial assets not carried at fair value:
Securities, held-to-maturity	$ 370,096	$ 368,318
Non-performing residential loans		215,371
Financial liabilities not carried at fair value:
Convertible senior notes	1,003,847	997,851
Carrying Value
Financial assets not carried at fair value:
Loans held-for-investment and loans transferred as secured borrowings	5,028,721	4,544,132
Securities, held-to-maturity	370,096	368,318
European servicing rights	18,528	27,024
Non-performing residential loans		215,371
Financial liabilities not carried at fair value:
Secured financing agreements and secured borrowings on transferred loans	2,704,082	2,438,798
Convertible senior notes	1,003,847	997,851
Fair Value
Financial assets not carried at fair value:
Loans held-for-investment and loans transferred as secured borrowings	5,163,988	4,609,040
Securities, held-to-maturity	371,603	368,453
European servicing rights	18,528	29,327
Non-performing residential loans		215,371
Financial liabilities not carried at fair value:
Secured financing agreements and secured borrowings on transferred loans	2,705,452	2,436,708
Convertible senior notes	$ 1,190,700	$ 1,160,000