Fair Value of Financial Instruments-Level 3 Assets (Details) - Recurring basis - Level 3 - Residential whole loans - USD ($) $ in Thousands	3 Months Ended	9 Months Ended
	Sep. 30, 2015	Sep. 30, 2015
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Balance at beginning of period	$ 183,861	$ 143,472
Purchases and capitalized advances	357,258	410,981
Changes in fair value recorded in Net gain on residential whole loans held at fair value	2,362	3,872
Collection of principal, net of liquidation gains/losses	(7,152)	(15,716)
Transfer to REO	(4,792)	(11,072)
Balance at end of period	$ 531,537	$ 531,537