Schedule I - Financial Information (Balance Sheets) (Details) (USD $) In Thousands, unless otherwise specified	Dec. 31, 2014	Dec. 31, 2013	Dec. 31, 2012	Dec. 31, 2011
ASSETS
Cash and cash equivalents	$ 320,175	$ 491,885	$ 442,054	$ 18,739
Restricted cash and cash equivalents	733,015	804,803
Residential loans at amortized cost, net	1,314,539	1,394,871
Receivables, net	215,629	319,195
Premises and equipment, net	124,926	155,847
Other assets	280,794	413,076
Due from affiliates, net	0	0
Investments in consolidated subsidiaries and variable interest entities	0	0
Total assets	18,991,977	17,387,529
LIABILITIES AND STOCKHOLDERS' EQUITY
Payables and accrued liabilities	663,829	494,139
Corporate debt	2,267,799	2,272,085
Deferred tax liability, net	86,617	121,607
Total liabilities	17,915,318	16,220,513
Stockholders' equity:
Preferred stock, $0.01 par value per share: Authorized - 10,000,000 shares; Issued and outstanding - 0 shares at December 31, 2014 and 2013	0	0
Common stock , $0.01 par value per share: Authorized - 90,000,000 shares; Issued and outstanding - 37,711,623 and 37,377,274 shares at December 31, 2014 and 2013, respectively	377	374
Additional paid-in capital	600,643	580,572
Retained earnings	475,244	585,572
Accumulated other comprehensive income	395	498
Total stockholders' equity	1,076,659	1,167,016	894,928	533,532
Total liabilities and stockholders' equity	18,991,977	17,387,529
Parent Company [Member]
ASSETS
Cash and cash equivalents	3,162	100,009	366,393	550
Restricted cash and cash equivalents	10,507	14,753
Residential loans at amortized cost, net	13,302	6,341
Receivables, net	16,363	51,681
Premises and equipment, net	157	277
Other assets	69,780	65,293
Due from affiliates, net	697,979	711,797
Investments in consolidated subsidiaries and variable interest entities	2,641,527	2,621,934
Total assets	3,452,777	3,572,085
LIABILITIES AND STOCKHOLDERS' EQUITY
Payables and accrued liabilities	42,076	57,187
Corporate debt	2,265,668	2,267,979
Deferred tax liability, net	68,374	79,903
Total liabilities	2,376,118	2,405,069
Stockholders' equity:
Preferred stock, $0.01 par value per share: Authorized - 10,000,000 shares; Issued and outstanding - 0 shares at December 31, 2014 and 2013	0	0
Common stock , $0.01 par value per share: Authorized - 90,000,000 shares; Issued and outstanding - 37,711,623 and 37,377,274 shares at December 31, 2014 and 2013, respectively	377	374
Additional paid-in capital	600,643	580,572
Retained earnings	475,244	585,572
Accumulated other comprehensive income	395	498
Total stockholders' equity	1,076,659	1,167,016
Total liabilities and stockholders' equity	$ 3,452,777	$ 3,572,085