Schedule I - Financial Information (Balance Sheets) (Details) - USD ($) $ in Thousands	Dec. 31, 2016	Dec. 31, 2015	Dec. 31, 2014	Dec. 31, 2013
ASSETS
Cash and cash equivalents	$ 224,598	$ 202,828	$ 320,175	$ 491,885
Restricted cash and cash equivalents	204,463	708,099
Residential loans at amortized cost, net	665,209	541,406
Receivables, net	267,962	137,190
Premises and equipment, net	82,628	106,481
Deferred tax assets, net	299,926	108,050
Other assets	242,290	200,364
Due from affiliates, net	0	0
Investments in consolidated subsidiaries and VIEs	0	0
Total assets	16,758,896	18,549,447
LIABILITIES AND STOCKHOLDERS' EQUITY
Payables and accrued liabilities	759,011	597,926
Corporate debt	2,129,000	2,157,424
Deferred tax liability, net		0
Total liabilities	16,478,636	17,744,771
Stockholders' equity:
Preferred stock	0	0
Common stock	364	355
Additional paid-in capital	596,067	591,454
Retained earnings (accumulated deficit)	(317,104)	212,054
Accumulated other comprehensive income	933	813
Total stockholders' equity	280,260	804,676	1,076,659	1,167,016
Total liabilities and stockholders' equity	16,758,896	18,549,447
Reportable Legal Entities [Member] | Parent Company [Member]
ASSETS
Cash and cash equivalents	773	4,016	$ 3,162	$ 100,009
Restricted cash and cash equivalents	1,502	10,512
Residential loans at amortized cost, net	12,891	14,130
Receivables, net	97,424	11,465
Premises and equipment, net	1,181	1,559
Deferred tax assets, net	27,310	0
Other assets	30,789	37,724
Due from affiliates, net	392,998	674,139
Investments in consolidated subsidiaries and VIEs	1,897,729	2,278,009
Total assets	2,462,597	3,031,554
LIABILITIES AND STOCKHOLDERS' EQUITY
Payables and accrued liabilities	53,337	43,778
Corporate debt	2,129,000	2,156,944
Deferred tax liability, net	0	26,156
Total liabilities	2,182,337	2,226,878
Stockholders' equity:
Preferred stock	0	0
Common stock	364	355
Additional paid-in capital	596,067	591,454
Retained earnings (accumulated deficit)	(317,104)	212,054
Accumulated other comprehensive income	933	813
Total stockholders' equity	280,260	804,676
Total liabilities and stockholders' equity	$ 2,462,597	$ 3,031,554