Investments in Servicing Related Assets, Summary (Details) - USD ($) $ in Thousands		6 Months Ended	12 Months Ended
		Jun. 30, 2019	Dec. 31, 2018
Servicing Asset [Abstract]
Unpaid principal balance		$ 27,803,138	$ 24,846,989
Cost basis		344,916	298,480
Carrying value	[1]	$ 273,699	$ 294,907
Weighted average coupon		4.28%	4.23%
Weighted average maturity	[2]	27 years	27 years 2 months 12 days
Changes in fair value recorded in other income (loss)		$ (71,217)	$ (3,573)
Mortgage Service Right Conventional [Member]
Servicing Asset [Abstract]
Unpaid principal balance		24,564,754	21,366,980
Cost basis	[3]	304,700	258,070
Carrying value	[1]	$ 239,623	$ 254,691
Weighted average coupon		4.40%	4.37%
Weighted average maturity	[2]	27 years 1 month 6 days	27 years 3 months 18 days
Changes in fair value recorded in other income (loss)		$ (65,077)	$ (3,379)
Mortgage Service Right Government [Member]
Servicing Asset [Abstract]
Unpaid principal balance		3,238,384	3,480,009
Cost basis	[3]	40,216	40,410
Carrying value	[1]	$ 34,076	$ 40,216
Weighted average coupon		3.37%	3.37%
Weighted average maturity	[2]	26 years 3 months 18 days	26 years 9 months 18 days
Changes in fair value recorded in other income (loss)		$ (6,140)	$ (194)

[1] Carrying value represents the fair value of the pools (see Note 9).

[2] The weighted average maturity represents the weighted average expected timing of the receipt of cash flows of each investment.

[3] MSR cost basis consists of the carrying value of the prior period, adjusted for any purchases, sales and principal paydowns of the underlying mortgage loans.