Schedule of Derivative Instruments in Statement of Financial Position, Fair Value (Details) - USD ($) $ in Thousands	Jun. 30, 2016	Mar. 31, 2016	Dec. 31, 2015	Jun. 30, 2015	Mar. 31, 2015	Dec. 31, 2014
Notional Disclosures [Abstract]
Derivative, Notional Amount	$ 26,141,370	$ 26,390,596	$ 21,998,381	$ 27,401,335	$ 33,709,043	$ 48,115,523
Inverse Interest-Only Securities [Member]
Derivative, Fair Value, Net [Abstract]
Derivative, Fair Value, Net	153,512		159,582
Notional Disclosures [Abstract]
Derivative, Notional Amount	834,866	882,726	932,037	1,049,743	1,106,210	1,168,226
Interest Rate Swap [Member]
Notional Disclosures [Abstract]
Derivative, Notional Amount	13,697,000	15,425,513	14,268,806	16,225,523	19,929,000	18,584,000
Credit Default Swap [Member]
Notional Disclosures [Abstract]
Derivative, Notional Amount	25,000	125,000	125,000	125,000	125,000	125,000
Interest Rate Swaption [Member]
Notional Disclosures [Abstract]
Derivative, Notional Amount	1,800,000	5,200,000	5,200,000	9,410,000	12,960,000	12,410,000
Forward Contracts [Member]
Notional Disclosures [Abstract]
Derivative, Notional Amount	337,000	1,637,000	297,000	1,024,000	2,496,000	1,325,000
Options Held [Member]
Notional Disclosures [Abstract]
Derivative, Notional Amount	8,897,000	2,000,000	0	0	2,500,000	2,000,000
Swap [Member]
Notional Disclosures [Abstract]
Derivative, Notional Amount				0	3,000,000	14,000,000
Total Return Swap [Member]
Derivative, Fair Value, Net [Abstract]
Derivative, Fair Value, Net	(7,419)		1,645
Notional Disclosures [Abstract]
Derivative, Notional Amount	588,037	868,145	889,418	988,409	877,529	598,459
Loan Purchase Commitments [Member]
Notional Disclosures [Abstract]
Derivative, Notional Amount	636,467	$ 252,212	286,120	$ 626,660	$ 707,304	$ 554,838
Derivative Financial Instruments, Assets [Member]
Derivative, Fair Value, Net [Abstract]
Derivative, Fair Value, Net	223,985		271,509
Notional Disclosures [Abstract]
Derivative, Notional Amount	16,940,106		21,735,997
Derivative Financial Instruments, Assets [Member] | Inverse Interest-Only Securities [Member]
Notional Disclosures [Abstract]
Derivative, Notional Amount	834,866		932,037
Derivative Financial Instruments, Assets [Member] | Interest Rate Swap [Member]
Derivative, Fair Value, Net [Abstract]
Derivative, Fair Value, Net	23,465		91,757
Notional Disclosures [Abstract]
Derivative, Notional Amount	8,314,000		14,268,806
Derivative Financial Instruments, Assets [Member] | Credit Default Swap [Member]
Derivative, Fair Value, Net [Abstract]
Derivative, Fair Value, Net	0		0
Notional Disclosures [Abstract]
Derivative, Notional Amount	0		0
Derivative Financial Instruments, Assets [Member] | Interest Rate Swaption [Member]
Derivative, Fair Value, Net [Abstract]
Derivative, Fair Value, Net	27,559		17,374
Notional Disclosures [Abstract]
Derivative, Notional Amount	2,600,000		4,700,000
Derivative Financial Instruments, Assets [Member] | Forward Contracts [Member]
Derivative, Fair Value, Net [Abstract]
Derivative, Fair Value, Net	16,597		1,074
Notional Disclosures [Abstract]
Derivative, Notional Amount	1,835,000		847,000
Derivative Financial Instruments, Assets [Member] | Options Held [Member]
Derivative, Fair Value, Net [Abstract]
Derivative, Fair Value, Net	2,042		0
Notional Disclosures [Abstract]
Derivative, Notional Amount	2,835,000
Derivative Financial Instruments, Assets [Member] | Total Return Swap [Member]
Derivative, Fair Value, Net [Abstract]
Derivative, Fair Value, Net	0		1,645
Notional Disclosures [Abstract]
Derivative, Notional Amount	0		889,418
Derivative Financial Instruments, Assets [Member] | Loan Purchase Commitments [Member]
Derivative, Fair Value, Net [Abstract]
Derivative, Fair Value, Net	810		77
Notional Disclosures [Abstract]
Derivative, Notional Amount	521,240		98,736
Derivative Financial Instruments, Liabilities [Member]
Derivative, Fair Value, Net [Abstract]
Derivative, Fair Value, Net	(160,724)		(7,285)
Notional Disclosures [Abstract]
Derivative, Notional Amount	15,145,264		1,362,384
Derivative Financial Instruments, Liabilities [Member] | Inverse Interest-Only Securities [Member]
Derivative, Fair Value, Net [Abstract]
Derivative, Fair Value, Net	0		0
Notional Disclosures [Abstract]
Derivative, Notional Amount	0		0
Derivative Financial Instruments, Liabilities [Member] | Interest Rate Swap [Member]
Derivative, Fair Value, Net [Abstract]
Derivative, Fair Value, Net	(99,366)		0
Notional Disclosures [Abstract]
Derivative, Notional Amount	5,383,000		0
Derivative Financial Instruments, Liabilities [Member] | Credit Default Swap [Member]
Derivative, Fair Value, Net [Abstract]
Derivative, Fair Value, Net	(99)		(703)
Notional Disclosures [Abstract]
Derivative, Notional Amount	25,000		125,000
Derivative Financial Instruments, Liabilities [Member] | Interest Rate Swaption [Member]
Derivative, Fair Value, Net [Abstract]
Derivative, Fair Value, Net	(10,499)		(4,831)
Notional Disclosures [Abstract]
Derivative, Notional Amount	800,000		500,000
Derivative Financial Instruments, Liabilities [Member] | Forward Contracts [Member]
Derivative, Fair Value, Net [Abstract]
Derivative, Fair Value, Net	(21,707)		(1,324)
Notional Disclosures [Abstract]
Derivative, Notional Amount	2,172,000		550,000
Derivative Financial Instruments, Liabilities [Member] | Options Held [Member]
Derivative, Fair Value, Net [Abstract]
Derivative, Fair Value, Net	(21,564)		0
Notional Disclosures [Abstract]
Derivative, Notional Amount	6,062,000
Derivative Financial Instruments, Liabilities [Member] | Total Return Swap [Member]
Derivative, Fair Value, Net [Abstract]
Derivative, Fair Value, Net	(7,419)		0
Notional Disclosures [Abstract]
Derivative, Notional Amount	588,037		0
Derivative Financial Instruments, Liabilities [Member] | Loan Purchase Commitments [Member]
Derivative, Fair Value, Net [Abstract]
Derivative, Fair Value, Net	(70)		(427)
Notional Disclosures [Abstract]
Derivative, Notional Amount	$ 115,227		$ 187,384