Mortgage-Backed and Credit Risk Transfer Securities - Schedule of Fair Value of Available-for-sale Securities and Securities Accounted for under Fair Value Option by Asset Type (Details) - USD ($) $ in Thousands	Dec. 31, 2019	Dec. 31, 2018
Debt Securities, Available-for-sale [Line Items]
Marketable Securities	$ 21,771,786	$ 17,396,642
Agency RMBS
Debt Securities, Available-for-sale [Line Items]
Available-for-sale Securities	884,778	1,924,363
Securities accounted for under fair value option	9,988,749	8,932,608
Marketable Securities	10,873,527	10,856,971
15 year fixed-rate
Debt Securities, Available-for-sale [Line Items]
Available-for-sale Securities	98,666	204,347
Securities accounted for under fair value option	193,748	219,907
Marketable Securities	292,414	424,254
30 year fixed-rate
Debt Securities, Available-for-sale [Line Items]
Available-for-sale Securities	754,590	1,093,070
Securities accounted for under fair value option	9,769,630	8,679,699
Marketable Securities	10,524,220	9,772,769
ARM
Debt Securities, Available-for-sale [Line Items]
Available-for-sale Securities		105,747
Securities accounted for under fair value option		0
Marketable Securities		105,747
Hybrid ARM
Debt Securities, Available-for-sale [Line Items]
Available-for-sale Securities	31,522	521,199
Securities accounted for under fair value option	25,371	33,002
Marketable Securities	56,893	554,201
Agency-CMO
Debt Securities, Available-for-sale [Line Items]
Available-for-sale Securities	146,733	168,385
Securities accounted for under fair value option	280,779	99,306
Marketable Securities	427,512	267,691
Agency CMBS
Debt Securities, Available-for-sale [Line Items]
Available-for-sale Securities	0	0
Securities accounted for under fair value option	4,767,930	1,002,510
Marketable Securities	4,767,930	1,002,510
Non-Agency CMBS
Debt Securities, Available-for-sale [Line Items]
Available-for-sale Securities	2,150,991	2,153,403
Securities accounted for under fair value option	1,672,483	1,133,056
Marketable Securities	3,823,474	3,286,459
Non-Agency RMBS
Debt Securities, Available-for-sale [Line Items]
Available-for-sale Securities	715,479	961,445
Securities accounted for under fair value option	240,192	202,237
Marketable Securities	955,671	1,163,682
GSE CRT
Debt Securities, Available-for-sale [Line Items]
Available-for-sale Securities	507,445	586,231
Securities accounted for under fair value option	416,227	233,098
Marketable Securities	$ 923,672	$ 819,329
Mortgage-backed and credit risk transfer securities
Debt Securities, Available-for-sale [Line Items]
Percentage of MBS and GSE CRT accounted for under the fair value option	80.00%	67.00%
Available-for-sale Securities	$ 4,405,426	$ 5,793,827
Securities accounted for under fair value option	17,366,360	11,602,815
Marketable Securities	$ 21,719,190	$ 17,322,201