Derivative Financial Instruments - Aggregate Fair Value and Notional Amount of Derivative Financial Instruments (Details) - USD ($) $ in Thousands	Mar. 31, 2020	Dec. 31, 2019
Derivative [Line Items]
Fair Value	$ (23,897)	$ (127,723)
Notional Amount	9,206,372	13,589,356
TBAs
Derivative [Line Items]
Notional Amount	8,980,000	6,610,000
Interest rate futures
Derivative [Line Items]
Notional Amount		226,000
Derivative Liabilities
Derivative [Line Items]
Fair Value	(114,614)	(163,424)
Notional Amount	4,716,372	6,929,494
Derivative Liabilities | Interest rate swaps
Derivative [Line Items]
Fair Value	0	(97,235)
Notional Amount	0	2,314,300
Derivative Liabilities | Interest rate swaps | Cash Flow Hedging
Derivative [Line Items]
Fair Value	0	(51,530)
Notional Amount	0	139,500
Derivative Liabilities | TBAs
Derivative [Line Items]
Fair Value	(110,648)	(13,359)
Notional Amount	4,490,000	4,160,000
Derivative Liabilities | Interest rate futures
Derivative [Line Items]
Fair Value	0	(10)
Notional Amount	0	12,300
Derivative Liabilities | Loan purchase and interest rate lock commitments
Derivative [Line Items]
Fair Value	(3,966)	(1,290)
Notional Amount	226,372	303,394
Derivative Assets
Derivative [Line Items]
Fair Value	90,717	35,701
Notional Amount	4,490,000	6,659,862
Derivative Assets | Interest rate swaps
Derivative [Line Items]
Fair Value	0	17,095
Notional Amount	0	1,399,000
Derivative Assets | TBAs
Derivative [Line Items]
Fair Value	90,717	5,755
Notional Amount	4,490,000	2,445,000
Derivative Assets | Interest rate futures
Derivative [Line Items]
Fair Value	0	777
Notional Amount	0	213,700
Derivative Assets | Swaptions
Derivative [Line Items]
Fair Value	0	1,925
Notional Amount	0	1,065,000
Derivative Assets | Loan purchase and interest rate lock commitments
Derivative [Line Items]
Fair Value	0	10,149
Notional Amount	$ 0	$ 1,537,162