Derivative Financial Instruments and Hedging Activities - Summary of Changes in Notional Balances of Holdings of Derivatives (Details) - USD ($) $ in Thousands	12 Months Ended
	Dec. 31, 2019	Dec. 31, 2018
Fair value of derivative assets (liabilities) at:
Derivatives, at fair value	$ 6,007	$ 4,552
IRLCs
Derivative [Line Items]
Notional balance at December 31, 2019	232,566
Fair value of derivative assets (liabilities) at:
Derivatives, at fair value	4,878	3,871
Gain (loss) on derivatives, net	756	3,809
TBA / Forward MBS Trades
Derivative [Line Items]
Notional balance at December 31, 2019	1,200,000
Fair value of derivative assets (liabilities) at:
Derivatives, at fair value	1,121	0
Gain (loss) on derivatives, net	525	0
Forward Trades
Derivative [Line Items]
Notional balance at December 31, 2019	60,000
Fair value of derivative assets (liabilities) at:
Derivatives, at fair value	(92)	(4,983)
Gain (loss) on derivatives, net	(2,689)	136
Interest Rate Caps
Derivative [Line Items]
Notional balance at December 31, 2019	27,083
Fair value of derivative assets (liabilities) at:
Derivatives, at fair value	0	678
Gain (loss) on derivatives, net	$ (358)	$ (841)