Fair Value - Financial Instruments Not Carried at Fair Value (Details) - USD ($) $ in Thousands	Mar. 31, 2021	Dec. 31, 2020
Financial assets not carried at fair value:
HTM securities	$ 493,462	$ 544,280
Carrying Value
Financial assets not carried at fair value:
Loans held-for-investment, loans held-for-sale and loans transferred as secured borrowings	12,402,351	11,116,929
HTM securities	488,075	538,605
Financial liabilities not carried at fair value:
Secured financing agreements and secured borrowings on transferred loans	11,827,110	11,076,744
Unsecured senior notes	1,735,658	1,732,520
Fair Value
Financial assets not carried at fair value:
Loans held-for-investment, loans held-for-sale and loans transferred as secured borrowings	12,460,388	11,107,316
HTM securities	466,687	515,253
Financial liabilities not carried at fair value:
Secured financing agreements and secured borrowings on transferred loans	11,900,381	11,108,364
Unsecured senior notes	$ 1,803,224	$ 1,786,667