Segment Information - Components of Corporate/Other (Details) - USD ($) $ in Thousands	12 Months Ended
	Dec. 31, 2020	Dec. 31, 2019	Dec. 31, 2018
Segment Reporting Information [Line Items]
Interest income	$ 571,916	$ 622,281	$ 378,717
Interest expense	(448,005)	(479,808)	(239,039)
Net Interest Income	123,911	142,473	139,678
Investment fair value changes, net	(588,438)	35,500	(25,689)
Other income	4,337	1,799	0
Realized gains, net	30,424	23,821	27,041
Total non-interest income, net	(475,354)	165,844	73,988
General and administrative expenses	(115,204)	(108,737)	(75,298)
Loan acquisition costs	(11,023)	(9,935)	(7,484)
Other expenses	(108,785)	(13,022)	(196)
Net (Loss) Income	(581,847)	169,183	119,600
Legacy Consolidated VIEs
Segment Reporting Information [Line Items]
Other expenses	0	0	0
Other
Segment Reporting Information [Line Items]
Other expenses	(718)	(3,395)	(178)
Corporate/ Other
Segment Reporting Information [Line Items]
Interest income	9,136	17,649	20,036
Interest expense	(47,620)	(63,145)	(60,964)
Net Interest Income	(38,484)	(45,496)	(40,928)
Investment fair value changes, net	(2,004)	(1,617)	(996)
Other income	2,685	2,711	618
Realized gains, net	25,182	0	0
Total non-interest income, net	25,863	1,094	(378)
General and administrative expenses	(52,900)	(52,868)	(44,313)
Loan acquisition costs	(10)	(137)	(9)
Other expenses	(718)	(3,395)	(178)
Net (Loss) Income	(66,249)	(100,802)	(85,806)
Corporate/ Other | Legacy Consolidated VIEs
Segment Reporting Information [Line Items]
Interest income	9,061	17,649	20,036
Interest expense	(5,945)	(14,418)	(16,519)
Net Interest Income	3,116	3,231	3,517
Investment fair value changes, net	(1,512)	(1,545)	(1,016)
Other income	0	0	0
Realized gains, net	0	0	0
Total non-interest income, net	(1,512)	(1,545)	(1,016)
General and administrative expenses	0	0	0
Loan acquisition costs	0	0	0
Net (Loss) Income	1,604	1,686	2,501
Corporate/ Other | Other
Segment Reporting Information [Line Items]
Interest income	75	0	0
Interest expense	(41,675)	(48,727)	(44,445)
Net Interest Income	(41,600)	(48,727)	(44,445)
Investment fair value changes, net	(492)	(72)	20
Other income	2,685	2,711	618
Realized gains, net	25,182	0	0
Total non-interest income, net	27,375	2,639	638
General and administrative expenses	(52,900)	(52,868)	(44,313)
Loan acquisition costs	(10)	(137)	(9)
Net (Loss) Income	$ (67,853)	$ (102,488)	$ (88,307)