Interest Income and Interest Expense (Details) - USD ($) $ in Thousands	3 Months Ended		9 Months Ended
	Sep. 30, 2020	Sep. 30, 2019	Sep. 30, 2020	Sep. 30, 2019
Interest Income Expense Net [Abstract]
Loans held for investment, net	$ 76,281	$ 95,436	$ 250,565	$ 284,541
Investment securities	31,966	54,945	121,630	182,127
Margin balances	5,803	13,334	22,298	40,437
Financial instruments owned	2,924	5,668	10,357	18,395
Other	(2,563)	9,401	(894)	32,295
Total interest income	114,411	178,784	403,956	557,795
Bank deposits	1,478	24,815	13,324	82,228
Senior notes	15,411	11,121	39,677	33,365
Federal Home Loan Bank advances	515	1,146	3,550	5,735
Other	(3,582)	7,062	(5,288)	25,155
Total interest expense	$ 13,822	$ 44,144	$ 51,263	$ 146,483