Loans and Allowance for Credit Losses - Delinquency (Details) - USD ($) $ in Thousands	6 Months Ended	12 Months Ended
	Jun. 30, 2024	Dec. 31, 2023	Mar. 31, 2024	Jun. 30, 2023	Mar. 31, 2023	Dec. 31, 2022
Loan delinquency information
Total Loans Carrying Value	$ 9,482,906	$ 10,675,958
Non-Accrual Loans	192,391	634,001
90+ Days Past Due but Accruing	120,676	82,821
General allowance for loan losses	(36,618)	(44,473)
Total Loans, net of allowance for loan losses, including loans in consolidated VIEs	$ 9,446,288	$ 10,631,485
Percentage of loans outstanding	100.00%	100.00%
Percentage of outstanding, Non-Accrual Loans	2.00%	5.90%
Percentage of outstanding, 90+Days Past Due Accruing	1.30%	0.80%
Specific allowance for loan losses	$ 44,832	$ 101,605	$ 67,423	$ 112,154	$ 67,780	$ 90,519
Current and less than 30 days past due
Loan delinquency information
Total Loans Carrying Value	$ 9,070,814	$ 9,587,845
Percentage of loans outstanding	95.60%	89.80%
30-59 Days Past Due
Loan delinquency information
Total Loans Carrying Value	$ 45,908	$ 171,498
Percentage of loans outstanding	0.50%	1.60%
60+ Days Past Due
Loan delinquency information
Total Loans Carrying Value	$ 366,184	$ 916,615
Percentage of loans outstanding	3.90%	8.60%
Bridge
Loan delinquency information
Total Loans Carrying Value	$ 6,069,908	$ 6,796,082
Non-Accrual Loans	103,996	339,073
90+ Days Past Due but Accruing	119,009
General allowance for loan losses	(12,732)	(17,302)
Specific allowance for loan losses	14,398	36,241	13,181	43,857	40,319	49,905
Bridge | Current and less than 30 days past due
Loan delinquency information
Total Loans Carrying Value	5,752,441	6,186,367
Bridge | 30-59 Days Past Due
Loan delinquency information
Total Loans Carrying Value	25,074	87,163
Bridge | 60+ Days Past Due
Loan delinquency information
Total Loans Carrying Value	292,393	522,552
Fixed rate
Loan delinquency information
Total Loans Carrying Value	959,022	1,031,830
Non-Accrual Loans	22,511	13,928
General allowance for loan losses	(5,415)	(7,884)
Specific allowance for loan losses	6,480	13,598	7,264	12,204	9,085	6,531
Fixed rate | Current and less than 30 days past due
Loan delinquency information
Total Loans Carrying Value	933,012	986,755
Fixed rate | 30-59 Days Past Due
Loan delinquency information
Total Loans Carrying Value	3,500	21,798
Fixed rate | 60+ Days Past Due
Loan delinquency information
Total Loans Carrying Value	22,510	23,277
Construction
Loan delinquency information
Total Loans Carrying Value	839,625	1,180,635
Non-Accrual Loans	19,578	241,751
90+ Days Past Due but Accruing	1,667	82,781
General allowance for loan losses	(5,840)	(3,722)
Specific allowance for loan losses	7,910	30,870	23,755	35,925	373	17,334
Construction | Current and less than 30 days past due
Loan delinquency information
Total Loans Carrying Value	803,198	782,123
Construction | 30-59 Days Past Due
Loan delinquency information
Total Loans Carrying Value	15,182	49,694
Construction | 60+ Days Past Due
Loan delinquency information
Total Loans Carrying Value	21,245	348,818
Freddie Mac
Loan delinquency information
Total Loans Carrying Value		9,500
Non-Accrual Loans		2,695
Freddie Mac | Current and less than 30 days past due
Loan delinquency information
Total Loans Carrying Value		9,500
SBA 7(a)
Loan delinquency information
Total Loans Carrying Value	1,218,807	1,204,678
Non-Accrual Loans	43,484	30,549
90+ Days Past Due but Accruing		40
General allowance for loan losses	(10,204)	(12,679)
Specific allowance for loan losses	13,566	17,867	20,579	16,809	15,110	14,299
SBA 7(a) | Current and less than 30 days past due
Loan delinquency information
Total Loans Carrying Value	1,192,886	1,179,231
SBA 7(a) | 30-59 Days Past Due
Loan delinquency information
Total Loans Carrying Value	262	8,619
SBA 7(a) | 60+ Days Past Due
Loan delinquency information
Total Loans Carrying Value	25,659	16,828
Other
Loan delinquency information
Total Loans Carrying Value	395,544	453,233
Non-Accrual Loans	2,822	6,005
General allowance for loan losses	(2,427)	(2,886)
Specific allowance for loan losses	2,478	3,029	$ 2,644	$ 3,359	$ 2,893	$ 2,450
Other | Current and less than 30 days past due
Loan delinquency information
Total Loans Carrying Value	389,277	443,869
Other | 30-59 Days Past Due
Loan delinquency information
Total Loans Carrying Value	1,890	4,224
Other | 60+ Days Past Due
Loan delinquency information
Total Loans Carrying Value	$ 4,377	$ 5,140