Loans and Allowance for Credit Losses - Delinquency (Details) - USD ($) $ in Thousands	12 Months Ended
	Dec. 31, 2023	Dec. 31, 2022	Dec. 31, 2021
Loan delinquency information
Total Loans Carrying Value	$ 10,675,958	$ 9,941,207
Non-Accrual Loans	634,001	241,499
90+ Days Past Due but Accruing	82,821
General allowance for loan losses	(44,473)	(57,710)
Total Loans, net of allowance for loan losses, including loans in consolidated VIEs	$ 10,631,485	$ 9,883,497
Percentage of loans outstanding	100.00%	100.00%
Percentage of outstanding, Non-Accrual Loans	5.90%	2.40%
Specific allowance for loan losses	$ 101,605	$ 90,519	$ 45,317
Current and less than 30 days past due
Loan delinquency information
Total Loans Carrying Value	$ 9,587,845	$ 9,593,934
Percentage of loans outstanding	89.80%	96.50%
30-59 Days Past Due
Loan delinquency information
Total Loans Carrying Value	$ 171,498	$ 110,312
Percentage of loans outstanding	1.60%	1.10%
60+ Days Past Due
Loan delinquency information
Total Loans Carrying Value	$ 916,615	$ 236,961
Percentage of loans outstanding	8.60%	2.40%
90+ Days Past Due
Loan delinquency information
Percentage of outstanding, 90+Days Past Due Accruing	0.80%	0.00%
Bridge
Loan delinquency information
Total Loans Carrying Value	$ 6,796,082	$ 7,327,946
Non-Accrual Loans	339,073	113,360
General allowance for loan losses	(17,302)	(42,979)
Specific allowance for loan losses	36,241	49,905	19,519
Bridge | Current and less than 30 days past due
Loan delinquency information
Total Loans Carrying Value	6,186,367	7,120,162
Bridge | 30-59 Days Past Due
Loan delinquency information
Total Loans Carrying Value	87,163	94,823
Bridge | 60+ Days Past Due
Loan delinquency information
Total Loans Carrying Value	522,552	112,961
Fixed rate
Loan delinquency information
Total Loans Carrying Value	1,031,830	1,034,626
Non-Accrual Loans	13,928	28,719
General allowance for loan losses	(7,884)	(2,397)
Specific allowance for loan losses	13,598	6,531	6,861
Fixed rate | Current and less than 30 days past due
Loan delinquency information
Total Loans Carrying Value	986,755	993,832
Fixed rate | 30-59 Days Past Due
Loan delinquency information
Total Loans Carrying Value	21,798	8,101
Fixed rate | 60+ Days Past Due
Loan delinquency information
Total Loans Carrying Value	23,277	32,693
Construction
Loan delinquency information
Total Loans Carrying Value	1,180,635	428,805
Non-Accrual Loans	241,751	55,993
90+ Days Past Due but Accruing	82,781
General allowance for loan losses	(3,722)	(325)
Specific allowance for loan losses	30,870	17,334
Construction | Current and less than 30 days past due
Loan delinquency information
Total Loans Carrying Value	782,123	372,812
Construction | 30-59 Days Past Due
Loan delinquency information
Total Loans Carrying Value	49,694
Construction | 60+ Days Past Due
Loan delinquency information
Total Loans Carrying Value	348,818	55,993
Freddie Mac
Loan delinquency information
Total Loans Carrying Value	9,500	10,040
Non-Accrual Loans	2,695	3,093
Freddie Mac | Current and less than 30 days past due
Loan delinquency information
Total Loans Carrying Value	9,500	6,947
Freddie Mac | 60+ Days Past Due
Loan delinquency information
Total Loans Carrying Value		3,093
SBA 7(a)
Loan delinquency information
Total Loans Carrying Value	1,204,678	552,260
Non-Accrual Loans	30,549	12,790
90+ Days Past Due but Accruing	40
General allowance for loan losses	(12,679)	(10,801)
Specific allowance for loan losses	17,867	14,299	12,180
SBA 7(a) | Current and less than 30 days past due
Loan delinquency information
Total Loans Carrying Value	1,179,231	541,378
SBA 7(a) | 30-59 Days Past Due
Loan delinquency information
Total Loans Carrying Value	8,619	6,690
SBA 7(a) | 60+ Days Past Due
Loan delinquency information
Total Loans Carrying Value	16,828	4,192
Other
Loan delinquency information
Total Loans Carrying Value	453,233	587,530
Non-Accrual Loans	6,005	27,544
General allowance for loan losses	(2,886)	(1,208)
Specific allowance for loan losses	3,029	2,450	$ 6,757
Other | Current and less than 30 days past due
Loan delinquency information
Total Loans Carrying Value	443,869	558,803
Other | 30-59 Days Past Due
Loan delinquency information
Total Loans Carrying Value	4,224	698
Other | 60+ Days Past Due
Loan delinquency information
Total Loans Carrying Value	$ 5,140	$ 28,029