Residential Whole Loans and Bridge Loans - Investment Portfolio (Details) $ in Thousands	6 Months Ended	12 Months Ended
	Jun. 30, 2022 USD ($) loan	Dec. 31, 2021 USD ($) loan
2.01% – 3.00%
Variable Interest Entity [Line Items]
Coupon  Rate	2.90%	2.80%
Number of Loans | loan	40	27
Principal  Balance | $	$ 22,650	$ 15,640
Original LTV	66.30%	65.10%
Original FICO score	758	757
Expected  Life (years)	9 years	5 years 3 months 18 days
Contractual  Maturity  (years)	28 years 9 months 18 days	28 years 9 months 18 days
2.01% – 3.00% | Minimum
Variable Interest Entity [Line Items]
Coupon  Rate	2.01%	2.01%
2.01% – 3.00% | Maximum
Variable Interest Entity [Line Items]
Coupon  Rate	3.00%	3.00%
3.01% – 4.00%
Variable Interest Entity [Line Items]
Coupon  Rate	3.70%	3.70%
Number of Loans | loan	484	496
Principal  Balance | $	$ 247,017	$ 244,022
Original LTV	65.00%	63.70%
Original FICO score	757	756
Expected  Life (years)	6 years 2 months 12 days	3 years 3 months 18 days
Contractual  Maturity  (years)	28 years 2 months 12 days	28 years
3.01% – 4.00% | Minimum
Variable Interest Entity [Line Items]
Coupon  Rate	3.01%	3.01%
3.01% – 4.00% | Maximum
Variable Interest Entity [Line Items]
Coupon  Rate	4.00%	400.00%
4.01% – 5.00%
Variable Interest Entity [Line Items]
Coupon  Rate	4.60%	4.70%
Number of Loans | loan	1,451	1,051
Principal  Balance | $	$ 498,639	$ 413,451
Original LTV	63.60%	65.10%
Original FICO score	749	747
Expected  Life (years)	4 years 9 months 18 days	2 years 10 months 24 days
Contractual  Maturity  (years)	26 years 4 months 24 days	28 years 2 months 12 days
4.01% – 5.00% | Minimum
Variable Interest Entity [Line Items]
Coupon  Rate	4.01%	4.01%
4.01% – 5.00% | Maximum
Variable Interest Entity [Line Items]
Coupon  Rate	5.00%	5.00%
5.01% – 6.00%
Variable Interest Entity [Line Items]
Coupon  Rate	5.50%	5.40%
Number of Loans | loan	895	757
Principal  Balance | $	$ 366,805	$ 305,344
Original LTV	66.20%	64.90%
Original FICO score	742	738
Expected  Life (years)	4 years	3 years
Contractual  Maturity  (years)	27 years 6 months	26 years 9 months 18 days
5.01% – 6.00% | Minimum
Variable Interest Entity [Line Items]
Coupon  Rate	5.01%	5.01%
5.01% – 6.00% | Maximum
Variable Interest Entity [Line Items]
Coupon  Rate	6.00%	6.00%
6.01% – 7.00%
Variable Interest Entity [Line Items]
Coupon  Rate	6.40%	6.30%
Number of Loans | loan	216	28
Principal  Balance | $	$ 98,409	$ 10,181
Original LTV	71.70%	67.90%
Original FICO score	742	721
Expected  Life (years)	3 years 1 month 6 days	3 years 1 month 6 days
Contractual  Maturity  (years)	29 years 4 months 24 days	25 years 9 months 18 days
6.01% – 7.00% | Minimum
Variable Interest Entity [Line Items]
Coupon  Rate	6.01%	6.01%
6.01% – 7.00% | Maximum
Variable Interest Entity [Line Items]
Coupon  Rate	7.00%	7.00%
7.01% - 8.00%
Variable Interest Entity [Line Items]
Coupon  Rate	7.40%	7.10%
Number of Loans | loan	16	2
Principal  Balance | $	$ 6,450	$ 505
Original LTV	75.10%	73.20%
Original FICO score	737	753
Expected  Life (years)	2 years 8 months 12 days	4 years 6 months
Contractual  Maturity  (years)	29 years 7 months 6 days	26 years 9 months 18 days
7.01% - 8.00% | Minimum
Variable Interest Entity [Line Items]
Coupon  Rate	7.01%	7.01%
7.01% - 8.00% | Maximum
Variable Interest Entity [Line Items]
Coupon  Rate	8.00%	8.00%
Residential  Whole Loans
Variable Interest Entity [Line Items]
Coupon  Rate	4.80%	4.60%
Number of Loans | loan	3,102	2,361
Principal  Balance | $	$ 1,239,970	$ 989,143
Original LTV	65.40%	64.80%
Original FICO score	748	746
Expected  Life (years)	4 years 9 months 18 days	3 years 1 month 6 days
Contractual  Maturity  (years)	27 years 4 months 24 days	27 years 8 months 12 days
Residential portfolio segment with no FICO score
Variable Interest Entity [Line Items]
Number of Loans | loan	250	230
Principal  Balance | $	$ 83,200	$ 74,300
7.01% – 9.00%
Variable Interest Entity [Line Items]
Coupon  Rate	8.80%	8.80%
Number of Loans | loan	3	3
Principal  Balance | $	$ 2,946	$ 2,946
Original LTV	70.40%	70.40%
Contractual  Maturity  (years)	0 years	0 years
7.01% – 9.00% | Minimum
Variable Interest Entity [Line Items]
Coupon  Rate	7.01%	7.01%
7.01% – 9.00% | Maximum
Variable Interest Entity [Line Items]
Coupon  Rate	9.00%	9.00%
9.01% – 11.00%
Variable Interest Entity [Line Items]
Coupon  Rate	10.40%	10.40%
Number of Loans | loan	2	4
Principal  Balance | $	$ 2,144	$ 2,393
Original LTV	78.10%	76.70%
Contractual  Maturity  (years)	0 years	0 years
9.01% – 11.00% | Minimum
Variable Interest Entity [Line Items]
Coupon  Rate	9.01%	9.01%
9.01% – 11.00% | Maximum
Variable Interest Entity [Line Items]
Coupon  Rate	11.00%	11.00%
11.01% – 13.00%
Variable Interest Entity [Line Items]
Coupon  Rate	11.40%	11.40%
Number of Loans | loan	2	2
Principal  Balance | $	$ 495	$ 495
Original LTV	69.70%	69.70%
Contractual  Maturity  (years)	0 years	0 years
11.01% – 13.00% | Minimum
Variable Interest Entity [Line Items]
Coupon  Rate	11.01%	11.01%
11.01% – 13.00% | Maximum
Variable Interest Entity [Line Items]
Coupon  Rate	13.00%	13.00%
Residential Bridge Loans | VIE
Variable Interest Entity [Line Items]
Coupon  Rate	9.70%	9.70%
Number of Loans | loan	7	9
Principal  Balance | $	$ 5,585	$ 5,834
Original LTV	73.30%	72.90%
Contractual  Maturity  (years)	0 years	0 years