Fair Value - Financial Assets and Liabilities (Details) - USD ($) $ in Thousands	Sep. 30, 2023	Dec. 31, 2022
Financial Assets:
Available-for-sale securities	$ 102,076	$ 113,386
Domestic servicing rights	18,188	17,790
Derivative assets	133,016	108,621
VIE assets	70,293,689	79,043,129
Liabilities, Fair Value Disclosure [Abstract]
Derivative liabilities	85,657	91,404
VIE liabilities	63,200,233	71,844,422
Primary Beneficiary
Financial Assets:
Woodstar Fund investments	1,979,184	1,761,002
VIE assets	74,800
Liabilities, Fair Value Disclosure [Abstract]
VIE liabilities	34,800
Fair value measurements on recurring basis
Financial Assets:
Equity security	8,829	9,840
Derivative assets	133,016	108,621
Total	49,531,202	57,267,382
Liabilities, Fair Value Disclosure [Abstract]
Derivative liabilities	85,657	91,404
Total	43,082,761	50,845,759
Fair value measurements on recurring basis | Loans under fair value option
Financial Assets:
Loans under fair value option	2,602,265	2,784,594
Fair value measurements on recurring basis | RMBS
Financial Assets:
Available-for-sale securities	102,076	113,386
Fair value measurements on recurring basis | CMBS
Financial Assets:
Available-for-sale securities	18,740	19,108
Fair value measurements on recurring basis | Woodstar Fund investments
Financial Assets:
Woodstar Fund investments	1,979,184	1,761,002
Fair value measurements on recurring basis | Domestic servicing rights
Financial Assets:
Domestic servicing rights	18,188	17,790
Fair value measurements on recurring basis | VIE assets | Primary Beneficiary
Financial Assets:
VIE assets	44,668,904	52,453,041
Fair value measurements on recurring basis | VIE liabilities | Primary Beneficiary
Liabilities, Fair Value Disclosure [Abstract]
VIE liabilities	42,997,104	50,754,355
Fair value measurements on recurring basis | Level I
Financial Assets:
Equity security	8,829	9,840
Derivative assets	0	0
Total	8,829	9,840
Liabilities, Fair Value Disclosure [Abstract]
Derivative liabilities	0	0
Total	0	0
Fair value measurements on recurring basis | Level I | Loans under fair value option
Financial Assets:
Loans under fair value option	0	0
Fair value measurements on recurring basis | Level I | RMBS
Financial Assets:
Available-for-sale securities	0	0
Fair value measurements on recurring basis | Level I | CMBS
Financial Assets:
Available-for-sale securities	0	0
Fair value measurements on recurring basis | Level I | Woodstar Fund investments
Financial Assets:
Woodstar Fund investments	0	0
Fair value measurements on recurring basis | Level I | Domestic servicing rights
Financial Assets:
Domestic servicing rights	0	0
Fair value measurements on recurring basis | Level I | VIE assets | Primary Beneficiary
Financial Assets:
VIE assets	0	0
Fair value measurements on recurring basis | Level I | VIE liabilities | Primary Beneficiary
Liabilities, Fair Value Disclosure [Abstract]
VIE liabilities	0	0
Fair value measurements on recurring basis | Level II
Financial Assets:
Equity security	0	0
Derivative assets	133,016	108,621
Total	133,016	108,621
Liabilities, Fair Value Disclosure [Abstract]
Derivative liabilities	85,657	91,404
Total	37,714,357	45,339,816
Fair value measurements on recurring basis | Level II | Loans under fair value option
Financial Assets:
Loans under fair value option	0	0
Fair value measurements on recurring basis | Level II | RMBS
Financial Assets:
Available-for-sale securities	0	0
Fair value measurements on recurring basis | Level II | CMBS
Financial Assets:
Available-for-sale securities	0	0
Fair value measurements on recurring basis | Level II | Woodstar Fund investments
Financial Assets:
Woodstar Fund investments	0	0
Fair value measurements on recurring basis | Level II | Domestic servicing rights
Financial Assets:
Domestic servicing rights	0	0
Fair value measurements on recurring basis | Level II | VIE assets | Primary Beneficiary
Financial Assets:
VIE assets	0	0
Fair value measurements on recurring basis | Level II | VIE liabilities | Primary Beneficiary
Liabilities, Fair Value Disclosure [Abstract]
VIE liabilities	37,628,700	45,248,412
Fair value measurements on recurring basis | Level III
Financial Assets:
Equity security	0	0
Derivative assets	0	0
Total	49,389,357	57,148,921
Liabilities, Fair Value Disclosure [Abstract]
Derivative liabilities	0	0
Total	5,368,404	5,505,943
Fair value measurements on recurring basis | Level III | Loans under fair value option
Financial Assets:
Loans under fair value option	2,602,265	2,784,594
Fair value measurements on recurring basis | Level III | RMBS
Financial Assets:
Available-for-sale securities	102,076	113,386
Fair value measurements on recurring basis | Level III | CMBS
Financial Assets:
Available-for-sale securities	18,740	19,108
Fair value measurements on recurring basis | Level III | Woodstar Fund investments
Financial Assets:
Woodstar Fund investments	1,979,184	1,761,002
Fair value measurements on recurring basis | Level III | Domestic servicing rights
Financial Assets:
Domestic servicing rights	18,188	17,790
Fair value measurements on recurring basis | Level III | VIE assets | Primary Beneficiary
Financial Assets:
VIE assets	44,668,904	52,453,041
Fair value measurements on recurring basis | Level III | VIE liabilities | Primary Beneficiary
Liabilities, Fair Value Disclosure [Abstract]
VIE liabilities	$ 5,368,404	$ 5,505,943