Fair Value Measurements - Financial Instruments Measured at Fair Value on a Recurring Basis (Details) - USD ($) $ in Thousands	Sep. 30, 2022	Dec. 31, 2021
Financial assets:
Loans, at fair value	$ 29,375	$ 29,697
Derivative Asset, Subject to Master Netting Arrangement, before Offset	4,422	250
Financial liabilities:
Derivative Liability	139	228
Fair Value, Measurements, Recurring
Financial assets:
MSR financing receivables	164,585	125,018
Loans, at fair value	29,375	29,697
Derivative Asset, Subject to Master Netting Arrangement, before Offset	4,422	250
Other assets	5,600	12,655
Financial liabilities:
Derivative Liability	139	228
Fair Value, Measurements, Recurring | Fair Value, Inputs, Level 1
Financial assets:
MSR financing receivables	0	0
Loans, at fair value	0	0
Derivative Asset, Subject to Master Netting Arrangement, before Offset	0	20
Other assets	352	5,267
Financial liabilities:
Derivative Liability	0	0
Fair Value, Measurements, Recurring | Fair Value, Inputs, Level 2
Financial assets:
MSR financing receivables	0	0
Loans, at fair value	0	0
Derivative Asset, Subject to Master Netting Arrangement, before Offset	4,422	230
Other assets	0	0
Financial liabilities:
Derivative Liability	139	228
Fair Value, Measurements, Recurring | Fair Value, Inputs, Level 3
Financial assets:
MSR financing receivables	164,585	125,018
Loans, at fair value	29,375	29,697
Derivative Asset, Subject to Master Netting Arrangement, before Offset	0	0
Other assets	5,248	7,388
Financial liabilities:
Derivative Liability	0	0
Fair Value, Measurements, Recurring | Agency MBS
Financial assets:
Trading securities	445,122	483,927
Fair Value, Measurements, Recurring | Agency MBS | Fair Value, Inputs, Level 1
Financial assets:
Trading securities	0	0
Fair Value, Measurements, Recurring | Agency MBS | Fair Value, Inputs, Level 2
Financial assets:
Trading securities	445,122	483,927
Fair Value, Measurements, Recurring | Agency MBS | Fair Value, Inputs, Level 3
Financial assets:
Trading securities	0	0
Fair Value, Measurements, Recurring | Variable Interest Entity, Primary Beneficiary
Financial assets:
Loans, at fair value	203,456	7,442
Financial liabilities:
Secured debt of consolidated VIEs, at fair value	182,336	508
Fair Value, Measurements, Recurring | Variable Interest Entity, Primary Beneficiary | Fair Value, Inputs, Level 1
Financial assets:
Loans, at fair value	0	0
Financial liabilities:
Secured debt of consolidated VIEs, at fair value	0	0
Fair Value, Measurements, Recurring | Variable Interest Entity, Primary Beneficiary | Fair Value, Inputs, Level 2
Financial assets:
Loans, at fair value	0	0
Financial liabilities:
Secured debt of consolidated VIEs, at fair value	172,012	0
Fair Value, Measurements, Recurring | Variable Interest Entity, Primary Beneficiary | Fair Value, Inputs, Level 3
Financial assets:
Loans, at fair value	203,456	7,442
Financial liabilities:
Secured debt of consolidated VIEs, at fair value	10,324	508
Fair Value, Measurements, Recurring | Credit Securities
Financial assets:
Trading securities	115,783	26,222
Fair Value, Measurements, Recurring | Credit Securities | Fair Value, Inputs, Level 1
Financial assets:
Trading securities	0	0
Fair Value, Measurements, Recurring | Credit Securities | Fair Value, Inputs, Level 2
Financial assets:
Trading securities	108,330	0
Fair Value, Measurements, Recurring | Credit Securities | Fair Value, Inputs, Level 3
Financial assets:
Trading securities	$ 7,453	$ 26,222