Fair Value of Financial Instruments - Quantitative Information about Significant Unobservable Inputs Used in Valuation of Level 3 Assets and Liabilities Measured at Fair Value (Details) $ in Thousands	12 Months Ended
	Dec. 31, 2022 USD ($) $ / loan	Dec. 31, 2021 USD ($)
Assets
ABS issued | $	$ 7,424,132	$ 8,843,147
Variable Interest Entity, Primary Beneficiary | Collateralized Financing Entities | HEI
Liabilities
Fair value of securities owned | $	13,000
ABS Issued Securities
Liabilities
ABS issued | $	$ 3,155,300
ABS Issued Securities | Prepayment rate (annual CPR) | Minimum
Liabilities
ABS issued, measurement inputs	0.05
ABS Issued Securities | Prepayment rate (annual CPR) | Maximum
Liabilities
ABS issued, measurement inputs	0.23
ABS Issued Securities | Prepayment rate (annual CPR) | Weighted Average
Liabilities
ABS issued, measurement inputs	0.10
ABS Issued Securities | Discount rate | Minimum
Liabilities
ABS issued, measurement inputs	0.04
ABS Issued Securities | Discount rate | Maximum
Liabilities
ABS issued, measurement inputs	0.18
ABS Issued Securities | Discount rate | Weighted Average
Liabilities
ABS issued, measurement inputs	0.07
ABS Issued Securities | Default rate | Minimum
Liabilities
ABS issued, measurement inputs	0
ABS Issued Securities | Default rate | Maximum
Liabilities
ABS issued, measurement inputs	0.14
ABS Issued Securities | Default rate | Weighted Average
Liabilities
ABS issued, measurement inputs	0.01
ABS Issued Securities | Loss severity | Minimum
Liabilities
ABS issued, measurement inputs	0.25
ABS Issued Securities | Loss severity | Maximum
Liabilities
ABS issued, measurement inputs	0.50
ABS Issued Securities | Loss severity | Weighted Average
Liabilities
ABS issued, measurement inputs	0.32
CAFL
Liabilities
ABS issued | $	$ 2,638,183
CAFL | Prepayment rate (annual CPR) | Minimum
Liabilities
ABS issued, measurement inputs	0
CAFL | Prepayment rate (annual CPR) | Maximum
Liabilities
ABS issued, measurement inputs	0.03
CAFL | Prepayment rate (annual CPR) | Weighted Average
Liabilities
ABS issued, measurement inputs	0.002
CAFL | Discount rate | Minimum
Liabilities
ABS issued, measurement inputs	0.02
CAFL | Discount rate | Maximum
Liabilities
ABS issued, measurement inputs	0.23
CAFL | Discount rate | Weighted Average
Liabilities
ABS issued, measurement inputs	0.07
CAFL | Default rate | Minimum
Liabilities
ABS issued, measurement inputs	0.05
CAFL | Default rate | Maximum
Liabilities
ABS issued, measurement inputs	0.23
CAFL | Default rate | Weighted Average
Liabilities
ABS issued, measurement inputs	0.08
CAFL | Loss severity | Minimum
Liabilities
ABS issued, measurement inputs	0.27
CAFL | Loss severity | Maximum
Liabilities
ABS issued, measurement inputs	0.40
CAFL | Loss severity | Weighted Average
Liabilities
ABS issued, measurement inputs	0.30
Freddie Mac SLST
Liabilities
ABS issued | $	$ 1,137,154
Freddie Mac SLST | Prepayment rate (annual CPR) | Minimum
Liabilities
ABS issued, measurement inputs	0.06
Freddie Mac SLST | Prepayment rate (annual CPR) | Maximum
Liabilities
ABS issued, measurement inputs	0.07
Freddie Mac SLST | Prepayment rate (annual CPR) | Weighted Average
Liabilities
ABS issued, measurement inputs	0.06
Freddie Mac SLST | Discount rate | Minimum
Liabilities
ABS issued, measurement inputs	0.05
Freddie Mac SLST | Discount rate | Maximum
Liabilities
ABS issued, measurement inputs	0.16
Freddie Mac SLST | Discount rate | Weighted Average
Liabilities
ABS issued, measurement inputs	0.06
Freddie Mac SLST | Default rate | Minimum
Liabilities
ABS issued, measurement inputs	0.13
Freddie Mac SLST | Default rate | Maximum
Liabilities
ABS issued, measurement inputs	0.14
Freddie Mac SLST | Default rate | Weighted Average
Liabilities
ABS issued, measurement inputs	0.14
Freddie Mac SLST | Loss severity | Minimum
Liabilities
ABS issued, measurement inputs	0.35
Freddie Mac SLST | Loss severity | Maximum
Liabilities
ABS issued, measurement inputs	0.35
Freddie Mac SLST | Loss severity | Weighted Average
Liabilities
ABS issued, measurement inputs	0.35
Freddie Mac K-Series
Liabilities
ABS issued | $	$ 392,785
Freddie Mac K-Series | Discount rate | Minimum
Liabilities
ABS issued, measurement inputs	0.03
Freddie Mac K-Series | Discount rate | Maximum
Liabilities
ABS issued, measurement inputs	0.10
Freddie Mac K-Series | Discount rate | Weighted Average
Liabilities
ABS issued, measurement inputs	0.05
Consolidated Point Entities
Liabilities
ABS issued | $	$ 100,710
Consolidated Point Entities | Prepayment rate (annual CPR) | Minimum
Liabilities
ABS issued, measurement inputs	0.20
Consolidated Point Entities | Prepayment rate (annual CPR) | Maximum
Liabilities
ABS issued, measurement inputs	0.20
Consolidated Point Entities | Prepayment rate (annual CPR) | Weighted Average
Liabilities
ABS issued, measurement inputs	0.20
Consolidated Point Entities | Discount rate | Minimum
Liabilities
ABS issued, measurement inputs	0.09
Consolidated Point Entities | Discount rate | Maximum
Liabilities
ABS issued, measurement inputs	0.15
Consolidated Point Entities | Discount rate | Weighted Average
Liabilities
ABS issued, measurement inputs	0.10
Consolidated Point Entities | Home price appreciation | Minimum
Liabilities
ABS issued, measurement inputs	(0.07)
Consolidated Point Entities | Home price appreciation | Maximum
Liabilities
ABS issued, measurement inputs	0.04
Consolidated Point Entities | Home price appreciation | Weighted Average
Liabilities
ABS issued, measurement inputs	0.03
Jumbo fixed-rate loans
Assets
ABS issued | $	$ 643,845
Jumbo fixed-rate loans | Whole loan spread to swap rate | Minimum
Assets
Loans held-for-sale, measurement input	0.0252
Jumbo fixed-rate loans | Whole loan spread to swap rate | Maximum
Assets
Loans held-for-sale, measurement input	0.0252
Jumbo fixed-rate loans | Whole loan spread to swap rate | Weighted Average
Assets
Loans held-for-sale, measurement input	0.0252
Jumbo fixed-rate loans | CRT dollar price | Minimum
Assets
Loans held-for-sale, measurement input	91
Jumbo fixed-rate loans | CRT dollar price | Maximum
Assets
Loans held-for-sale, measurement input	91
Jumbo fixed-rate loans | CRT dollar price | Weighted Average
Assets
Loans held-for-sale, measurement input	91
Jumbo loans committed to sell
Assets
Loans, held-for-investment | $	$ 136,905
Jumbo loans committed to sell | Whole loan committed sales price | Minimum
Assets
Loans held-for-sale, measurement input	94
Jumbo loans committed to sell | Whole loan committed sales price | Maximum
Assets
Loans held-for-sale, measurement input	101
Jumbo loans committed to sell | Whole loan committed sales price | Weighted Average
Assets
Loans held-for-sale, measurement input	94
Legacy Sequoia
Assets
Loans, held-for-investment | $	$ 184,932
Sequoia
Assets
Loans, held-for-investment | $	3,190,417
Liabilities
Fair value of securities owned | $	219,000
Freddie Mac SLST
Assets
Loans, held-for-investment | $	1,457,058
Liabilities
Fair value of securities owned | $	323,000	445,000
BPL term loans
Assets
Loans, held-for-investment | $	$ 358,791
BPL term loans | Prepayment rate (annual CPR) | Minimum
Assets
Loans receivable, measurement input	0.03
BPL term loans | Prepayment rate (annual CPR) | Maximum
Assets
Loans receivable, measurement input	0.03
BPL term loans | Prepayment rate (annual CPR) | Weighted Average
Assets
Loans receivable, measurement input	0.03
BPL term loans | Senior credit spread | Minimum
Assets
Loans receivable, measurement input	0.0175
BPL term loans | Senior credit spread | Maximum
Assets
Loans receivable, measurement input	0.0275
BPL term loans | Senior credit spread | Weighted Average
Assets
Loans receivable, measurement input	0.0225
BPL term loans | Subordinate credit spread | Minimum
Assets
Loans receivable, measurement input	0.0225
BPL term loans | Subordinate credit spread | Maximum
Assets
Loans receivable, measurement input	0.0962
BPL term loans | Subordinate credit spread | Weighted Average
Assets
Loans receivable, measurement input	0.0431
BPL term loans | Senior credit support | Minimum
Assets
Loans receivable, measurement input	0.36
BPL term loans | Senior credit support | Maximum
Assets
Loans receivable, measurement input	0.36
BPL term loans | Senior credit support | Weighted Average
Assets
Loans receivable, measurement input	0.36
BPL term loans | IO discount rate | Minimum
Assets
Loans receivable, measurement input	0.09
BPL term loans | IO discount rate | Maximum
Assets
Loans receivable, measurement input	0.10
BPL term loans | IO discount rate | Weighted Average
Assets
Loans receivable, measurement input	0.08
BPL term loans | Whole loan spread | Minimum
Assets
Loans receivable, measurement input	0.0275
BPL term loans | Whole loan spread | Maximum
Assets
Loans receivable, measurement input	0.0550
BPL term loans | Whole loan spread | Weighted Average
Assets
Loans receivable, measurement input	0.0361
CAFL
Assets
Loans, held-for-investment | $	$ 2,944,984
Liabilities
Fair value of securities owned | $	304,000	$ 302,000
Residential Bridge, Held-for-investment at fair value
Assets
Loans, held-for-investment | $	$ 2,028,811
Residential Bridge, Held-for-investment at fair value | Prepayment rate (annual CPR) | Minimum
Assets
Loans held-for-investment, measurement input	0
Residential Bridge, Held-for-investment at fair value | Prepayment rate (annual CPR) | Maximum
Assets
Loans held-for-investment, measurement input	0
Residential Bridge, Held-for-investment at fair value | Prepayment rate (annual CPR) | Weighted Average
Assets
Loans held-for-investment, measurement input	0
Residential Bridge, Held-for-investment at fair value | Senior credit spread | Minimum
Assets
Loans held-for-investment, measurement input	0.0310
Residential Bridge, Held-for-investment at fair value | Senior credit spread | Maximum
Assets
Loans held-for-investment, measurement input	0.0310
Residential Bridge, Held-for-investment at fair value | Senior credit spread | Weighted Average
Assets
Loans held-for-investment, measurement input	0.0310
Residential Bridge, Held-for-investment at fair value | Subordinate credit spread | Minimum
Assets
Loans held-for-investment, measurement input	0.0360
Residential Bridge, Held-for-investment at fair value | Subordinate credit spread | Maximum
Assets
Loans held-for-investment, measurement input	0.1150
Residential Bridge, Held-for-investment at fair value | Subordinate credit spread | Weighted Average
Assets
Loans held-for-investment, measurement input	0.0665
Residential Bridge, Held-for-investment at fair value | Senior credit support | Minimum
Assets
Loans held-for-investment, measurement input	0.43
Residential Bridge, Held-for-investment at fair value | Senior credit support | Maximum
Assets
Loans held-for-investment, measurement input	0.43
Residential Bridge, Held-for-investment at fair value | Senior credit support | Weighted Average
Assets
Loans held-for-investment, measurement input	0.43
Residential Bridge, Held-for-investment at fair value | Discount rate | Minimum
Assets
Loans held-for-investment, measurement input	0.05
Residential Bridge, Held-for-investment at fair value | Discount rate | Maximum
Assets
Loans held-for-investment, measurement input	0.15
Residential Bridge, Held-for-investment at fair value | Discount rate | Weighted Average
Assets
Loans held-for-investment, measurement input	0.10
Freddie Mac K-Series
Assets
Loans, held-for-investment | $	$ 424,551
Liabilities
Fair value of securities owned | $	32,000
Trading and AFS securities
Assets
Trading and AFS securities | $	$ 240,475
Trading and AFS securities | Prepayment rate (annual CPR) | Minimum
Assets
Trading and AFS securities, measurement inputs	0.05
Trading and AFS securities | Prepayment rate (annual CPR) | Maximum
Assets
Trading and AFS securities, measurement inputs	0.65
Trading and AFS securities | Prepayment rate (annual CPR) | Weighted Average
Assets
Trading and AFS securities, measurement inputs	0.10
Trading and AFS securities | CRT dollar price | Minimum
Assets
Trading and AFS securities, measurement inputs	72
Trading and AFS securities | CRT dollar price | Maximum
Assets
Trading and AFS securities, measurement inputs	93
Trading and AFS securities | CRT dollar price | Weighted Average
Assets
Trading and AFS securities, measurement inputs	84
Trading and AFS securities | Discount rate | Minimum
Assets
Trading and AFS securities, measurement inputs	0.06
Trading and AFS securities | Discount rate | Maximum
Assets
Trading and AFS securities, measurement inputs	0.18
Trading and AFS securities | Discount rate | Weighted Average
Assets
Trading and AFS securities, measurement inputs	0.10
Trading and AFS securities | Default rate | Minimum
Assets
Trading and AFS securities, measurement inputs	0
Trading and AFS securities | Default rate | Maximum
Assets
Trading and AFS securities, measurement inputs	0.14
Trading and AFS securities | Default rate | Weighted Average
Assets
Trading and AFS securities, measurement inputs	0.005
Trading and AFS securities | Loss severity | Minimum
Assets
Trading and AFS securities, measurement inputs	0
Trading and AFS securities | Loss severity | Maximum
Assets
Trading and AFS securities, measurement inputs	0.50
Trading and AFS securities | Loss severity | Weighted Average
Assets
Trading and AFS securities, measurement inputs	0.26
Servicer advance investments
Assets
Servicing asset | $	$ 269,259
Servicer advance investments | Minimum
Assets
Expected remaining life	5 years
Servicer advance investments | Maximum
Assets
Expected remaining life	5 years
Servicer advance investments | Weighted Average
Assets
Expected remaining life	5 years
Servicer advance investments | Prepayment rate (annual CPR) | Minimum
Assets
Servicing assets, measurement inputs	0.14
Servicer advance investments | Prepayment rate (annual CPR) | Maximum
Assets
Servicing assets, measurement inputs	0.30
Servicer advance investments | Prepayment rate (annual CPR) | Weighted Average
Assets
Servicing assets, measurement inputs	0.14
Servicer advance investments | Discount rate | Minimum
Assets
Servicing assets, measurement inputs	0.02
Servicer advance investments | Discount rate | Maximum
Assets
Servicing assets, measurement inputs	0.05
Servicer advance investments | Discount rate | Weighted Average
Assets
Servicing assets, measurement inputs	0.03
Servicer advance investments | Mortgage servicing income | Minimum
Assets
Loan purchase commitments, net, measurement inputs	0
Servicer advance investments | Mortgage servicing income | Maximum
Assets
Loan purchase commitments, net, measurement inputs	0.0018
Servicer advance investments | Mortgage servicing income | Weighted Average
Assets
Loan purchase commitments, net, measurement inputs	0.0005
MSRs
Assets
Servicing asset | $	$ 25,421
MSRs | Prepayment rate (annual CPR) | Minimum
Assets
Servicing assets, measurement inputs	0.04
MSRs | Prepayment rate (annual CPR) | Maximum
Assets
Servicing assets, measurement inputs	0.28
MSRs | Prepayment rate (annual CPR) | Weighted Average
Assets
Servicing assets, measurement inputs	0.08
MSRs | Discount rate | Minimum
Assets
Servicing assets, measurement inputs	0.11
MSRs | Discount rate | Maximum
Assets
Servicing assets, measurement inputs	0.22
MSRs | Discount rate | Weighted Average
Assets
Servicing assets, measurement inputs	0.11
MSRs | Per loan annual cost to service | Minimum
Assets
Servicing assets, measurement inputs	93
MSRs | Per loan annual cost to service | Maximum
Assets
Servicing assets, measurement inputs	93
MSRs | Per loan annual cost to service | Weighted Average
Assets
Servicing assets, measurement inputs	93
Excess MSRs
Assets
Servicing asset | $	$ 39,035
Excess MSRs | Prepayment rate (annual CPR) | Minimum
Assets
Servicing assets, measurement inputs	0.10
Excess MSRs | Prepayment rate (annual CPR) | Maximum
Assets
Servicing assets, measurement inputs	1
Excess MSRs | Prepayment rate (annual CPR) | Weighted Average
Assets
Servicing assets, measurement inputs	0.18
Excess MSRs | Discount rate | Minimum
Assets
Servicing assets, measurement inputs	0.13
Excess MSRs | Discount rate | Maximum
Assets
Servicing assets, measurement inputs	0.19
Excess MSRs | Discount rate | Weighted Average
Assets
Servicing assets, measurement inputs	0.18
Excess MSRs | Excess mortgage servicing amount | Minimum
Assets
Servicing assets, measurement inputs	0.0008
Excess MSRs | Excess mortgage servicing amount | Maximum
Assets
Servicing assets, measurement inputs	0.0019
Excess MSRs | Excess mortgage servicing amount | Weighted Average
Assets
Servicing assets, measurement inputs	0.0011
HEIs
Assets
Home appreciation options, fair value disclosure | $	$ 270,835
HEIs | Prepayment rate (annual CPR) | Minimum
Assets
Shared home appreciation options, measurement inputs	0.01
HEIs | Prepayment rate (annual CPR) | Maximum
Assets
Shared home appreciation options, measurement inputs	0.23
HEIs | Prepayment rate (annual CPR) | Weighted Average
Assets
Shared home appreciation options, measurement inputs	0.16
HEIs | Discount rate | Minimum
Assets
Shared home appreciation options, measurement inputs	0.10
HEIs | Discount rate | Maximum
Assets
Shared home appreciation options, measurement inputs	0.10
HEIs | Discount rate | Weighted Average
Assets
Shared home appreciation options, measurement inputs	0.10
HEIs | Home price appreciation | Minimum
Assets
Shared home appreciation options, measurement inputs	(0.07)
HEIs | Home price appreciation | Maximum
Assets
Shared home appreciation options, measurement inputs	0.04
HEIs | Home price appreciation | Weighted Average
Assets
Shared home appreciation options, measurement inputs	0.03
Point Options Held By Point
Assets
Home appreciation options, fair value disclosure | $	$ 132,627
Loan purchase and interest rate lock commitments
Assets
Loan purchase commitments, net | $	$ 322
Loan purchase and interest rate lock commitments | Whole loan spread to swap rate | Minimum
Assets
Loan purchase commitments, net, measurement inputs	0.0252
Loan purchase and interest rate lock commitments | Whole loan spread to swap rate | Maximum
Assets
Loan purchase commitments, net, measurement inputs	0.0252
Loan purchase and interest rate lock commitments | Whole loan spread to swap rate | Weighted Average
Assets
Loan purchase commitments, net, measurement inputs	0.0252
Loan purchase and interest rate lock commitments | Pull-through rate | Minimum
Assets
Loan purchase commitments, net, measurement inputs	0.48
Loan purchase and interest rate lock commitments | Pull-through rate | Maximum
Assets
Loan purchase commitments, net, measurement inputs	1
Loan purchase and interest rate lock commitments | Pull-through rate | Weighted Average
Assets
Loan purchase commitments, net, measurement inputs	0.94
Loan purchase and interest rate lock commitments | Committed sales price | Minimum
Assets
Loan purchase commitments, measurement input	101
Loan purchase and interest rate lock commitments | Committed sales price | Maximum
Assets
Loan purchase commitments, measurement input	101
Loan purchase and interest rate lock commitments | Committed sales price | Weighted Average
Assets
Loan purchase commitments, measurement input	101