Securitization Transactions (Schedule of Residential Loan Securitizations) (Details) - USD ($) $ in Thousands	12 Months Ended
	Dec. 31, 2021	Dec. 31, 2020	Dec. 31, 2019
Securitization Transactions [Line Items]
Unrealized gain (loss) on Other secured borrowings, at fair value	$ 15,800	$ (9,300)	$ (800)
Non-QM loan securitization | February 2021
Securitization Transactions [Line Items]
Total Face Amount of Certificates Issued	251,771
Purchase of certificates	1,600
Non-QM loan securitization | November 2019 [Member]
Securitization Transactions [Line Items]
Total Face Amount of Certificates Issued	$ 267,255
Economic interest retained	6.40%
Purchase of certificates	$ 1,700
Non-QM loan securitization | June 2020 [Member]
Securitization Transactions [Line Items]
Total Face Amount of Certificates Issued	259,273
Purchase of certificates	1,900
Non-QM loan securitization | November 2020
Securitization Transactions [Line Items]
Total Face Amount of Certificates Issued	219,732
Purchase of certificates	1,400
Non-QM loan securitization | June 2021
Securitization Transactions [Line Items]
Total Face Amount of Certificates Issued	331,777
Purchase of certificates	2,100
The Company [Member] | Non-QM loan securitization | February 2021
Securitization Transactions [Line Items]
Principal Balance of Loans Transferred to the Depositor	251,771
The Company [Member] | Non-QM loan securitization | November 2019 [Member]
Securitization Transactions [Line Items]
Principal Balance of Loans Transferred to the Depositor	267,255
The Company [Member] | Non-QM loan securitization | June 2020 [Member]
Securitization Transactions [Line Items]
Principal Balance of Loans Transferred to the Depositor	259,273
The Company [Member] | Non-QM loan securitization | November 2020
Securitization Transactions [Line Items]
Principal Balance of Loans Transferred to the Depositor	219,732
The Company [Member] | Non-QM loan securitization | June 2021
Securitization Transactions [Line Items]
Principal Balance of Loans Transferred to the Depositor	$ 331,777
Retained, non-risk retention [Member] | Non-QM loan securitization | February 2021
Securitization Transactions [Line Items]
Economic interest retained	0.40%
Retained, non-risk retention [Member] | Non-QM loan securitization | June 2020 [Member]
Securitization Transactions [Line Items]
Economic interest retained	6.40%
Retained, non-risk retention [Member] | Non-QM loan securitization | November 2020
Securitization Transactions [Line Items]
Economic interest retained	2.40%
Retained, risk retention rules [Member] | Non-QM loan securitization | February 2021
Securitization Transactions [Line Items]
Economic interest retained	5.40%
Retained, risk retention rules [Member] | Non-QM loan securitization | June 2020 [Member]
Securitization Transactions [Line Items]
Economic interest retained	5.10%
Retained, risk retention rules [Member] | Non-QM loan securitization | November 2020
Securitization Transactions [Line Items]
Economic interest retained	5.20%
Retained, risk retention rules [Member] | Non-QM loan securitization | June 2021
Securitization Transactions [Line Items]
Economic interest retained	6.50%