Debt - Narrative (Details)			1 Months Ended	3 Months Ended		12 Months Ended
		Apr. 30, 2024 USD ($)	Aug. 31, 2022 USD ($)	Mar. 31, 2024 USD ($) securitization counterparty facility trust	Mar. 31, 2023 USD ($)	Dec. 31, 2023 USD ($)	Dec. 31, 2022 USD ($)	Dec. 31, 2020	Nov. 30, 2023 USD ($)	Jan. 01, 2022 USD ($)
Debt Instrument [Line Items]
Percentage of guarantors beneficial interest (in percentage)				100.00%
Cash collateral				$ 4,511,000		$ 3,756,000
Number of securitizations completed | securitization				18
Number of securitizations outstanding | securitization				5
Convertible senior notes, net	[1]			$ 103,516,000		103,516,000
Interest expense				14,106,000	$ 14,925,000
Repurchase of the company's senior convertible notes				0	952,000
Equity, attributable to parent				254,297,000		308,933,000
Proceeds used to repurchase and retire securities				$ 2,054,000	13,775,000	$ 16,644,000	$ 12,153,000
Series A Preferred Stock
Debt Instrument [Line Items]
Preferred stock, fixed-to-floating rate cumulative redeemable				7.25%		7.25%	7.25%	7.25%
Series B Preferred Stock
Debt Instrument [Line Items]
Preferred stock, fixed-to-floating rate cumulative redeemable				5.00%		5.00%	5.00%	5.00%
Accounting Standards Update 2020-06
Debt Instrument [Line Items]
Convertible senior notes, net										$ 700,000
Additional Paid-in Capital | Accounting Standards Update 2020-06
Debt Instrument [Line Items]
Equity, attributable to parent										$ (700,000)
Gregory Funding LLC
Debt Instrument [Line Items]
Ceiling for each repurchase facility				$ 12,000,000					$ 3,500,000
Convertible notes payable
Debt Instrument [Line Items]
Interest expense				$ 1,900,000	2,100,000
Master repurchase agreement
Debt Instrument [Line Items]
Interest rate (in percent)				7.30%		7.44%
Amount outstanding				$ 354,039,000		$ 375,745,000
Servicing agreement | Minimum | Gregory Funding LLC
Debt Instrument [Line Items]
Servicing fees (in percent)				0.65%
Servicing agreement | Maximum | Gregory Funding LLC
Debt Instrument [Line Items]
Servicing fees (in percent)				1.25%
Mortgage loans | Non-rated secured borrowings
Debt Instrument [Line Items]
Number of securitizations outstanding | securitization				1
Mortgage loans | Rated secured borrowings
Debt Instrument [Line Items]
Number of securitizations outstanding | securitization				4
Convertible notes payable
Debt Instrument [Line Items]
Interest rate (in percent)				7.25%
If-converted value in excess of principal				$ 76,100,000
Unpaid principal balance				103,500,000		103,500,000
Interest expense				2,700,000	2,600,000
Amortization of debt discount				$ 0	$ 200,000
Interest rate, effective percentage				7.25%	8.01%
Debt instrument, repurchase amount				$ 0	$ 1,000,000
Repurchase of the company's senior convertible notes					1,000,000
Amortization of debt discount and deferred expenses				200,000	$ 200,000
Convertible notes payable | Subsequent event
Debt Instrument [Line Items]
Repayments of debt		$ 103,500,000
Accrued interest period (in days)		15 days
Convertible Debt
Debt Instrument [Line Items]
Debt issuance costs, net				$ 0		0
Senior notes | 8.875% Senior Unsecured Notes Due September 2027
Debt Instrument [Line Items]
Interest rate (in percent)			8.875%
Interest rate, effective percentage				9.94%	9.89%
Aggregate principal			$ 110,000,000
Debt instrument, term (in years)			5 years
Issuance amount, amount of par value (percent)			0.99009
Proceeds from unsecured notes payable			$ 106,100,000
Proceeds used to repurchase and retire securities				$ 90,000,000
Amount outstanding				110,000,000		110,000,000
Notes Payable
Debt Instrument [Line Items]
Debt issuance costs, net				$ (2,900,000)		$ (3,200,000)
Delaware Trust | Mortgage loans | Master repurchase agreement
Debt Instrument [Line Items]
Number of facilities repurchased | facility				2
Number of wholly-owned Delaware trusts | trust				2
Number of counterparties | counterparty				2
Ceiling for each repurchase facility				$ 150,000,000
Delaware Trust | Mortgage loans | Master repurchase agreement | Minimum
Debt Instrument [Line Items]
Percentage of purchase price for each mortgage loan or REO (in percentage)				75.00%
Delaware Trust | Mortgage loans | Master repurchase agreement | Maximum
Debt Instrument [Line Items]
Percentage of purchase price for each mortgage loan or REO (in percentage)				90.00%
Delaware Trust | Mortgages one | Master repurchase agreement
Debt Instrument [Line Items]
Ceiling for each repurchase facility				$ 400,000,000
Delaware Trust | Bonds | Master repurchase agreement
Debt Instrument [Line Items]
Number of facilities repurchased | facility				4

[1] Secured borrowings, net are presented net of deferred issuance costs of $2.8 million at March 31, 2024 and $3.1 million at December 31, 2023. Convertible senior notes are presented net of deferred issuance costs of zero at both March 31, 2024 and December 31, 2023. Notes payable, net are presented net of deferred issuance costs and discount of $2.9 million at March 31, 2024 and $3.2 million at December 31, 2023.