Income Taxes (Details) - USD ($) $ / shares in Units, $ in Thousands		12 Months Ended
		Dec. 31, 2023		Dec. 31, 2022		Dec. 31, 2021		Jan. 31, 2024	Jan. 31, 2023	Jan. 31, 2022
Income Taxes [Abstract]
Percentage of taxable income that must be distributed to qualify as a REIT		90.00%
Components of Income Tax Expense (Benefit) [Abstract]
Current federal income tax benefit		$ 0		$ 0		$ (127)
Deferred federal income tax expense		280		4,116		1,180
Deferred state income tax expense		243		954		(272)
Provision for Corporate Business Taxes/Effective Tax Rate	[1]	523		5,070		781
Effective Income Tax Rate Reconciliation, Amount [Abstract]
Computed income tax expense (benefit) at federal rate		(7,336)		5,724		2,795
State tax expense, net of federal tax, if applicable		30		494		120
Tax provision due to state tax rate change		206		329		(413)
Permanent differences in taxable income from GAAP pre-tax income		0		0		185
Provision to return adjustment		(7)		(7)		(6)
REIT income not subject to tax expense (benefit)		7,630		(1,470)		(1,900)
Provision for Corporate Business Taxes/Effective Tax Rate	[1]	$ 523		$ 5,070		$ 781
Effective Income Tax Rate Reconciliation, Percent [Abstract]
Computed income tax (benefit) expense at federal rate		21.00%		21.00%		21.00%
State tax (benefit), net of federal tax, if applicable		(0.10%)		1.80%		0.90%
Tax provision due to state tax rate change		(0.60%)		1.20%		(3.10%)
Permanent differences in taxable income from GAAP pre-tax income		0.00%		0.00%		1.40%
Provision to return adjustment		0.00%		0.00%		0.00%
REIT income not subject to tax expense (benefit)		(21.80%)		(5.40%)		(14.30%)
Provision for (Benefit from) effective Tax Rate	[1]	(1.50%)		18.60%		5.90%
Income taxes recoverable [Abstract]
Income taxes recoverable		$ 0		$ 128		$ 128
Deferred tax assets [Abstract]
Deferred tax - mortgage servicing rights		(1,789)		1,082		10,539
Deferred tax - net operating loss		16,811		13,844		10,075
Deferred tax - other		0		619		0
Total net deferred tax assets		15,022		$ 15,545		$ 20,614
Net operating loss carryforwards		$ 72,600
Series A Preferred Stock [Member]
Distributions to Stockholders [Abstract]
Dividend declared (in dollars per share)		$ 2.05	[2]	$ 2.05	[3]	$ 2.05	[4]
Ordinary income		100.00%		100.00%		10.00%
Long-term capital gain		0.00%		0.00%		0.00%
Return of capital		0.00%		0.00%		90.00%
Series A Preferred Stock [Member] | Dividend Declared December 2022 [Member]
Distributions to Stockholders [Abstract]
Dividends payable, date declared		Dec. 31, 2023
Dividend payable, date to be paid		Jan. 31, 2024
Series A Preferred Stock [Member] | Dividend Declared December 2022 [Member] | Subsequent Event [Member]
Distributions to Stockholders [Abstract]
Dividend declared (in dollars per share)								$ 0.51
Series A Preferred Stock [Member] | Dividend Declared December 2021 [Member]
Distributions to Stockholders [Abstract]
Dividend declared (in dollars per share)									$ 0.51
Dividends payable, date declared				Dec. 31, 2022
Dividend payable, date to be paid				Jan. 31, 2023
Series A Preferred Stock [Member] | Dividend Declared December 2020 [Member]
Distributions to Stockholders [Abstract]
Dividend declared (in dollars per share)										$ 0.51
Dividends payable, date declared						Dec. 31, 2021
Dividend payable, date to be paid						Jan. 31, 2022
Series B Preferred Stock [Member]
Distributions to Stockholders [Abstract]
Dividend declared (in dollars per share)		$ 2.06	[5]	$ 2.06	[6]	$ 2.06	[7]
Ordinary income		100.00%		100.00%		10.00%
Long-term capital gain		0.00%		0.00%		0.00%
Return of capital		0.00%		0.00%		90.00%
Series B Preferred Stock [Member] | Dividend Declared December 2022 [Member]
Distributions to Stockholders [Abstract]
Dividends payable, date declared		Dec. 31, 2023
Dividend payable, date to be paid		Jan. 31, 2024
Series B Preferred Stock [Member] | Dividend Declared December 2022 [Member] | Subsequent Event [Member]
Distributions to Stockholders [Abstract]
Dividend declared (in dollars per share)								0.52
Series B Preferred Stock [Member] | Dividend Declared December 2021 [Member]
Distributions to Stockholders [Abstract]
Dividend declared (in dollars per share)									0.52
Dividends payable, date declared				Dec. 31, 2022
Dividend payable, date to be paid				Jan. 31, 2023
Series B Preferred Stock [Member] | Dividend Declared December 2020 [Member]
Distributions to Stockholders [Abstract]
Dividend declared (in dollars per share)										0.52
Dividends payable, date declared						Dec. 31, 2021
Dividend payable, date to be paid						Jan. 31, 2022
Common Stock [Member]
Distributions to Stockholders [Abstract]
Dividend declared (in dollars per share)		$ 0.84	[8]	$ 1.08	[9]	$ 1.08	[10]
Ordinary income		95.00%		60.00%		0.00%
Long-term capital gain		0.00%		0.00%		0.00%
Return of capital		5.00%		40.00%		100.00%
Common Stock [Member] | Dividend Declared December 2022 [Member]
Distributions to Stockholders [Abstract]
Dividends payable, date declared		Dec. 31, 2023
Dividend payable, date to be paid		Jan. 31, 2024
Common Stock [Member] | Dividend Declared December 2022 [Member] | Subsequent Event [Member]
Distributions to Stockholders [Abstract]
Dividend declared (in dollars per share)								$ 0.15
Common Stock [Member] | Dividend Declared December 2021 [Member]
Distributions to Stockholders [Abstract]
Dividend declared (in dollars per share)									$ 0.27
Dividends payable, date declared				Dec. 31, 2022
Dividend payable, date to be paid				Jan. 31, 2023
Common Stock [Member] | Dividend Declared December 2020 [Member]
Distributions to Stockholders [Abstract]
Dividend declared (in dollars per share)										$ 0.27
Dividends payable, date declared						Dec. 31, 2021
Dividend payable, date to be paid						Jan. 31, 2022

[1] The provision for income taxes is recorded at the TRS level.

[2] The entire $0.51 per share dividend declared in December 2023 and paid in January 2024 is treated as received by stockholders in 2024

[3] The entire $0.51 per share dividend declared in December 2022 and paid in January 2023 is treated as received by stockholders in 2023

[4] The entire $0.51 per share dividend declared in December 2021 and paid in January 2022 is treated as received by stockholders in 2022

[5] The entire $0.52 per share dividend declared in December 2023 and paid in January 2024 is treated as received by stockholders in 2024

[6] The entire $0.52 per share dividend declared in December 2022 and paid in January 2023 is treated as received by stockholders in 2023

[7] The entire $0.52 per share dividend declared in December 2021 and paid in January 2022 is treated as received by stockholders in 2022

[8] The entire $0.15 per share dividend declared in December 2023 and paid in January 2024 is treated as received by stockholders in 2024

[9] The entire $0.27 per share dividend declared in December 2022 and paid in January 2023 is treated as received by stockholders in 2023

[10] The entire $0.27 per share dividend declared in December 2021 and paid in January 2022 is treated as received by stockholders in 2022