Investment Portfolio Financing - Asset-Specific Financing Arrangements (Details) $ in Thousands	Jun. 30, 2023 USD ($) loan	Dec. 31, 2022 USD ($) loan
Debt Instrument [Line Items]
Principal balance	$ 3,705,887	$ 4,179,211
Carrying value	$ 3,705,887
Number of loans | loan	63	70
Unamortized deferred financing costs	$ (6,800)	$ (7,900)
Asset-specific financing arrangements
Debt Instrument [Line Items]
Principal balance	492,563	565,376
Carrying value	492,563
Asset-specific financing arrangements | Financing
Debt Instrument [Line Items]
Aggregate collateral principal balance	665,324	726,330
Principal balance	492,563	565,376
Carrying value	$ 490,581	$ 561,017
Weighted average credit spread (in percent)	3.60%	3.50%
Wtd. avg. term	1 year 9 months 18 days	2 years 4 months 24 days
Unamortized deferred financing costs	$ 2,000	$ 4,400
Asset-specific financing arrangements | Collateral
Debt Instrument [Line Items]
Principal balance	$ 714,577	$ 799,720
Wtd. avg. term	1 year 8 months 12 days	2 years 3 months 18 days
Amortized cost of collateral	$ 701,838	$ 780,160
Axos Bank | Asset-specific financing arrangements | Office Property Mortgage Loan
Debt Instrument [Line Items]
Number of asset-specific financing arrangements | loan	2	2
Number of loans | loan	2	2
Axos Bank | Asset-specific financing arrangements | Financing
Debt Instrument [Line Items]
Aggregate collateral principal balance	$ 100,915	$ 105,152
Principal balance	100,915	105,152
Carrying value	$ 100,265	$ 104,504
Weighted average credit spread (in percent)	4.40%	4.40%
Wtd. avg. term	8 months 12 days	1 year 3 months 18 days
Axos Bank | Asset-specific financing arrangements | Collateral
Debt Instrument [Line Items]
Principal balance	$ 189,905	$ 198,603
Wtd. avg. term	8 months 12 days	1 year 2 months 12 days
Amortized cost of collateral	$ 189,671	$ 198,246
BMO Facility | Asset-specific financing arrangements | Office Property Mortgage Loan
Debt Instrument [Line Items]
Number of asset-specific financing arrangements | loan	1	1
Number of loans | loan	1	2
BMO Facility | Asset-specific financing arrangements | Financing
Debt Instrument [Line Items]
Aggregate collateral principal balance	$ 200,000	$ 200,000
Principal balance	29,110	47,545
Carrying value	$ 28,781	$ 46,985
Weighted average credit spread (in percent)	2.00%	1.80%
Wtd. avg. term	4 years 2 months 12 days	4 years 6 months
BMO Facility | Asset-specific financing arrangements | Collateral
Debt Instrument [Line Items]
Principal balance	$ 36,525	$ 59,431
Wtd. avg. term	4 years 2 months 12 days	4 years 6 months
Amortized cost of collateral	$ 36,197	$ 58,717
Institutional Lender 2 | Asset-specific financing arrangements | Office Property Mortgage Loan
Debt Instrument [Line Items]
Number of asset-specific financing arrangements | loan	1	1
Number of loans | loan	4	5
Institutional Lender 2 | Asset-specific financing arrangements | Financing
Debt Instrument [Line Items]
Aggregate collateral principal balance	$ 341,159	$ 397,928
Principal balance	341,159	392,070
Carrying value	$ 340,585	$ 389,442
Weighted average credit spread (in percent)	3.50%	3.50%
Wtd. avg. term	1 year 10 months 24 days	2 years 4 months 24 days
Institutional Lender 2 | Asset-specific financing arrangements | Collateral
Debt Instrument [Line Items]
Principal balance	$ 459,348	$ 513,181
Wtd. avg. term	1 year 10 months 24 days	2 years 4 months 24 days
Amortized cost of collateral	$ 447,395	$ 494,965
Customers Bank | Asset-specific financing arrangements | Office Property Mortgage Loan
Debt Instrument [Line Items]
Number of asset-specific financing arrangements | loan	1	1
Number of loans | loan	1	1
Customers Bank | Asset-specific financing arrangements | Financing
Debt Instrument [Line Items]
Aggregate collateral principal balance	$ 23,250	$ 23,250
Principal balance	21,379	20,609
Carrying value	$ 20,950	$ 20,086
Weighted average credit spread (in percent)	2.50%	2.50%
Wtd. avg. term	1 year 10 months 24 days	2 years 4 months 24 days
Customers Bank | Asset-specific financing arrangements | Collateral
Debt Instrument [Line Items]
Principal balance	$ 28,799	$ 28,505
Wtd. avg. term	1 year 10 months 24 days	2 years 4 months 24 days
Amortized cost of collateral	$ 28,575	$ 28,232