Loans Held for Investment - Schedule of loan forbearance related to COVID-19 hardship requests (Details)	3 Months Ended
	Sep. 30, 2020 USD ($)
Accounts, Notes, Loans and Financing Receivable [Line Items]
Forbearance Granted, Number of Loans	63
Forbearance Granted, Amount	$ 26,148,000
Forbearance Completed, Number of Loans	18
Forbearance Completed, Amount	$ 8,529,000
Forbearance Remaining, Number of Loans	45
Forbearance Remaining, Amount	$ 17,619,000
% of Total Loans	1.99%
Weighted Avg. LTV	62.00%
Weighted Avg. FICO	733
Weighted Avg. Debt Coverage Ratio	1.32
Weighted Avg. Forbearance Period Granted	5 months 27 days
Mortgage loans, Single-family
Accounts, Notes, Loans and Financing Receivable [Line Items]
Forbearance Granted, Number of Loans	57
Forbearance Granted, Amount	$ 23,036,000
Forbearance Completed, Number of Loans	13
Forbearance Completed, Amount	$ 5,872,000
Forbearance Remaining, Number of Loans	44
Forbearance Remaining, Amount	$ 17,164,000
% of Total Loans	1.94%
Weighted Avg. LTV	62.00%
Weighted Avg. FICO	737
Weighted Avg. Forbearance Period Granted	6 months
Mortgage Loans, Multi Family
Accounts, Notes, Loans and Financing Receivable [Line Items]
Forbearance Granted, Number of Loans	4
Forbearance Granted, Amount	$ 2,043,000
Forbearance Completed, Number of Loans	3
Forbearance Completed, Amount	$ 1,588,000
Forbearance Remaining, Number of Loans	1
Forbearance Remaining, Amount	$ 455,000
% of Total Loans	0.05%
Weighted Avg. LTV	60.00%
Weighted Avg. FICO	687
Weighted Avg. Debt Coverage Ratio	1.32
Weighted Avg. Forbearance Period Granted	3 months
Mortgage loans, Commercial real estate
Accounts, Notes, Loans and Financing Receivable [Line Items]
Forbearance Granted, Number of Loans	2
Forbearance Granted, Amount	$ 1,069,000
Forbearance Completed, Number of Loans	2
Forbearance Completed, Amount	$ 1,069,000