Loans Held for Investment - Schedule of Total Recorded Investment in Non-Performing Loans by Type (Details) - USD ($) $ in Thousands	Dec. 31, 2020	Jun. 30, 2020
Financing Receivable, Allowance for Credit Losses [Line Items]
Related Allowance	$ (928)	$ (442)
Recorded Investment, with No Related Allowance, Net		4,924
Unpaid Principal Balance	11,651	5,833
Related Charge-Offs	453	467
Recorded Investment	11,198	5,366
Recorded Investment, Net of Allowance	10,270
Mortgage loans, Single-family
Financing Receivable, Allowance for Credit Losses [Line Items]
With a related allowance Unpaid Principal Balance	9,636	3,289
With Related Allowance, Related Charge-Offs		0
With Related Allowance, Recorded Investment	9,636	3,289
Related Allowance	(928)	(438)
Recorded Investment, with Related Allowance, Net	8,708	2,851
Without a related allowance, Unpaid Principal Balance	2,015	2,509
With No Related Allowance, Related Charge-Offs	(453)	(467)
With No Related Allowance, Recorded Investment	1,562	2,042
Recorded Investment, with No Related Allowance, Net	1,562	2,042
Unpaid Principal Balance	11,651	5,798
Related Charge-Offs	453	467
Recorded Investment	11,198	5,331
Recorded Investment, Net of Allowance	$ 10,270	4,893
Commercial business loans
Financing Receivable, Allowance for Credit Losses [Line Items]
With a related allowance Unpaid Principal Balance		35
With Related Allowance, Related Charge-Offs		0
With Related Allowance, Recorded Investment		35
Related Allowance		(4)
Recorded Investment, with No Related Allowance, Net		31
Unpaid Principal Balance		35
Related Charge-Offs		0
Recorded Investment		35
Recorded Investment, Net of Allowance		$ 31