Loans Held for Investment - Schedule of Past Due Status of Loans Held for Investment, Gross (Details) - USD ($) $ in Thousands	12 Months Ended
	Jun. 30, 2021	Jun. 30, 2020
Financing Receivable, Recorded Investment, Past Due [Line Items]
Loans deemed uncollectible, period of delinquency	90 days	90 days
Current	$ 842,668	$ 898,947
30-89 Days Past Due	7	219
Non-Accrual	9,407	5,300
Total Loans Held for Investment, Gross	852,082	904,466
Mortgage loans, Single-family
Financing Receivable, Recorded Investment, Past Due [Line Items]
Current	259,984	293,326
30-89 Days Past Due	0	219
Non-Accrual	8,288	5,265
Total Loans Held for Investment, Gross	268,272	298,810
Mortgage Loans, Multi Family
Financing Receivable, Recorded Investment, Past Due [Line Items]
Current	483,289	491,903
30-89 Days Past Due	0	0
Non-Accrual	1,119	0
Total Loans Held for Investment, Gross	484,408	491,903
Mortgage loans, Commercial real estate
Financing Receivable, Recorded Investment, Past Due [Line Items]
Current	95,279	105,235
30-89 Days Past Due	0	0
Non-Accrual	0	0
Total Loans Held for Investment, Gross	95,279	105,235
Construction
Financing Receivable, Recorded Investment, Past Due [Line Items]
Current	3,040	7,801
30-89 Days Past Due	0	0
Non-Accrual	0	0
Total Loans Held for Investment, Gross	3,040	7,801
Mortgage Loans Other
Financing Receivable, Recorded Investment, Past Due [Line Items]
Current	139	143
30-89 Days Past Due	0	0
Non-Accrual	0	0
Total Loans Held for Investment, Gross	139	143
Commercial business loans
Financing Receivable, Recorded Investment, Past Due [Line Items]
Current	849	445
30-89 Days Past Due	0	0
Non-Accrual	0	35
Total Loans Held for Investment, Gross	849	480
Consumer loans
Financing Receivable, Recorded Investment, Past Due [Line Items]
Current	88	94
30-89 Days Past Due	7	0
Non-Accrual	0	0
Total Loans Held for Investment, Gross	$ 95	$ 94
Bankruptcy [Member]
Financing Receivable, Recorded Investment, Past Due [Line Items]
Loans deemed uncollectible, period of delinquency	60 days
Troubled Debt Restructurings [Member]
Financing Receivable, Recorded Investment, Past Due [Line Items]
Loans deemed uncollectible, period of delinquency	90 days
Commercial Real Estate Or Second Mortgage [Member]
Financing Receivable, Recorded Investment, Past Due [Line Items]
Loans deemed uncollectible, period of delinquency	120 days