Loans Held for Investment - Schedule of Loans Held for Investment Contractually Repricing (Details) - USD ($) $ in Thousands	Sep. 30, 2020	Jun. 30, 2020
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans receivable, Fixed Rate	$ 11,140
Total loans held for investment, gross	886,849	$ 904,466
Mortgage loans, Single-family
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans receivable, Fixed Rate	7,988
Total loans held for investment, gross	288,790	298,810
Mortgage Loans, Multi Family
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans receivable, Fixed Rate	148
Total loans held for investment, gross	482,900	491,903
Mortgage loans, Commercial real estate
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans receivable, Fixed Rate	313
Total loans held for investment, gross	105,207	105,235
Construction
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans receivable, Fixed Rate	2,161
Total loans held for investment, gross	8,787	7,801
Mortgage Loans Other
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans receivable, Fixed Rate	142
Total loans held for investment, gross	142	143
Commercial business loans
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans receivable, Fixed Rate	388
Total loans held for investment, gross	923	480
Consumer loans
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans receivable, Fixed Rate	0
Total loans held for investment, gross	100	$ 94
Within One Year [Member]
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans receivable, Adjustable Rate	289,626
Within One Year [Member] | Mortgage loans, Single-family
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans receivable, Adjustable Rate	75,377
Within One Year [Member] | Mortgage Loans, Multi Family
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans receivable, Adjustable Rate	158,412
Within One Year [Member] | Mortgage loans, Commercial real estate
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans receivable, Adjustable Rate	48,576
Within One Year [Member] | Construction
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans receivable, Adjustable Rate	6,626
Within One Year [Member] | Commercial business loans
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans receivable, Adjustable Rate	535
Within One Year [Member] | Consumer loans
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans receivable, Adjustable Rate	100
After One Year Through 3 Years [Member]
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans receivable, Adjustable Rate	240,384
After One Year Through 3 Years [Member] | Mortgage loans, Single-family
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans receivable, Adjustable Rate	56,552
After One Year Through 3 Years [Member] | Mortgage Loans, Multi Family
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans receivable, Adjustable Rate	155,709
After One Year Through 3 Years [Member] | Mortgage loans, Commercial real estate
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans receivable, Adjustable Rate	28,123
After One Year Through 3 Years [Member] | Construction
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans receivable, Adjustable Rate	0
After One Year Through 3 Years [Member] | Mortgage Loans Other
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans receivable, Adjustable Rate	0
After One Year Through 3 Years [Member] | Commercial business loans
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans receivable, Adjustable Rate	0
After One Year Through 3 Years [Member] | Consumer loans
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans receivable, Adjustable Rate	0
After 3 Years Through 5 Years [Member]
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans receivable, Adjustable Rate	256,567
After 3 Years Through 5 Years [Member] | Mortgage loans, Single-family
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans receivable, Adjustable Rate	74,327
After 3 Years Through 5 Years [Member] | Mortgage Loans, Multi Family
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans receivable, Adjustable Rate	154,045
After 3 Years Through 5 Years [Member] | Mortgage loans, Commercial real estate
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans receivable, Adjustable Rate	28,195
After 3 Years Through 5 Years [Member] | Construction
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans receivable, Adjustable Rate	0
After 3 Years Through 5 Years [Member] | Mortgage Loans Other
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans receivable, Adjustable Rate	0
After 3 Years Through 5 Years [Member] | Commercial business loans
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans receivable, Adjustable Rate	0
After 3 Years Through 5 Years [Member] | Consumer loans
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans receivable, Adjustable Rate	0
After 5 Years Through 10 Years [Member]
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans receivable, Adjustable Rate	89,132
After 5 Years Through 10 Years [Member] | Mortgage loans, Single-family
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans receivable, Adjustable Rate	74,546
After 5 Years Through 10 Years [Member] | Mortgage Loans, Multi Family
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans receivable, Adjustable Rate	14,586
After 5 Years Through 10 Years [Member] | Mortgage loans, Commercial real estate
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans receivable, Adjustable Rate	0
After 5 Years Through 10 Years [Member] | Construction
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans receivable, Adjustable Rate	0
After 5 Years Through 10 Years [Member] | Mortgage Loans Other
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans receivable, Adjustable Rate	0
After 5 Years Through 10 Years [Member] | Commercial business loans
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans receivable, Adjustable Rate	0
After 5 Years Through 10 Years [Member] | Consumer loans
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans receivable, Adjustable Rate	$ 0