Loans - Schedule of Debt Restructurings on Financing Receivables (Details) $ in Thousands	12 Months Ended
	Jun. 30, 2020 USD ($) loan	Jun. 30, 2019 USD ($) loan	Jun. 30, 2018 USD ($) loan
Financing Receivable, Modifications
Number of loans | loan	15	29	31
Pre-modification outstanding recorded investment	$ 1,797	$ 7,291	$ 4,198
Post-modification outstanding recorded investment	$ 1,651	$ 7,249	$ 4,283
Extended term
Financing Receivable, Modifications
Number of loans | loan	3	3	5
Pre-modification outstanding recorded investment	$ 896	$ 68	$ 550
Post-modification outstanding recorded investment	$ 887	$ 67	$ 534
Other Troubled Debt Restructing [Member]
Financing Receivable, Modifications
Number of loans | loan	11	25	26
Pre-modification outstanding recorded investment	$ 813	$ 7,138	$ 3,648
Post-modification outstanding recorded investment	$ 678	$ 7,098	$ 3,749
Retail Consumer Loans | Below market interest rate
Financing Receivable, Modifications
Number of loans | loan	1	1	0
Pre-modification outstanding recorded investment	$ 88	$ 85	$ 0
Post-modification outstanding recorded investment	$ 86	$ 84	$ 0
Retail Consumer Loans | Below market interest rate | One-to-four family
Financing Receivable, Modifications
Number of loans | loan	0	1	0
Pre-modification outstanding recorded investment	$ 0	$ 85	$ 0
Post-modification outstanding recorded investment	$ 0	$ 84	$ 0
Retail Consumer Loans | Extended term | One-to-four family
Financing Receivable, Modifications
Number of loans | loan	2	1	4
Pre-modification outstanding recorded investment	$ 70	$ 34	$ 514
Post-modification outstanding recorded investment	$ 61	$ 34	$ 502
Retail Consumer Loans | Extended term | Construction and land/lots
Financing Receivable, Modifications
Number of loans | loan	0	0	1
Pre-modification outstanding recorded investment	$ 0	$ 0	$ 36
Post-modification outstanding recorded investment	$ 0	$ 0	$ 32
Retail Consumer Loans | Extended term | Consumer
Financing Receivable, Modifications
Number of loans | loan	0	2	0
Pre-modification outstanding recorded investment	$ 0	$ 34	$ 0
Post-modification outstanding recorded investment	$ 0	$ 33	$ 0
Retail Consumer Loans | Other Troubled Debt Restructing [Member] | One-to-four family
Financing Receivable, Modifications
Number of loans | loan	5	18	25
Pre-modification outstanding recorded investment	$ 511	$ 1,452	$ 3,646
Post-modification outstanding recorded investment	$ 502	$ 1,433	$ 3,747
Retail Consumer Loans | Other Troubled Debt Restructing [Member] | HELOCs - originated
Financing Receivable, Modifications
Number of loans | loan	1	0	0
Pre-modification outstanding recorded investment	$ 27	$ 0	$ 0
Post-modification outstanding recorded investment	$ 27	$ 0	$ 0
Retail Consumer Loans | Other Troubled Debt Restructing [Member] | Construction and land/lots
Financing Receivable, Modifications
Number of loans | loan	0	1	0
Pre-modification outstanding recorded investment	$ 0	$ 29	$ 0
Post-modification outstanding recorded investment	$ 0	$ 28	$ 0
Retail Consumer Loans | Other Troubled Debt Restructing [Member] | Indirect Auto Finance [Member]
Financing Receivable, Modifications
Number of loans | loan	3	1	0
Pre-modification outstanding recorded investment	$ 63	$ 33	$ 0
Post-modification outstanding recorded investment	$ 49	$ 26	$ 0
Retail Consumer Loans | Other Troubled Debt Restructing [Member] | Consumer
Financing Receivable, Modifications
Number of loans | loan	0	1	1
Pre-modification outstanding recorded investment	$ 0	$ 2	$ 2
Post-modification outstanding recorded investment	$ 0	$ 2	$ 2
Commercial Loans | Below market interest rate | Commercial real estate
Financing Receivable, Modifications
Number of loans | loan	1	0	0
Pre-modification outstanding recorded investment	$ 88	$ 0	$ 0
Post-modification outstanding recorded investment	$ 86	$ 0	$ 0
Commercial Loans | Extended term | Commercial and industrial
Financing Receivable, Modifications
Number of loans | loan	1	0	0
Pre-modification outstanding recorded investment	$ 826	$ 0	$ 0
Post-modification outstanding recorded investment	$ 826	$ 0	$ 0
Commercial Loans | Other Troubled Debt Restructing [Member] | Construction and development
Financing Receivable, Modifications
Number of loans | loan	1	1	0
Pre-modification outstanding recorded investment	$ 182	$ 182	$ 0
Post-modification outstanding recorded investment	$ 79	$ 182	$ 0
Commercial Loans | Other Troubled Debt Restructing [Member] | Commercial real estate
Financing Receivable, Modifications
Number of loans | loan	1	3	0
Pre-modification outstanding recorded investment	$ 30	$ 5,440	$ 0
Post-modification outstanding recorded investment	$ 21	$ 5,427	$ 0