Loans - Financing Receivable Credit Quality Indicators (Details) - USD ($) $ in Thousands	Jun. 30, 2020	Jun. 30, 2019
Financing Receivable, Recorded Investment
Non-purchased and purchased performing loans	$ 2,756,138	$ 2,688,436
Pass
Financing Receivable, Recorded Investment
Non-purchased and purchased performing loans	2,694,017	2,650,990
Special Mention
Financing Receivable, Recorded Investment
Non-purchased and purchased performing loans	33,803	12,968
Substandard
Financing Receivable, Recorded Investment
Non-purchased and purchased performing loans	28,095	24,058
Doubtful
Financing Receivable, Recorded Investment
Non-purchased and purchased performing loans	207	388
Loss
Financing Receivable, Recorded Investment
Non-purchased and purchased performing loans	16	32
Retail Consumer Loans | One-to-four family
Financing Receivable, Recorded Investment
Non-purchased and purchased performing loans	469,220	654,723
Retail Consumer Loans | HELOCs - originated
Financing Receivable, Recorded Investment
Non-purchased and purchased performing loans	137,447	139,210
Retail Consumer Loans | HELOCs - purchased
Financing Receivable, Recorded Investment
Non-purchased and purchased performing loans	71,781	116,972
Retail Consumer Loans | Construction and land/lots
Financing Receivable, Recorded Investment
Non-purchased and purchased performing loans	81,514	80,230
Retail Consumer Loans | Indirect auto finance
Financing Receivable, Recorded Investment
Non-purchased and purchased performing loans	132,303	153,448
Retail Consumer Loans | Consumer
Financing Receivable, Recorded Investment
Non-purchased and purchased performing loans	10,259	11,416
Retail Consumer Loans | Pass | One-to-four family
Financing Receivable, Recorded Investment
Non-purchased and purchased performing loans	458,248	644,159
Retail Consumer Loans | Pass | HELOCs - originated
Financing Receivable, Recorded Investment
Non-purchased and purchased performing loans	134,697	137,001
Retail Consumer Loans | Pass | HELOCs - purchased
Financing Receivable, Recorded Investment
Non-purchased and purchased performing loans	71,119	116,306
Retail Consumer Loans | Pass | Construction and land/lots
Financing Receivable, Recorded Investment
Non-purchased and purchased performing loans	81,112	79,995
Retail Consumer Loans | Pass | Indirect auto finance
Financing Receivable, Recorded Investment
Non-purchased and purchased performing loans	130,975	152,393
Retail Consumer Loans | Pass | Consumer
Financing Receivable, Recorded Investment
Non-purchased and purchased performing loans	9,894	11,375
Retail Consumer Loans | Special Mention | One-to-four family
Financing Receivable, Recorded Investment
Non-purchased and purchased performing loans	1,724	2,089
Retail Consumer Loans | Special Mention | HELOCs - originated
Financing Receivable, Recorded Investment
Non-purchased and purchased performing loans	902	766
Retail Consumer Loans | Special Mention | HELOCs - purchased
Financing Receivable, Recorded Investment
Non-purchased and purchased performing loans	0	0
Retail Consumer Loans | Special Mention | Construction and land/lots
Financing Receivable, Recorded Investment
Non-purchased and purchased performing loans	0	71
Retail Consumer Loans | Special Mention | Indirect auto finance
Financing Receivable, Recorded Investment
Non-purchased and purchased performing loans	0	13
Retail Consumer Loans | Special Mention | Consumer
Financing Receivable, Recorded Investment
Non-purchased and purchased performing loans	4	1
Retail Consumer Loans | Substandard | One-to-four family
Financing Receivable, Recorded Investment
Non-purchased and purchased performing loans	9,042	8,072
Retail Consumer Loans | Substandard | HELOCs - originated
Financing Receivable, Recorded Investment
Non-purchased and purchased performing loans	1,848	1,434
Retail Consumer Loans | Substandard | HELOCs - purchased
Financing Receivable, Recorded Investment
Non-purchased and purchased performing loans	662	666
Retail Consumer Loans | Substandard | Construction and land/lots
Financing Receivable, Recorded Investment
Non-purchased and purchased performing loans	402	164
Retail Consumer Loans | Substandard | Indirect auto finance
Financing Receivable, Recorded Investment
Non-purchased and purchased performing loans	1,328	1,042
Retail Consumer Loans | Substandard | Consumer
Financing Receivable, Recorded Investment
Non-purchased and purchased performing loans	361	33
Retail Consumer Loans | Doubtful | One-to-four family
Financing Receivable, Recorded Investment
Non-purchased and purchased performing loans	206	384
Retail Consumer Loans | Doubtful | HELOCs - originated
Financing Receivable, Recorded Investment
Non-purchased and purchased performing loans	0	0
Retail Consumer Loans | Doubtful | HELOCs - purchased
Financing Receivable, Recorded Investment
Non-purchased and purchased performing loans	0	0
Retail Consumer Loans | Doubtful | Construction and land/lots
Financing Receivable, Recorded Investment
Non-purchased and purchased performing loans	0	0
Retail Consumer Loans | Doubtful | Indirect auto finance
Financing Receivable, Recorded Investment
Non-purchased and purchased performing loans
Retail Consumer Loans | Doubtful | Consumer
Financing Receivable, Recorded Investment
Non-purchased and purchased performing loans	0	3
Retail Consumer Loans | Loss | One-to-four family
Financing Receivable, Recorded Investment
Non-purchased and purchased performing loans	0	19
Retail Consumer Loans | Loss | HELOCs - originated
Financing Receivable, Recorded Investment
Non-purchased and purchased performing loans	0	9
Retail Consumer Loans | Loss | HELOCs - purchased
Financing Receivable, Recorded Investment
Non-purchased and purchased performing loans	0	0
Retail Consumer Loans | Loss | Construction and land/lots
Financing Receivable, Recorded Investment
Non-purchased and purchased performing loans	0	0
Retail Consumer Loans | Loss | Indirect auto finance
Financing Receivable, Recorded Investment
Non-purchased and purchased performing loans	0	0
Retail Consumer Loans | Loss | Consumer
Financing Receivable, Recorded Investment
Non-purchased and purchased performing loans	0	4
Commercial Loans | Commercial real estate
Financing Receivable, Recorded Investment
Non-purchased and purchased performing loans	1,047,084	919,586
Commercial Loans | Construction and development
Financing Receivable, Recorded Investment
Non-purchased and purchased performing loans	215,344	209,975
Commercial Loans | Commercial and industrial
Financing Receivable, Recorded Investment
Non-purchased and purchased performing loans	153,263	158,802
Commercial Loans | Equipment finance
Financing Receivable, Recorded Investment
Non-purchased and purchased performing loans	229,239	132,058
Commercial Loans | Municipal leases
Financing Receivable, Recorded Investment
Non-purchased and purchased performing loans	127,987	112,016
Commercial Loans | paycheck protection program [Domain]
Financing Receivable, Recorded Investment
Non-purchased and purchased performing loans	80,697
Commercial Loans | Pass | Commercial real estate
Financing Receivable, Recorded Investment
Non-purchased and purchased performing loans	1,028,709	901,214
Commercial Loans | Pass | Construction and development
Financing Receivable, Recorded Investment
Non-purchased and purchased performing loans	212,370	207,827
Commercial Loans | Pass | Commercial and industrial
Financing Receivable, Recorded Investment
Non-purchased and purchased performing loans	130,202	157,325
Commercial Loans | Pass | Equipment finance
Financing Receivable, Recorded Investment
Non-purchased and purchased performing loans	228,288	131,674
Commercial Loans | Pass | Municipal leases
Financing Receivable, Recorded Investment
Non-purchased and purchased performing loans	127,706	111,721
Commercial Loans | Pass | paycheck protection program [Domain]
Financing Receivable, Recorded Investment
Non-purchased and purchased performing loans	80,697
Commercial Loans | Special Mention | Commercial real estate
Financing Receivable, Recorded Investment
Non-purchased and purchased performing loans	7,580	8,066
Commercial Loans | Special Mention | Construction and development
Financing Receivable, Recorded Investment
Non-purchased and purchased performing loans	2,723	790
Commercial Loans | Special Mention | Commercial and industrial
Financing Receivable, Recorded Investment
Non-purchased and purchased performing loans	20,439	877
Commercial Loans | Special Mention | Equipment finance
Financing Receivable, Recorded Investment
Non-purchased and purchased performing loans	150	0
Commercial Loans | Special Mention | Municipal leases
Financing Receivable, Recorded Investment
Non-purchased and purchased performing loans	281	295
Commercial Loans | Special Mention | paycheck protection program [Domain]
Financing Receivable, Recorded Investment
Non-purchased and purchased performing loans	0
Commercial Loans | Substandard | Commercial real estate
Financing Receivable, Recorded Investment
Non-purchased and purchased performing loans	10,779	10,306
Commercial Loans | Substandard | Construction and development
Financing Receivable, Recorded Investment
Non-purchased and purchased performing loans	250	1,357
Commercial Loans | Substandard | Commercial and industrial
Financing Receivable, Recorded Investment
Non-purchased and purchased performing loans	2,622	600
Commercial Loans | Substandard | Equipment finance
Financing Receivable, Recorded Investment
Non-purchased and purchased performing loans	801	384
Commercial Loans | Substandard | Municipal leases
Financing Receivable, Recorded Investment
Non-purchased and purchased performing loans	0	0
Commercial Loans | Substandard | paycheck protection program [Domain]
Financing Receivable, Recorded Investment
Non-purchased and purchased performing loans	0
Commercial Loans | Doubtful | Commercial real estate
Financing Receivable, Recorded Investment
Non-purchased and purchased performing loans	0	0
Commercial Loans | Doubtful | Construction and development
Financing Receivable, Recorded Investment
Non-purchased and purchased performing loans	1	1
Commercial Loans | Doubtful | Commercial and industrial
Financing Receivable, Recorded Investment
Non-purchased and purchased performing loans	0	0
Commercial Loans | Doubtful | Equipment finance
Financing Receivable, Recorded Investment
Non-purchased and purchased performing loans	0	0
Commercial Loans | Doubtful | Municipal leases
Financing Receivable, Recorded Investment
Non-purchased and purchased performing loans	0	0
Commercial Loans | Doubtful | paycheck protection program [Domain]
Financing Receivable, Recorded Investment
Non-purchased and purchased performing loans	0
Commercial Loans | Loss | Commercial real estate
Financing Receivable, Recorded Investment
Non-purchased and purchased performing loans	16	0
Commercial Loans | Loss | Construction and development
Financing Receivable, Recorded Investment
Non-purchased and purchased performing loans	0	0
Commercial Loans | Loss | Commercial and industrial
Financing Receivable, Recorded Investment
Non-purchased and purchased performing loans	0	0
Commercial Loans | Loss | Equipment finance
Financing Receivable, Recorded Investment
Non-purchased and purchased performing loans	0	0
Commercial Loans | Loss | Municipal leases
Financing Receivable, Recorded Investment
Non-purchased and purchased performing loans	0	$ 0
Commercial Loans | Loss | paycheck protection program [Domain]
Financing Receivable, Recorded Investment
Non-purchased and purchased performing loans	$ 0