Condensed Parent Company Only Financial Statements - Parent Company Condensed Balance Sheets (Details) - USD ($) $ in Thousands	Jun. 30, 2020	Jun. 30, 2019	Jun. 30, 2018	Jun. 30, 2017
Cash and cash equivalents	$ 54,245	$ 35,400	$ 26,326	$ 30,786
TOTAL ASSETS	2,542,157	2,214,402
Subordinated debt (Note 10)	15,142	15,043
Stockholders' equity	258,347	238,392
TOTAL LIABILITIES AND STOCKHOLDERS' EQUITY	2,542,157	2,214,402
Parent Company
Cash and cash equivalents	4,576	8,149	$ 8,383	$ 10,856
Other Assets	13,823	13,438
Investment in common stock of Bank	255,601	234,716
TOTAL ASSETS	274,000	256,303
Accrued expenses and other liabilities	511	2,868
Subordinated debt (Note 10)	15,142	15,043
TOTAL LIABILITIES	15,653	17,911
Stockholders' equity	258,347	238,392
TOTAL LIABILITIES AND STOCKHOLDERS' EQUITY	$ 274,000	$ 256,303