Loans and Allowance for Credit Losses - Classes of loans (Details) $ in Thousands	12 Months Ended
	Jun. 30, 2023 USD ($)	Jun. 30, 2022 USD ($)	Jun. 30, 2021 USD ($)
Accounts, Notes, Loans and Financing Receivable [Line Items]
Real Estate Loans	$ 3,978,393	$ 2,843,499
Loans in Process	(359,196)	(123,656)
Deferred loan fees, net	(299)	(453)
Allowance for credit losses	(47,820)	(33,192)	$ (33,222)
Total loans	$ 3,571,078	$ 2,686,198
Number of purchased participation loans	86	31
Purchased participation loans	$ 155,600	$ 70,000
Residential Real Estate
Accounts, Notes, Loans and Financing Receivable [Line Items]
Real Estate Loans	1,133,417	904,160
Allowance for credit losses	(15,641)	(8,908)	(11,192)
Construction Real Estate
Accounts, Notes, Loans and Financing Receivable [Line Items]
Real Estate Loans	550,052	258,072
Allowance for credit losses	(2,664)	(2,220)	(2,170)
Commercial Real Estate
Accounts, Notes, Loans and Financing Receivable [Line Items]
Real Estate Loans	1,562,379	1,146,673
Allowance for credit losses	(22,838)	(16,838)	(14,535)
Consumer loans
Accounts, Notes, Loans and Financing Receivable [Line Items]
Real Estate Loans	133,515	92,996
Allowance for credit losses	(909)	(710)	(916)
Commercial loans
Accounts, Notes, Loans and Financing Receivable [Line Items]
Real Estate Loans	599,030	441,598
Allowance for credit losses	$ (5,768)	$ (4,516)	$ (4,409)