Fair Value of Financial Instruments - Schedule of Reconciliation of Beginning and Ending Balances of Recurring Fair Value Measurements Using Level 3 Inputs (Details) - USD ($) $ in Thousands	12 Months Ended
	Jun. 30, 2022	Jun. 30, 2021
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Fair Value, Asset, Recurring Basis, Unobservable Input Reconciliation, Gain (Loss), Statement of Income or Comprehensive Income [Extensible Enumeration]	Revenue from Contract with Customer, Including Assessed Tax	Revenue from Contract with Customer, Including Assessed Tax
Fair Value, Liability, Recurring Basis, Unobservable Input Reconciliation, Liability, Gain (Loss), Statement of Other Comprehensive Income or Comprehensive Income [Extensible Enumeration]	Other Comprehensive Income (Loss), Securities, Available-for-sale, Adjustment, before Tax	Other Comprehensive Income (Loss), Securities, Available-for-sale, Adjustment, before Tax
Fair Value, Inputs, Level 3
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Beginning balance	$ 2,038	$ 2,469
Total gains or losses (realized/unrealized) Included in earnings	(113)	51
Total gains or losses (realized/unrealized) Included in other comprehensive income (loss)	(10)	5
Purchases	0	0
Issuances	0	0
Settlements	(399)	(487)
Transfers in and/or out of Level 3	0	0
Ending balance	1,516	2,038
Private issue CMO | Fair Value, Inputs, Level 3
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Beginning balance	154	197
Total gains or losses (realized/unrealized) Included in earnings	0	0
Total gains or losses (realized/unrealized) Included in other comprehensive income (loss)	(7)	9
Purchases	0	0
Issuances	0	0
Settlements	(34)	(52)
Transfers in and/or out of Level 3	0	0
Ending balance	113	154
Loans Held For Investment, at Fair Value [Member] | Fair Value, Inputs, Level 3
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Beginning balance	1,874	2,258
Total gains or losses (realized/unrealized) Included in earnings	(113)	51
Total gains or losses (realized/unrealized) Included in other comprehensive income (loss)	0	0
Purchases	0	0
Issuances	0	0
Settlements	(365)	(435)
Transfers in and/or out of Level 3	0	0
Ending balance	1,396	1,874
Interest-Only Strips | Fair Value, Inputs, Level 3
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Beginning balance	10	14
Total gains or losses (realized/unrealized) Included in earnings	0	0
Total gains or losses (realized/unrealized) Included in other comprehensive income (loss)	(3)	(4)
Purchases	0	0
Issuances	0	0
Settlements	0	0
Transfers in and/or out of Level 3	0	0
Ending balance	$ 7	$ 10