Fair Value of Financial Instruments - Schedule of Reconciliation of Beginning and Ending Balances of Recurring Fair Value Measurements Using Level 3 Inputs (Details) - Fair Value, Inputs, Level 3 - USD ($) $ in Thousands	3 Months Ended
	Sep. 30, 2022	Sep. 30, 2021
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Beginning balance	$ 1,516	$ 2,038
Total gains or losses (realized/unrealized) Included in earnings	(26)	2
Total gains or losses (realized/unrealized) Included in other comprehensive income (loss)	(1)	0
Purchases	0	0
Issuances	0	0
Settlements	(25)	(304)
Transfers in and/or out of Level 3	0	0
Ending balance	1,464	1,736
Private issue CMO
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Beginning balance	113	154
Total gains or losses (realized/unrealized) Included in earnings	0	0
Total gains or losses (realized/unrealized) Included in other comprehensive income (loss)	(1)	1
Purchases	0	0
Issuances	0	0
Settlements	(5)	(5)
Transfers in and/or out of Level 3	0	0
Ending balance	107	150
Loans Held For Investment at Fair Value
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Beginning balance	1,396	1,874
Total gains or losses (realized/unrealized) Included in earnings	(26)	2
Total gains or losses (realized/unrealized) Included in other comprehensive income (loss)	0	0
Purchases	0	0
Issuances	0	0
Settlements	(20)	(299)
Transfers in and/or out of Level 3	0	0
Ending balance	1,350	1,577
Interest-Only Strips
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Beginning balance	7	10
Total gains or losses (realized/unrealized) Included in earnings	0	0
Total gains or losses (realized/unrealized) Included in other comprehensive income (loss)	0	(1)
Purchases	0	0
Issuances	0	0
Settlements	0	0
Transfers in and/or out of Level 3	0	0
Ending balance	$ 7	$ 9