Fair Value of Financial Instruments - Valuation techniques and inputs used (Details) - Fair Value, Inputs, Level 3 $ in Thousands	3 Months Ended
	Sep. 30, 2020 USD ($)
Private issue CMO
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Assets, Fair Value Disclosure	$ 184
Private issue CMO | Market comparable pricing | Comparability adjustment
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Impact to valuation from an increase in inputs on assets	Increase
Private issue CMO | Minimum | Market comparable pricing | Comparability adjustment
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Assets Fair Value Measurement Input	(1.30%)
Private issue CMO | Maximum | Market comparable pricing | Comparability adjustment
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Assets Fair Value Measurement Input	(2.20%)
Private issue CMO | Weighted Average | Market comparable pricing | Comparability adjustment
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Assets Fair Value Measurement Input	1.50%
Loans Held For Investment at Fair Value
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Assets, Fair Value Disclosure	$ 2,240
Loans Held For Investment at Fair Value | Minimum | Relative value analysis | Broker quotes
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Assets Fair Value Measurement Input	98.00%
Loans Held For Investment at Fair Value | Minimum | Relative value analysis | Credit risk factors
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Assets Fair Value Measurement Input	1.50%
Loans Held For Investment at Fair Value | Maximum | Relative value analysis | Broker quotes
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Assets Fair Value Measurement Input	105.70%
Loans Held For Investment at Fair Value | Maximum | Relative value analysis | Credit risk factors
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Assets Fair Value Measurement Input	100.00%
Loans Held For Investment at Fair Value | Weighted Average | Relative value analysis | Broker quotes
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Assets Fair Value Measurement Input	101.30%
Loans Held For Investment at Fair Value | Weighted Average | Relative value analysis | Credit risk factors
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Assets Fair Value Measurement Input	6.10%
Non-performing loans
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Assets, Fair Value Disclosure	$ 1,581
Non-performing loans | Discounted cash flow | Default rates.
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Impact to valuation from an increase in inputs on assets	Decrease
Assets Fair Value Measurement Input	5.00%
Mortgage servicing assets
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Assets, Fair Value Disclosure	$ 295
Mortgage servicing assets | Discounted cash flow | Discount rate
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Assets Fair Value Measurement Input	(9.10%)
Mortgage servicing assets | Minimum | Discounted cash flow | Prepayment speed (CPR)
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Assets Fair Value Measurement Input	19.50%
Mortgage servicing assets | Minimum | Discounted cash flow | Discount rate
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Assets Fair Value Measurement Input	9.00%
Mortgage servicing assets | Maximum | Discounted cash flow | Prepayment speed (CPR)
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Assets Fair Value Measurement Input	60.00%
Mortgage servicing assets | Maximum | Discounted cash flow | Discount rate
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Assets Fair Value Measurement Input	10.50%
Mortgage servicing assets | Weighted Average | Discounted cash flow | Prepayment speed (CPR)
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Assets Fair Value Measurement Input	29.60%
Interest-Only Strips
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Assets, Fair Value Disclosure	$ 13
Interest-Only Strips | Discounted cash flow | Discount rate
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Assets Fair Value Measurement Input	9.00%
Interest-Only Strips | Minimum | Discounted cash flow | Prepayment speed (CPR)
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Assets Fair Value Measurement Input	19.50%
Interest-Only Strips | Maximum | Discounted cash flow | Prepayment speed (CPR)
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Assets Fair Value Measurement Input	29.70%
Interest-Only Strips | Weighted Average | Discounted cash flow | Prepayment speed (CPR)
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Assets Fair Value Measurement Input	28.90%
Non-performing loans one
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Assets, Fair Value Disclosure	$ 1,368
Non-performing loans one | Relative value analysis | Credit risk factors
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Impact to valuation from an increase in inputs on assets	Decrease
Non-performing loans one | Minimum | Relative value analysis | Credit risk factors
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Assets Fair Value Measurement Input	20.00%
Non-performing loans one | Maximum | Relative value analysis | Credit risk factors
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Assets Fair Value Measurement Input	30.00%
Non-performing loans one | Weighted Average | Relative value analysis | Credit risk factors
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Assets Fair Value Measurement Input	20.10%