Income Taxes (Deferred Tax Assets and Liabilities) (Details) - USD ($) $ in Thousands	Jun. 30, 2019	Jun. 30, 2018
Deferred Tax Assets [Abstract]
Loss reserves	$ 2,685	$ 2,873
Non-accrued interest	483	502
Deferred compensation	2,396	2,509
Accrued vacation	124	224
Depreciation	95	99
Litigation reserves	876	1,441
Other	588	358
Total deferred tax assets	7,247	8,006
Deferred Tax Liabilities [Abstract]
FHLB - San Francisco stock dividends	(664)	(664)
Unrealized gain on derivative financial instruments, at fair value		(123)
Prepaid expenses	(56)	(49)
Unrealized gain on investment securities	(63)	(82)
Unrealized gain on interest-only strips	(5)	(6)
Deferred loan costs	(2,723)	(2,806)
State tax	(197)	(108)
Total deferred tax liabilities	(3,708)	(3,838)
Total net deferred tax assets	$ 3,539	$ 4,168