Mortgage Loan Servicing and Loans Originated for Sale (Loans Serviced for Others) (Details) - USD ($) $ in Thousands	Jun. 30, 2019	Jun. 30, 2018
Servicing Assets at Amortized Value [Line Items]
Total loans serviced for others	$ 120,236	$ 128,409
Loans serviced for Freddie Mac
Servicing Assets at Amortized Value [Line Items]
Total loans serviced for others	18,613	19,244
Loans serviced for Fannie Mae
Servicing Assets at Amortized Value [Line Items]
Total loans serviced for others	89,910	96,384
Loans serviced for FHLB - San Francisco
Servicing Assets at Amortized Value [Line Items]
Total loans serviced for others	9,724	11,786
Loans serviced for other investors
Servicing Assets at Amortized Value [Line Items]
Total loans serviced for others	$ 1,989	$ 995