Mortgage Loan Servicing and Loans Originated for Sale - Loans Serviced for Others (Details) - USD ($) $ in Thousands	Jun. 30, 2021	Jun. 30, 2020
Servicing Assets at Amortized Value
Loans serviced for others	$ 50,448	$ 86,505
Freddie Mac [Member]
Servicing Assets at Amortized Value
Loans serviced for others	9,658	14,210
Fannie Mae [Member]
Servicing Assets at Amortized Value
Loans serviced for others	35,486	64,910
FHLB - San Francisco [Member]
Servicing Assets at Amortized Value
Loans serviced for others	$ 5,304	$ 7,385