Mortgage Loan Servicing and Loans Originated for Sale (Loans Serviced for Others) (Details) (USD $) In Thousands, unless otherwise specified	Jun. 30, 2013	Jun. 30, 2012	Jun. 30, 2011
Servicing Assets at Amortized Value [Line Items]
Loans serviced for others	$ 92,156	$ 98,875	$ 109,351
Freddie Mac [Member]
Servicing Assets at Amortized Value [Line Items]
Loans serviced for others	4,160	4,727	3,269
Fannie Mae [Member]
Servicing Assets at Amortized Value [Line Items]
Loans serviced for others	34,023	24,063	16,791
FHLB - San Francisco [Member]
Servicing Assets at Amortized Value [Line Items]
Loans serviced for others	52,096	68,013	87,022
Other Investors
Servicing Assets at Amortized Value [Line Items]
Loans serviced for others	$ 1,877	$ 2,072	$ 2,269