Impaired Loans and Valuation Allowance for Loan Losses (Details 5) (USD $) In Thousands, unless otherwise specified	Jun. 30, 2013	Jun. 30, 2012
Loan portfolio and related allowance
Individually Evaluated for Impairment, Carrying Value	$ 62,878	$ 103,324
Collectively Evaluated for Impairment, Carrying Value	1,203,015	1,118,030
Carrying Value, Total	1,265,893	1,221,354
Individually Evaluated for Impairment, Associated Allowance	5,359	9,273
Collectively Evaluated for Impairment, Associated Allowance	14,719	15,362
Associated Allowance, Total	20,078	24,635
Construction and land development [Member]
Loan portfolio and related allowance
Individually Evaluated for Impairment, Carrying Value	3,848	4,871
Collectively Evaluated for Impairment, Carrying Value	57,268	52,357
Carrying Value, Total	61,116	57,228
Individually Evaluated for Impairment, Associated Allowance	175	449
Collectively Evaluated for Impairment, Associated Allowance	702	906
Associated Allowance, Total	877	1,355
Commercial real estate [Member]
Loan portfolio and related allowance
Individually Evaluated for Impairment, Carrying Value	21,156	55,178
Collectively Evaluated for Impairment, Carrying Value	492,442	438,438
Carrying Value, Total	513,598	493,616
Individually Evaluated for Impairment, Associated Allowance	1,054	4,615
Collectively Evaluated for Impairment, Associated Allowance	5,671	7,362
Associated Allowance, Total	6,725	11,977
Residential real estate [Member]
Loan portfolio and related allowance
Individually Evaluated for Impairment, Carrying Value	37,162	42,219
Collectively Evaluated for Impairment, Carrying Value	544,216	521,716
Carrying Value, Total	581,378	563,935
Individually Evaluated for Impairment, Associated Allowance	4,006	4,099
Collectively Evaluated for Impairment, Associated Allowance	7,147	6,213
Associated Allowance, Total	11,153	10,312
Commercial and financial [Member]
Loan portfolio and related allowance
Individually Evaluated for Impairment, Carrying Value	0	40
Collectively Evaluated for Impairment, Carrying Value	65,224	56,180
Carrying Value, Total	65,224	56,220
Individually Evaluated for Impairment, Associated Allowance	0	1
Collectively Evaluated for Impairment, Associated Allowance	563	328
Associated Allowance, Total	563	329
Consumer [Member]
Loan portfolio and related allowance
Individually Evaluated for Impairment, Carrying Value	712	1,016
Collectively Evaluated for Impairment, Carrying Value	43,865	49,339
Carrying Value, Total	44,577	50,355
Individually Evaluated for Impairment, Associated Allowance	124	109
Collectively Evaluated for Impairment, Associated Allowance	636	553
Associated Allowance, Total	$ 760	$ 662