Loans and Allowance for Loan Losses - Nonaccrual and Past Due Loans (Details) (USD $) In Thousands, unless otherwise specified	Jun. 30, 2012 Loans	Dec. 31, 2011 Loans
Financing Receivable, Recorded Investment, Past Due
Financing receivables on nonaccrual status	$ 63,305	$ 70,354
Ratio of nonperforming loans to all loans	4.53%	4.78%
Total number of loans on nonaccrual status	459	505
Past due Loans, Aging Analysis
30 to 59 Days Past Due	9,350	21,009
60 to 89 Days Past Due	4,073	4,879
Greater Than 90 Days	25,503	30,866
Total Past Due	38,926	56,754
Current	1,332,314	1,393,285
Loans	1,371,240	1,450,039
Real Estate Loans | Construction Loans
Financing Receivable, Recorded Investment, Past Due
Financing receivables on nonaccrual status	20,582	26,575
Past due Loans, Aging Analysis
30 to 59 Days Past Due	309	2,265
60 to 89 Days Past Due	1,736	402
Greater Than 90 Days	9,604	15,538
Total Past Due	11,649	18,205
Current	178,191	184,598
Loans	189,840	202,803
Real Estate Loans | Owner Occupied
Financing Receivable, Recorded Investment, Past Due
Financing receivables on nonaccrual status	16,310	16,339
Past due Loans, Aging Analysis
30 to 59 Days Past Due	970	4,342
60 to 89 Days Past Due	269	1,215
Greater Than 90 Days	3,654	3,727
Total Past Due	4,893	9,284
Current	330,214	339,647
Loans	335,107	348,931
Real Estate Loans | Non-owner Occupied
Financing Receivable, Recorded Investment, Past Due
Financing receivables on nonaccrual status	6,652	7,634
Past due Loans, Aging Analysis
30 to 59 Days Past Due	2,040	9,723
60 to 89 Days Past Due	0	86
Greater Than 90 Days	4,462	4,679
Total Past Due	6,502	14,488
Current	209,788	228,339
Loans	216,290	242,827
Real Estate Loans | One to Four Family
Financing Receivable, Recorded Investment, Past Due
Financing receivables on nonaccrual status	6,891	7,271
Past due Loans, Aging Analysis
30 to 59 Days Past Due	1,879	3,441
60 to 89 Days Past Due	1,281	1,049
Greater Than 90 Days	2,716	2,955
Total Past Due	5,876	7,445
Current	168,598	171,602
Loans	174,474	179,047
Real Estate Loans | Multifamily
Financing Receivable, Recorded Investment, Past Due
Financing receivables on nonaccrual status	720	942
Past due Loans, Aging Analysis
30 to 59 Days Past Due	278	277
60 to 89 Days Past Due	0	0
Greater Than 90 Days	439	896
Total Past Due	717	1,173
Current	35,983	38,708
Loans	36,700	39,881
Real Estate Loans | Home Equity Lines of Credit
Financing Receivable, Recorded Investment, Past Due
Financing receivables on nonaccrual status	2,856	2,222
Past due Loans, Aging Analysis
30 to 59 Days Past Due	2,511	0
60 to 89 Days Past Due	378	634
Greater Than 90 Days	1,422	856
Total Past Due	4,311	1,490
Current	192,236	199,730
Loans	196,547	201,220
Commercial
Financing Receivable, Recorded Investment, Past Due
Financing receivables on nonaccrual status	8,745	8,896
Past due Loans, Aging Analysis
30 to 59 Days Past Due	933	586
60 to 89 Days Past Due	285	1,432
Greater Than 90 Days	2,947	2,087
Total Past Due	4,165	4,105
Current	163,675	172,942
Loans	167,840	177,047
Consumer and other
Financing Receivable, Recorded Investment, Past Due
Financing receivables on nonaccrual status	549	475
Past due Loans, Aging Analysis
30 to 59 Days Past Due	430	375
60 to 89 Days Past Due	124	61
Greater Than 90 Days	259	128
Total Past Due	813	564
Current	53,629	57,719
Loans	$ 54,442	$ 58,283