Loans and Allowance for Loan Losses - Nonaccrual and Past Due Loans (Details) (USD $) In Thousands, unless otherwise specified	Dec. 31, 2012 Loans	Dec. 31, 2011 Loans
Financing Receivable, Recorded Investment, Past Due
Financing receivables on nonaccrual status	$ 22,817	$ 70,354
Ratio of nonperforming loans to all loans	1.71%	4.78%
Total number of loans on nonaccrual status	315	505
Past due Loans, Aging Analysis
30 to 59 Days Past Due	13,943	21,009
60 to 89 Days Past Due	2,000	4,879
Greater Than 90 Days	13,362	30,866
Total Past Due	29,305	56,754
Current	1,279,067	1,393,285
Loans	1,308,372	1,450,039
Real Estate Loans | Construction Loans
Financing Receivable, Recorded Investment, Past Due
Financing receivables on nonaccrual status	7,385	26,575
Past due Loans, Aging Analysis
30 to 59 Days Past Due	4,395	2,265
60 to 89 Days Past Due	345	402
Greater Than 90 Days	1,865	15,538
Total Past Due	6,605	18,205
Current	125,376	184,598
Loans	131,981	202,803
Real Estate Loans | Owner Occupied
Financing Receivable, Recorded Investment, Past Due
Financing receivables on nonaccrual status	5,787	16,339
Past due Loans, Aging Analysis
30 to 59 Days Past Due	838	4,342
60 to 89 Days Past Due	114	1,215
Greater Than 90 Days	4,237	3,727
Total Past Due	5,189	9,284
Current	337,773	339,647
Loans	342,962	348,931
Real Estate Loans | Non-owner Occupied
Financing Receivable, Recorded Investment, Past Due
Financing receivables on nonaccrual status	1,697	7,634
Past due Loans, Aging Analysis
30 to 59 Days Past Due	1,688	9,723
60 to 89 Days Past Due	500	86
Greater Than 90 Days	1,098	4,679
Total Past Due	3,286	14,488
Current	208,203	228,339
Loans	211,489	242,827
Real Estate Loans | One to Four Family
Financing Receivable, Recorded Investment, Past Due
Financing receivables on nonaccrual status	3,123	7,271
Past due Loans, Aging Analysis
30 to 59 Days Past Due	2,767	3,441
60 to 89 Days Past Due	910	1,049
Greater Than 90 Days	2,226	2,955
Total Past Due	5,903	7,445
Current	162,708	171,602
Loans	168,611	179,047
Real Estate Loans | Multifamily
Financing Receivable, Recorded Investment, Past Due
Financing receivables on nonaccrual status	539	942
Past due Loans, Aging Analysis
30 to 59 Days Past Due	12	277
60 to 89 Days Past Due	0	0
Greater Than 90 Days	10	896
Total Past Due	22	1,173
Current	35,315	38,708
Loans	35,337	39,881
Real Estate Loans | Home Equity Lines of Credit
Financing Receivable, Recorded Investment, Past Due
Financing receivables on nonaccrual status	1,041	2,222
Past due Loans, Aging Analysis
30 to 59 Days Past Due	1,980	0
60 to 89 Days Past Due	52	634
Greater Than 90 Days	924	856
Total Past Due	2,956	1,490
Current	188,932	199,730
Loans	191,888	201,220
Commercial
Financing Receivable, Recorded Investment, Past Due
Financing receivables on nonaccrual status	2,790	8,896
Past due Loans, Aging Analysis
30 to 59 Days Past Due	2,027	586
60 to 89 Days Past Due	34	1,432
Greater Than 90 Days	2,660	2,087
Total Past Due	4,721	4,105
Current	169,719	172,942
Loans	174,440	177,047
Consumer and other
Financing Receivable, Recorded Investment, Past Due
Financing receivables on nonaccrual status	455	475
Past due Loans, Aging Analysis
30 to 59 Days Past Due	236	375
60 to 89 Days Past Due	45	61
Greater Than 90 Days	342	128
Total Past Due	623	564
Current	51,041	57,719
Loans	$ 51,664	$ 58,283