Loans and Allowance for Loan Losses - Nonaccrual and Past Due Loans (Details) (USD $) In Thousands, unless otherwise specified	Sep. 30, 2013 Loans	Dec. 31, 2012 Loans
Financing Receivable, Recorded Investment, Past Due
Financing Receivable, Recorded Investment, 90 Days Past Due and Still Accruing	$ 0	$ 0
Financing receivables on nonaccrual status	17,874	22,817
Loans and Leases Receivable, Ratio of Nonperforming Loans to All Loans	1.33%	1.71%
Total number of loans on nonaccrual status	203	315
Past due Loans, Aging Analysis
30 to 59 Days Past Due	3,419	13,943
60 to 89 Days Past Due	992	2,000
Greater Than 90 Days	5,985	13,362
Total Past Due	10,396	29,305
Current	1,322,100	1,279,067
Loans	1,332,496	1,308,372
Real Estate Loans | Construction Loans
Financing Receivable, Recorded Investment, Past Due
Financing receivables on nonaccrual status	3,464	7,385
Past due Loans, Aging Analysis
30 to 59 Days Past Due	815	4,395
60 to 89 Days Past Due	194	345
Greater Than 90 Days	1,279	1,865
Total Past Due	2,288	6,605
Current	126,662	125,376
Loans	128,950	131,981
Real Estate Loans | Owner Occupied
Financing Receivable, Recorded Investment, Past Due
Financing receivables on nonaccrual status	5,419	5,787
Past due Loans, Aging Analysis
30 to 59 Days Past Due	641	838
60 to 89 Days Past Due	0	114
Greater Than 90 Days	1,028	4,237
Total Past Due	1,669	5,189
Current	372,708	337,773
Loans	374,377	342,962
Real Estate Loans | Non-owner Occupied
Financing Receivable, Recorded Investment, Past Due
Financing receivables on nonaccrual status	2,130	1,697
Past due Loans, Aging Analysis
30 to 59 Days Past Due	176	1,688
60 to 89 Days Past Due	42	500
Greater Than 90 Days	389	1,098
Total Past Due	607	3,286
Current	200,470	208,203
Loans	201,077	211,489
Real Estate Loans | One to Four Family
Financing Receivable, Recorded Investment, Past Due
Financing receivables on nonaccrual status	3,312	3,123
Past due Loans, Aging Analysis
30 to 59 Days Past Due	553	2,767
60 to 89 Days Past Due	668	910
Greater Than 90 Days	1,717	2,226
Total Past Due	2,938	5,903
Current	168,809	162,708
Loans	171,747	168,611
Real Estate Loans | Multifamily
Financing Receivable, Recorded Investment, Past Due
Financing receivables on nonaccrual status	283	539
Past due Loans, Aging Analysis
30 to 59 Days Past Due	0	12
60 to 89 Days Past Due	0	0
Greater Than 90 Days	27	10
Total Past Due	27	22
Current	40,635	35,315
Loans	40,662	35,337
Real Estate Loans | Home Equity Lines of Credit
Financing Receivable, Recorded Investment, Past Due
Financing receivables on nonaccrual status	1,373	1,041
Past due Loans, Aging Analysis
30 to 59 Days Past Due	423	1,980
60 to 89 Days Past Due	0	52
Greater Than 90 Days	79	924
Total Past Due	502	2,956
Current	192,797	188,932
Loans	193,299	191,888
Commercial
Financing Receivable, Recorded Investment, Past Due
Financing receivables on nonaccrual status	1,622	2,790
Past due Loans, Aging Analysis
30 to 59 Days Past Due	546	2,027
60 to 89 Days Past Due	76	34
Greater Than 90 Days	1,403	2,660
Total Past Due	2,025	4,721
Current	169,006	169,719
Loans	171,031	174,440
Consumer and other
Financing Receivable, Recorded Investment, Past Due
Financing receivables on nonaccrual status	271	455
Past due Loans, Aging Analysis
30 to 59 Days Past Due	265	236
60 to 89 Days Past Due	12	45
Greater Than 90 Days	63	342
Total Past Due	340	623
Current	51,013	51,041
Loans	$ 51,353	$ 51,664