Loans and Allowance for Loan Losses - Past Due Loans (Details) - USD ($) $ in Thousands	Sep. 30, 2016	Dec. 31, 2015
Financing Receivable Recorded Investment Past Due [Line Items]
Loans	$ 1,776,704	$ 1,680,488
Non Accruing
Financing Receivable Recorded Investment Past Due [Line Items]
Loans	9,622	10,700
30-89 Days Past Due
Financing Receivable Recorded Investment Past Due [Line Items]
Loans	5,773	4,988
90 Days or More and Accruing Interest
Financing Receivable Recorded Investment Past Due [Line Items]
Loans	1,342	887
Financing Receivables Current
Financing Receivable Recorded Investment Past Due [Line Items]
Loans	1,759,967	1,663,913
Commercial and Industrial
Financing Receivable Recorded Investment Past Due [Line Items]
Loans	369,848	316,164
Commercial and Industrial | Non Accruing
Financing Receivable Recorded Investment Past Due [Line Items]
Loans	1,219	1,659
Commercial and Industrial | 30-89 Days Past Due
Financing Receivable Recorded Investment Past Due [Line Items]
Loans	313	283
Commercial and Industrial | 90 Days or More and Accruing Interest
Financing Receivable Recorded Investment Past Due [Line Items]
Loans	264	103
Commercial and Industrial | Financing Receivables Current
Financing Receivable Recorded Investment Past Due [Line Items]
Loans	368,052	314,119
Construction
Financing Receivable Recorded Investment Past Due [Line Items]
Loans	222,803	224,166
Construction | Non Accruing
Financing Receivable Recorded Investment Past Due [Line Items]
Loans	257	272
Construction | 30-89 Days Past Due
Financing Receivable Recorded Investment Past Due [Line Items]
Loans	985	81
Construction | 90 Days or More and Accruing Interest
Financing Receivable Recorded Investment Past Due [Line Items]
Loans	201	33
Construction | Financing Receivables Current
Financing Receivable Recorded Investment Past Due [Line Items]
Loans	221,360	223,780
Residential Real Estate | 1-to-4 Family Mortgage
Financing Receivable Recorded Investment Past Due [Line Items]
Loans	284,330	284,921
Residential Real Estate | 1-to-4 Family Mortgage | Non Accruing
Financing Receivable Recorded Investment Past Due [Line Items]
Loans	2,020	2,124
Residential Real Estate | 1-to-4 Family Mortgage | 30-89 Days Past Due
Financing Receivable Recorded Investment Past Due [Line Items]
Loans	3,170	2,798
Residential Real Estate | 1-to-4 Family Mortgage | 90 Days or More and Accruing Interest
Financing Receivable Recorded Investment Past Due [Line Items]
Loans	520	268
Residential Real Estate | 1-to-4 Family Mortgage | Financing Receivables Current
Financing Receivable Recorded Investment Past Due [Line Items]
Loans	278,620	279,731
Residential Real Estate | Residential Line of Credit
Financing Receivable Recorded Investment Past Due [Line Items]
Loans	173,898	171,526
Residential Real Estate | Residential Line of Credit | Non Accruing
Financing Receivable Recorded Investment Past Due [Line Items]
Loans	644	1,114
Residential Real Estate | Residential Line of Credit | 30-89 Days Past Due
Financing Receivable Recorded Investment Past Due [Line Items]
Loans	405	1,053
Residential Real Estate | Residential Line of Credit | 90 Days or More and Accruing Interest
Financing Receivable Recorded Investment Past Due [Line Items]
Loans	194	323
Residential Real Estate | Residential Line of Credit | Financing Receivables Current
Financing Receivable Recorded Investment Past Due [Line Items]
Loans	172,655	169,036
Residential Real Estate | Multi-Family Mortgage
Financing Receivable Recorded Investment Past Due [Line Items]
Loans	54,099	59,474
Residential Real Estate | Multi-Family Mortgage | 30-89 Days Past Due
Financing Receivable Recorded Investment Past Due [Line Items]
Loans	138
Residential Real Estate | Multi-Family Mortgage | Financing Receivables Current
Financing Receivable Recorded Investment Past Due [Line Items]
Loans	53,961	59,474
Commercial Real Estate | Owner Occupied
Financing Receivable Recorded Investment Past Due [Line Items]
Loans	346,960	325,424
Commercial Real Estate | Owner Occupied | Non Accruing
Financing Receivable Recorded Investment Past Due [Line Items]
Loans	2,061	1,848
Commercial Real Estate | Owner Occupied | 30-89 Days Past Due
Financing Receivable Recorded Investment Past Due [Line Items]
Loans	180	382
Commercial Real Estate | Owner Occupied | 90 Days or More and Accruing Interest
Financing Receivable Recorded Investment Past Due [Line Items]
Loans	17
Commercial Real Estate | Owner Occupied | Financing Receivables Current
Financing Receivable Recorded Investment Past Due [Line Items]
Loans	344,702	323,194
Commercial Real Estate | Non-Owner Occupied
Financing Receivable Recorded Investment Past Due [Line Items]
Loans	259,715	231,231
Commercial Real Estate | Non-Owner Occupied | Non Accruing
Financing Receivable Recorded Investment Past Due [Line Items]
Loans	3,385	3,638
Commercial Real Estate | Non-Owner Occupied | 30-89 Days Past Due
Financing Receivable Recorded Investment Past Due [Line Items]
Loans	92	60
Commercial Real Estate | Non-Owner Occupied | Financing Receivables Current
Financing Receivable Recorded Investment Past Due [Line Items]
Loans	256,238	227,533
Consumer and Other
Financing Receivable Recorded Investment Past Due [Line Items]
Loans	65,051	67,582
Consumer and Other | Non Accruing
Financing Receivable Recorded Investment Past Due [Line Items]
Loans	36	45
Consumer and Other | 30-89 Days Past Due
Financing Receivable Recorded Investment Past Due [Line Items]
Loans	490	331
Consumer and Other | 90 Days or More and Accruing Interest
Financing Receivable Recorded Investment Past Due [Line Items]
Loans	146	160
Consumer and Other | Financing Receivables Current
Financing Receivable Recorded Investment Past Due [Line Items]
Loans	$ 64,379	$ 67,046