Loans (Aging Analysis of Past Due and Nonaccrual Loans by Class) (Details) - USD ($) $ in Thousands	Dec. 31, 2016	Dec. 31, 2015
Financing Receivable, Recorded Investment [Line Items]
Total past due	$ 13,279	$ 13,628
Nonaccrual	7,237	7,260
Current	960,135	875,330
Total loans held for investment, current	945,362	862,216
Total	958,641	875,844
SBA loans held for sale	14,773	13,114
Total loans	973,414	888,958
Nonaccrual loans, TDRs	153	293
Performing troubled debt restructurings		3,000
SBA loans held for investment
Financing Receivable, Recorded Investment [Line Items]
Total past due	2,056	3,373
Nonaccrual	1,168	1,764
Current	40,436	36,020
Total	42,492	39,393
SBA 504 loans
Financing Receivable, Recorded Investment [Line Items]
Total past due	513	518
Nonaccrual	513	518
Current	25,831	28,835
Total	26,344	29,353
Commercial loans
Financing Receivable, Recorded Investment [Line Items]
Total	509,171	465,518
Commercial loans | Other
Financing Receivable, Recorded Investment [Line Items]
Total past due	75	167
Nonaccrual	25	10
Current	58,372	49,165
Total	58,447	49,332
Commercial loans | Commercial real estate
Financing Receivable, Recorded Investment [Line Items]
Total past due	2,083	2,881
Nonaccrual	401	2,154
Current	420,335	388,190
Total	422,418	391,071
Commercial loans | Commercial real estate construction
Financing Receivable, Recorded Investment [Line Items]
Total past due	0	356
Nonaccrual	0	0
Current	28,306	24,759
Total	28,306	25,115
Residential mortgage loans
Financing Receivable, Recorded Investment [Line Items]
Total past due	5,867	5,609
Nonaccrual	2,672	2,224
Current	283,226	258,914
Total	289,093	264,523
Consumer loans
Financing Receivable, Recorded Investment [Line Items]
Total	91,541	77,057
Consumer loans | Home equity
Financing Receivable, Recorded Investment [Line Items]
Total past due	709	723
Nonaccrual	482	590
Current	46,702	44,319
Total	47,411	45,042
Consumer loans | Other
Financing Receivable, Recorded Investment [Line Items]
Total past due	1,976	1
Nonaccrual	1,976	0
Current	42,154	32,014
Total	44,130	32,015
SBA [Member]
Financing Receivable, Recorded Investment [Line Items]
Nonaccrual	60	288
30-59 days past due
Financing Receivable, Recorded Investment [Line Items]
Total past due	4,808	4,548
30-59 days past due | SBA loans held for investment
Financing Receivable, Recorded Investment [Line Items]
Total past due	491	1,153
30-59 days past due | SBA 504 loans
Financing Receivable, Recorded Investment [Line Items]
Total past due	0	0
30-59 days past due | Commercial loans | Other
Financing Receivable, Recorded Investment [Line Items]
Total past due	50	157
30-59 days past due | Commercial loans | Commercial real estate
Financing Receivable, Recorded Investment [Line Items]
Total past due	1,108	444
30-59 days past due | Commercial loans | Commercial real estate construction
Financing Receivable, Recorded Investment [Line Items]
Total past due	0	356
30-59 days past due | Residential mortgage loans
Financing Receivable, Recorded Investment [Line Items]
Total past due	2,932	2,307
30-59 days past due | Consumer loans | Home equity
Financing Receivable, Recorded Investment [Line Items]
Total past due	227	130
30-59 days past due | Consumer loans | Other
Financing Receivable, Recorded Investment [Line Items]
Total past due	0	1
60-89 days past due
Financing Receivable, Recorded Investment [Line Items]
Total past due	1,234	1,820
60-89 days past due | SBA loans held for investment
Financing Receivable, Recorded Investment [Line Items]
Total past due	397	456
60-89 days past due | SBA 504 loans
Financing Receivable, Recorded Investment [Line Items]
Total past due	0	0
60-89 days past due | Commercial loans | Other
Financing Receivable, Recorded Investment [Line Items]
Total past due	0	0
60-89 days past due | Commercial loans | Commercial real estate
Financing Receivable, Recorded Investment [Line Items]
Total past due	574	283
60-89 days past due | Commercial loans | Commercial real estate construction
Financing Receivable, Recorded Investment [Line Items]
Total past due	0	0
60-89 days past due | Residential mortgage loans
Financing Receivable, Recorded Investment [Line Items]
Total past due	263	1,078
60-89 days past due | Consumer loans | Home equity
Financing Receivable, Recorded Investment [Line Items]
Total past due	0	3
60-89 days past due | Consumer loans | Other
Financing Receivable, Recorded Investment [Line Items]
Total past due	0	0
90 days and still accruing
Financing Receivable, Recorded Investment [Line Items]
Total past due	0	0
90 days and still accruing | SBA loans held for investment
Financing Receivable, Recorded Investment [Line Items]
Total past due	0	0
90 days and still accruing | SBA 504 loans
Financing Receivable, Recorded Investment [Line Items]
Total past due	0	0
90 days and still accruing | Commercial loans | Other
Financing Receivable, Recorded Investment [Line Items]
Total past due	0	0
90 days and still accruing | Commercial loans | Commercial real estate
Financing Receivable, Recorded Investment [Line Items]
Total past due	0	0
90 days and still accruing | Commercial loans | Commercial real estate construction
Financing Receivable, Recorded Investment [Line Items]
Total past due	0	0
90 days and still accruing | Residential mortgage loans
Financing Receivable, Recorded Investment [Line Items]
Total past due	0	0
90 days and still accruing | Consumer loans | Home equity
Financing Receivable, Recorded Investment [Line Items]
Total past due	0	0
90 days and still accruing | Consumer loans | Other
Financing Receivable, Recorded Investment [Line Items]
Total past due	$ 0	$ 0