Loans (Loan Portfolio by Class According to Credit Quality Indicators) (Details) - USD ($) $ in Thousands	Dec. 31, 2015	Dec. 31, 2014
Financing Receivable, Recorded Investment [Line Items]
Total loans held for investment	$ 875,844	$ 756,646
SBA, SBA 504, and Commercial Loans [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans held for investment	534,264	476,672
Small Business Administration Portfolio Segment [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans held for investment	39,393	40,401
Small Business Administration, 504 Portfolio Segment [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans held for investment	29,353	34,322
Residential and Consumer Loans [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans held for investment	341,580	279,974
Commercial Other Receivable [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans held for investment	49,332	40,607
Commercial Real Estate Portfolio Segment [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans held for investment	391,071	339,693
Commercial Real Estate Construction Financing Receivable [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans held for investment	25,115	21,649
Commercial Portfolio Segment [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans held for investment	465,518	401,949
Residential Portfolio Segment [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans held for investment	264,523	220,878
Consumer Home Equity Line Financing Receivable [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans held for investment	45,042	41,451
Consumer Other Financing Receivable [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans held for investment	32,015	17,645
Consumer Portfolio Segment [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans held for investment	77,057	59,096
Pass [Member] | SBA, SBA 504, and Commercial Loans [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans held for investment	498,476	434,960
Pass [Member] | Small Business Administration Portfolio Segment [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans held for investment	35,032	35,827
Pass [Member] | Small Business Administration, 504 Portfolio Segment [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans held for investment	24,003	24,415
Pass [Member] | Commercial Other Receivable [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans held for investment	45,870	38,054
Pass [Member] | Commercial Real Estate Portfolio Segment [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans held for investment	369,510	315,015
Pass [Member] | Commercial Real Estate Construction Financing Receivable [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans held for investment	24,061	21,649
Pass [Member] | Commercial Portfolio Segment [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans held for investment	439,441	374,718
Special Mention [Member] | SBA, SBA 504, and Commercial Loans [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans held for investment	29,969	30,042
Special Mention [Member] | Small Business Administration Portfolio Segment [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans held for investment	2,647	2,250
Special Mention [Member] | Small Business Administration, 504 Portfolio Segment [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans held for investment	4,917	5,967
Special Mention [Member] | Commercial Other Receivable [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans held for investment	2,373	1,270
Special Mention [Member] | Commercial Real Estate Portfolio Segment [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans held for investment	18,978	20,555
Special Mention [Member] | Commercial Real Estate Construction Financing Receivable [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans held for investment	1,054
Special Mention [Member] | Commercial Portfolio Segment [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans held for investment	22,405	21,825
Substandard [Member] | SBA, SBA 504, and Commercial Loans [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans held for investment	5,819	11,670
Substandard [Member] | Small Business Administration Portfolio Segment [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans held for investment	1,714	2,324
Substandard [Member] | Small Business Administration, 504 Portfolio Segment [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans held for investment	433	3,940
Substandard [Member] | Commercial Other Receivable [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans held for investment	1,089	1,283
Substandard [Member] | Commercial Real Estate Portfolio Segment [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans held for investment	2,583	4,123
Substandard [Member] | Commercial Portfolio Segment [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans held for investment	3,672	5,406
Performing Financing Receivable [Member] | Residential and Consumer Loans [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans held for investment	338,766	278,784
Performing Financing Receivable [Member] | Residential Portfolio Segment [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans held for investment	262,299	220,233
Performing Financing Receivable [Member] | Consumer Home Equity Line Financing Receivable [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans held for investment	44,452	40,908
Performing Financing Receivable [Member] | Consumer Other Financing Receivable [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans held for investment	32,015	17,643
Performing Financing Receivable [Member] | Consumer Portfolio Segment [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans held for investment	76,467	58,551
Nonperforming Financing Receivable [Member] | Residential and Consumer Loans [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans held for investment	2,814	1,190
Nonperforming Financing Receivable [Member] | Residential Portfolio Segment [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans held for investment	2,224	645
Nonperforming Financing Receivable [Member] | Consumer Home Equity Line Financing Receivable [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans held for investment	590	543
Nonperforming Financing Receivable [Member] | Consumer Other Financing Receivable [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans held for investment		2
Nonperforming Financing Receivable [Member] | Consumer Portfolio Segment [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans held for investment	$ 590	$ 545