Fair Value - Carrying Amount and Fair Values (Details) - USD ($) $ in Thousands	Mar. 31, 2017	Dec. 31, 2016
Financial assets:
Accrued interest receivable	$ 4,483	$ 4,462
Financial liabilities:
Accrued interest payable	405	430
Estimated fair value	20,729	20,968
Impaired collateral-dependent loans	2,619	1,767
Corporate and other securities
Financial liabilities:
Estimated fair value	6,574	6,560
Fair Value, Inputs, Level 1
Financial liabilities:
Impaired collateral-dependent loans	0	0
Fair Value, Inputs, Level 2
Financial liabilities:
Impaired collateral-dependent loans	0	0
Fair Value, Inputs, Level 3
Financial liabilities:
Impaired collateral-dependent loans	2,619	1,767
Carrying amount | Fair Value, Inputs, Level 1
Financial assets:
Cash and cash equivalents	103,204	105,895
Carrying amount | Fair Value, Inputs, Level 2
Financial assets:
Securities	73,022	61,547
SBA loans held for sale	12,163	14,773
Loans, net of allowance for loan losses	975,833	946,062
FHLB stock	5,992	6,037
Accrued interest receivable	4,483	4,462
Financial liabilities:
Deposits	980,703	945,723
Borrowed funds and subordinated debentures	130,310	131,310
Accrued interest payable	405	430
Carrying amount | Fair Value, Inputs, Level 3
Financial assets:
Servicing assets	2,150	2,086
OREO	1,172	1,050
Carrying amount | Fair Value, Inputs, Level 3 | Corporate and other securities
Financial liabilities:
Estimated fair value	3,800	3,800
Estimated fair value | Fair Value, Inputs, Level 1
Financial assets:
Cash and cash equivalents	103,204	105,895
Estimated fair value | Fair Value, Inputs, Level 2
Financial assets:
Securities	72,975	61,536
SBA loans held for sale	13,665	16,440
Loans, net of allowance for loan losses	975,009	944,772
FHLB stock	5,992	6,037
Accrued interest receivable	4,483	4,462
Financial liabilities:
Deposits	979,793	944,886
Borrowed funds and subordinated debentures	129,834	130,319
Accrued interest payable	405	430
Estimated fair value | Fair Value, Inputs, Level 3
Financial assets:
Servicing assets	2,150	2,086
OREO	1,172	1,050
Financial liabilities:
Impaired collateral-dependent loans	2,600	1,800
Estimated fair value | Fair Value, Inputs, Level 3 | Corporate and other securities
Financial liabilities:
Estimated fair value	$ 3,600	$ 3,600