Consolidated Statements of Cash Flows - USD ($)	12 Months Ended
	Dec. 31, 2019	Dec. 31, 2018	Dec. 31, 2017
Cash flows from operating activities
Net (loss) income	$ (3,500,000)	$ 8,627,000	$ 8,905,000
Adjustments to reconcile net (loss) income to net cash provided by (used in) operating activities:
Provision for loan losses	1,905,000	2,300,000	900,000
Write down of other real estate owned	286,000	563,000	3,000,000
Depreciation and amortization	6,482,000	5,447,000	4,618,000
Deferred income taxes	1,744,000	1,527,000	(5,056,000)
Stock based compensation	2,632,000	2,116,000	1,842,000
Loss (gain) on sale of investment securities	(1,103,000)	67,000	146,000
Amortization of premiums on investment securities	3,730,000	2,878,000	2,469,000
Accretion of discounts on retained SBA loans	(1,411,000)	(1,332,000)	(1,088,000)
Fair value adjustments on SBA servicing assets	1,364,000	1,458,000	1,187,000
Proceeds from sales of SBA loans originated for sale	46,951,000	42,726,000	42,269,000
SBA loans originated for sale	(41,364,000)	(42,700,000)	(37,062,000)
Gains on sales of SBA loans originated for sale	(3,187,000)	(3,105,000)	(3,378,000)
Proceeds from sales of mortgage loans originated for sale	335,991,000	322,264,000	311,187,000
Mortgage loans originated for sale	(317,881,000)	(291,870,000)	(321,222,000)
Fair value adjustment for mortgage loans originated for sale	454,000	513,000	(846,000)
Gains on mortgage loans originated for sale	(8,117,000)	(8,378,000)	(8,128,000)
Amortization of intangible assets			61,000
Amortization of debt issuance costs	6,000	6,000	29,000
Non-cash expense related to leases	1,128,000
Increase in accrued interest receivable and other assets	(8,464,000)	(5,047,000)	(2,330,000)
Net increase in accrued interest payable and other liabilities	1,687,000	1,570,000	1,513,000
Net cash provided by (used in) operating activities	19,333,000	39,630,000	(984,000)
Cash flows from investing activities
Purchase of investment securities available for sale	(338,500,000)	(149,209,000)	(165,065,000)
Purchase of investment securities held to maturity		(123,265,000)	(89,350,000)
Proceeds from the sale of securities available for sale	54,742,000	6,439,000	31,197,000
Proceeds from the paydown, maturity, or call of securities available for sale	69,012,000	48,796,000	48,547,000
Proceeds from the paydown, maturity, or call of securities held to maturity	116,486,000	63,565,000	37,315,000
Net redemption (purchase) of restricted stock	3,008,000	(3,836,000)	(552,000)
Net increase in loans	(312,665,000)	(275,587,000)	(197,965,000)
Net proceeds from sale of other real estate owned	5,072,000	495,000	499,000
Premises and equipment expenditures	(35,777,000)	(18,161,000)	(22,525,000)
Net cash used in investing activities	(438,622,000)	(450,763,000)	(357,899,000)
Cash flows from financing activities
Net proceeds from exercise of stock options	261,143	670,413	646,263
Net increase in demand, money market and savings deposits	536,974,000	292,053,000	380,052,000
Net increase in time deposits	69,322,000	37,519,000	5,573,000
Increase (repayment) in short-term borrowings	(91,422,000)	91,422,000
Net cash provided by financing activities	515,135,000	421,664,000	386,271,000
Net increase in cash and cash equivalents	95,846,000	10,531,000	27,388,000
Cash and cash equivalents, beginning of year	72,473,000	61,942,000	34,554,000
Cash and cash equivalents, end of year	168,319,000	72,473,000	61,942,000
Supplemental disclosures
Interest paid	25,985,000	15,905,000	8,935,000
Income taxes paid			75,000
Non-cash transfers from loans to other real estate owned	1,225,000	315,000	291,000
Conversion of subordinated debt to common stock		10,094,000	229,000
Transfer of available-for-sale securities to held-to-maturity securities		$ 230,094,000