Income Taxes - Deferred Tax Asset and Liabilities (Details) - USD ($) $ in Thousands	Dec. 31, 2020	Dec. 31, 2019
Deferred tax asset:
Allowance for loan losses	$ 12,415	$ 9,383
Depreciation	2,174	2,355
Supplemental employee retirement plans	717	695
Non-accrual interest	610	793
Stock-based compensation expense	662	573
Lease liability	4,886	5,054
Cashflow hedge swap	791	0
Deferred fees on PPP loans	2,808	0
Other	343	334
Total	25,406	19,187
Deferred tax liability:
Goodwill	1,576	1,579
Net unrealized gains on equity securities	31	37
Net unrealized gains on debt securities	6,896	2,988
Deferred origination costs	645	725
Lease ROU asset	4,886	5,054
Other	82	72
Total	14,116	10,455
Net deferred tax asset	$ 11,290	$ 8,732