Income Taxes - Deferred Tax Assets and Liabilities (Details) - USD ($) $ in Thousands	Dec. 31, 2019	Dec. 31, 2018
Deferred tax assets:
Allowance for loan losses	$ 2,934	$ 2,882
Foreclosed assets	200	704
Deferred compensation	3,895	3,538
Accrued reserves	312	421
Non accrual loans	181	205
Net operating loss carryforward	1,909	2,131
Net unrealized loss on securities available-for-sale	0	2,798
Other	3,536	3,510
Total deferred tax assets	12,967	16,189
Deferred tax liabilities:
Premises and equipment	(324)	(833)
Deferred loan costs	(2,656)	(2,656)
Net unrealized gain on securities available-for-sale	(2,490)	0
Other	(4,034)	(4,046)
Total deferred tax liabilities	(9,504)	(7,535)
Net deferred tax assets	$ 3,463	$ 8,654