Income Taxes (Net Deferred Tax Asset) (Details) - USD ($) $ in Thousands	Dec. 31, 2019	Dec. 31, 2018
Deferred tax assets:
Allowance for loan losses and off-balance-sheet credit exposure	$ 8,803	$ 9,312
Operating lease liability	4,547	239
Unrealized loss on interest rate swaps	1,323	341
Stock-based compensation	1,317	1,384
Accrued bonuses and severance	1,308	686
Contract incentive	719	680
Asset writedown	121	80
Net operating loss carryforwards	77	371
Retirement expense	60	81
Interest on nonperforming loans	20	16
Unrealized losses on available-for-sale securities		1,274
Total deferred tax assets gross	18,295	14,464
Total deferred tax assets net	18,295	14,464
Deferred tax liabilities:
Prepaid pension	5,462	4,545
Right-of-use asset	4,285
Deferred loan costs	4,138	5,462
Unrealized gains on available-for-sale securities	2,967
Depreciation	1,050	951
Prepaid expenses	76	59
Total deferred income tax liabilities	17,978	11,017
Net deferred tax asset	$ 317	$ 3,447