Other Assets (Details) - USD ($) $ in Thousands	Dec. 31, 2019	Dec. 31, 2018	Dec. 31, 2017	Dec. 31, 2016
Servicing Assets at Fair Value [Line Items]
Right-of-use assets	$ 2,792
Servicing assets	2,026	$ 2,375	$ 1,800	$ 2,086
Prepaid expenses	969	1,061
Prepaid insurance	733	1,755
Unrealized gains on interest rate swap agreements	261	1,432
Net receivable due from SBA	127	151
Other	687	1,861
Total other assets	7,595	8,635
Unrealized gains on interest rate swap agreements
Servicing Assets at Fair Value [Line Items]
Servicing assets	920	1,024
Mortgage servicing asset
Servicing Assets at Fair Value [Line Items]
Servicing assets	$ 1,106	$ 1,351