Other Assets (Details) - USD ($) $ in Thousands	Dec. 31, 2016	Dec. 31, 2015	Dec. 31, 2014
Other Assets [Abstract]
Prepaid expenses	$ 479	$ 516
Net receivable due from SBA	236	226
Unrealized gains on interest rate swap agreements	1,204	0
Prepaid Insurance	1,514	0
Other	377	932
Total other assets	5,896	3,063
Servicing Assets at Fair Value [Line Items]
Servicing assets	2,086	1,389	$ 753
SBA Servicing Assets
Servicing Assets at Fair Value [Line Items]
Servicing assets	779	515
Mortgage Servicing Assets
Servicing Assets at Fair Value [Line Items]
Servicing assets	$ 1,307	$ 874