Investment Securities - Contractual Maturity - (Details) - USD ($) $ in Thousands	3 Months Ended		9 Months Ended
	Sep. 30, 2017	Sep. 30, 2016	Sep. 30, 2017	Sep. 30, 2016	Dec. 31, 2016
Amortized cost [Abstract]
Within 1 year	$ 1,798		$ 1,798
After 1 but within 5 years	7,476		7,476
After 5 but within 10 years	1,007		1,007
After 10 years	1,000		1,000
Total available for sale debt securities	153,645		153,645
Available-for-sale Equity Securities, Amortized Cost Basis	1,732		1,732
Total available for sale securities	155,377		155,377		$ 170,998
Fair value [Abstract]
Within 1 year	1,793		1,793
After 1 but within 5 years	7,558		7,558
After 5 but within 10 years	1,040		1,040
After 10 years	1,013		1,013
Total available for sale debt securities	153,776		153,776
No contractual maturity	1,732		1,732
Fair value	155,508		155,508		169,854
Proceeds from sales of AFS investment securities	32	$ 25,700	19,524	$ 36,427
U.S. Government Agencies [Member]
Amortized cost [Abstract]
Total available for sale securities	2,128		2,128		988
Fair value [Abstract]
Fair value	2,150		2,150		966
Mortgage-backed Securities-residential [Member]
Amortized cost [Abstract]
Available for sale securities Debt maturities, amortized cost basis	15,693		15,693
Total available for sale securities	15,693		15,693		6,985
Fair value [Abstract]
Available-for-sale Securities, Debt Maturities, without Single Maturity Date, Fair Value	15,850		15,850
Fair value	15,850		15,850		7,038
Collateralized Mortgage Obligations, Issued or Guaranteed by U.S. Government Agencies [Member]
Amortized cost [Abstract]
Available for sale securities Debt maturities, amortized cost basis	126,671		126,671
Total available for sale securities	126,671		126,671		146,666
Fair value [Abstract]
Available-for-sale Securities, Debt Maturities, without Single Maturity Date, Fair Value	126,522		126,522
Fair value	126,522		126,522		145,559
Mortgage-backed Securities, Issued by Private Enterprises [Member]
Amortized cost [Abstract]
Total available for sale securities					4,121
Fair value [Abstract]
Fair value					4,035
Corporate Bonds [Member]
Amortized cost [Abstract]
Total available for sale securities	1,700		1,700		1,700
Fair value [Abstract]
Fair value	1,713		1,713		1,701
Municipal Bonds [Member]
Amortized cost [Abstract]
Total available for sale securities	7,453		7,453		8,691
Fair value [Abstract]
Fair value	7,541		7,541		8,708
Private Equity Funds [Member]
Amortized cost [Abstract]
Total available for sale securities	1,706		1,706		1,821
Fair value [Abstract]
Fair value	1,706		1,706		1,821
Common Stock
Amortized cost [Abstract]
Total available for sale securities	26		26		26
Fair value [Abstract]
Fair value	$ 26		$ 26		$ 26