Investment Securities - Contractual Maturity - (Details) - USD ($) $ in Thousands	3 Months Ended
	Mar. 31, 2017	Mar. 31, 2016	Dec. 31, 2016
Amortized cost [Abstract]
Within 1 year	$ 1,638
After 1 but within 5 years	6,852
After 5 but within 10 years	3,012
After 10 years	1,000
Total available for sale debt securities	162,077
Available-for-sale Equity Securities, Amortized Cost Basis	1,847
Total available for sale securities	163,924		$ 170,998
Fair value [Abstract]
Within 1 year	1,625
After 1 but within 5 years	6,904
After 5 but within 10 years	3,013
After 10 years	1,025
Total available for sale debt securities	161,890
No contractual maturity	1,847
Fair value	163,737		169,854
Proceeds from sales of AFS investment securities	0	$ 10,022
U.S. Government Agencies [Member]
Amortized cost [Abstract]
Total available for sale securities	2,126		988
Fair value [Abstract]
Fair value	2,100		966
Mortgage-backed Securities-residential [Member]
Amortized cost [Abstract]
Available for sale securities Debt maturities, amortized cost basis	6,527
Total available for sale securities	6,527		6,985
Fair value [Abstract]
Available-for-sale Securities, Debt Maturities, without Single Maturity Date, Fair Value	6,662
Fair value	6,662		7,038
Collateralized Mortgage Obligations, Issued or Guaranteed by U.S. Government Agencies [Member]
Amortized cost [Abstract]
Available for sale securities Debt maturities, amortized cost basis	139,296
Total available for sale securities	139,296		146,666
Fair value [Abstract]
Available-for-sale Securities, Debt Maturities, without Single Maturity Date, Fair Value	138,980
Fair value	138,980		145,559
Mortgage-backed Securities, Issued by Private Enterprises [Member]
Amortized cost [Abstract]
Available for sale securities Debt maturities, amortized cost basis	3,752
Total available for sale securities	3,752		4,121
Fair value [Abstract]
Available-for-sale Securities, Debt Maturities, without Single Maturity Date, Fair Value	3,681
Fair value	3,681		4,035
Corporate Bonds [Member]
Amortized cost [Abstract]
Total available for sale securities	1,700		1,700
Fair value [Abstract]
Fair value	1,725		1,701
Municipal Bonds [Member]
Amortized cost [Abstract]
Total available for sale securities	8,676		8,691
Fair value [Abstract]
Fair value	8,742		8,708
Private Equity Funds [Member]
Amortized cost [Abstract]
Total available for sale securities	1,821		1,821
Fair value [Abstract]
Fair value	1,821		1,821
Common Stock
Amortized cost [Abstract]
Total available for sale securities	26		26
Fair value [Abstract]
Fair value	$ 26		$ 26