Investment Securities, Available-for-sale Securities, Maturities (Details) - USD ($) $ in Thousands	Dec. 31, 2017	Dec. 31, 2016
Amortized Cost [Abstract]
Due in one year or less	$ 45,783
Due in one year through five years	135,483
Due after five years through ten years	59
Due after ten years	0
Amortized Cost	580,670	$ 631,590
Fair Value [Abstract]
Due in one year or less	45,739
Due in one year through five years	133,390
Due after five years through ten years	59
Due after ten years	0
Fair Value	571,930	620,325
Mortgage-Backed Securities and Collateralized Mortgage Obligations-Residential [Member]
Amortized Cost [Abstract]
Securities, amortized cost	320,614
Amortized Cost	320,614	378,068
Fair Value [Abstract]
Securities, fair value	315,983
Fair Value	315,983	372,308
Small Business Administration-Guaranteed Participation Securities [Member]
Amortized Cost [Abstract]
Securities, amortized cost	68,921
Amortized Cost	68,921	81,026
Fair Value [Abstract]
Securities, fair value	67,059
Fair Value	67,059	78,499
Mortgage-Backed Securities and Collateralized Mortgage Obligations-Commercial [Member]
Amortized Cost [Abstract]
Securities, amortized cost	9,810
Amortized Cost	9,810	10,130
Fair Value [Abstract]
Securities, fair value	9,700
Fair Value	$ 9,700	$ 10,011