Securities - Amortized Cost and Fair Value of Securities Available for Sale and Held for Investment (Details) - USD ($) $ in Thousands	Dec. 31, 2017	Dec. 31, 2016
Amortized cost and fair value of securities available for sale and held for investment
Gross Amortized Cost, Available for Sale	$ 960,220	$ 960,809
Gross Unrealized Gains, Available for Sale	4,202	3,767
Gross Unrealized Losses, Available for Sale	(8,618)	(14,073)
Fair Value, Available for Sale	955,804	950,503
Gross Amortized Cost, Held for Investment Securities	416,863	372,498
Gross Unrealized Gains, Held for Investment Securities	1,819	1,753
Gross Unrealized Losses, Held for Investment Securities	(4,212)	(4,370)
Fair Value, Held for Investment Securities	414,470	369,881
U.S. Treasury Securities and obligations of U.S. Government Sponsored Entities [Member]
Amortized cost and fair value of securities available for sale and held for investment
Gross Amortized Cost, Available for Sale	9,475	12,073
Gross Unrealized Gains, Available for Sale	274	255
Gross Unrealized Losses, Available for Sale	(5)	0
Fair Value, Available for Sale	9,744	12,328
Mortgage-backed securities of U.S. Government Sponsored Entities [Member]
Amortized cost and fair value of securities available for sale and held for investment
Gross Amortized Cost, Available for Sale	318,771	287,726
Gross Unrealized Gains, Available for Sale	891	585
Gross Unrealized Losses, Available for Sale	(3,306)	(4,823)
Fair Value, Available for Sale	316,356	283,488
Gross Amortized Cost, Held for Investment Securities	172,261	159,941
Gross Unrealized Gains, Held for Investment Securities	746	704
Gross Unrealized Losses, Held for Investment Securities	(1,392)	(1,243)
Fair Value, Held for Investment Securities	171,615	159,402
Collateralized Mortgage Obligations Of US Government Sponsored Entities [Member]
Amortized cost and fair value of securities available for sale and held for investment
Gross Amortized Cost, Available for Sale	235,466	238,805
Gross Unrealized Gains, Available for Sale	272	314
Gross Unrealized Losses, Available for Sale	(4,694)	(5,065)
Fair Value, Available for Sale	231,044	234,054
Gross Amortized Cost, Held for Investment Securities	181,280	147,208
Gross Unrealized Gains, Held for Investment Securities	56	386
Gross Unrealized Losses, Held for Investment Securities	(2,767)	(2,630)
Fair Value, Held for Investment Securities	178,569	144,964
Commercial mortgage backed securities of U.S. Government Sponsored Entities [Member]
Amortized cost and fair value of securities available for sale and held for investment
Gross Amortized Cost, Available for Sale	16,210	22,351
Gross Unrealized Gains, Available for Sale	165	222
Gross Unrealized Losses, Available for Sale	(34)	(28)
Fair Value, Available for Sale	16,341	22,545
Gross Amortized Cost, Held for Investment Securities	17,462	17,375
Gross Unrealized Gains, Held for Investment Securities	705	233
Gross Unrealized Losses, Held for Investment Securities	0	(74)
Fair Value, Held for Investment Securities	18,167	17,534
Private mortgage backed securities [Member]
Amortized cost and fair value of securities available for sale and held for investment
Gross Amortized Cost, Available for Sale	18,056	32,780
Gross Unrealized Gains, Available for Sale	384	0
Gross Unrealized Losses, Available for Sale	0	(791)
Fair Value, Available for Sale	18,440	31,989
Gross Amortized Cost, Held for Investment Securities		6,427
Gross Unrealized Gains, Held for Investment Securities		0
Gross Unrealized Losses, Held for Investment Securities		(109)
Fair Value, Held for Investment Securities	0	6,318
Private mortgage backed securities [Member]
Amortized cost and fair value of securities available for sale and held for investment
Gross Amortized Cost, Available for Sale	47,045	67,542
Gross Unrealized Gains, Available for Sale	605	563
Gross Unrealized Losses, Available for Sale	(285)	(816)
Fair Value, Available for Sale	47,365	67,289
Gross Amortized Cost, Held for Investment Securities	5,337
Gross Unrealized Gains, Held for Investment Securities	9
Gross Unrealized Losses, Held for Investment Securities	(53)
Fair Value, Held for Investment Securities	5,293
Collateralized loan obligations [Member]
Amortized cost and fair value of securities available for sale and held for investment
Gross Amortized Cost, Available for Sale	263,579	124,716
Gross Unrealized Gains, Available for Sale	798	838
Gross Unrealized Losses, Available for Sale	(68)	(665)
Fair Value, Available for Sale	264,309	124,889
Gross Amortized Cost, Held for Investment Securities	40,523	41,547
Gross Unrealized Gains, Held for Investment Securities	303	430
Gross Unrealized Losses, Held for Investment Securities	0	(314)
Fair Value, Held for Investment Securities	40,826	41,663
Obligations of state and political subdivisions [Member]
Amortized cost and fair value of securities available for sale and held for investment
Gross Amortized Cost, Available for Sale	45,118	63,161
Gross Unrealized Gains, Available for Sale	813	622
Gross Unrealized Losses, Available for Sale	(70)	(895)
Fair Value, Available for Sale	45,861	62,888
Corporate and other debt securities [Member]
Amortized cost and fair value of securities available for sale and held for investment
Gross Amortized Cost, Available for Sale	6,500	74,121
Gross Unrealized Gains, Available for Sale	0	257
Gross Unrealized Losses, Available for Sale	(156)	(517)
Fair Value, Available for Sale	6,344	73,861
Fair Value, Held for Investment Securities	$ 0
Private commercial mortgage backed securities [Member]
Amortized cost and fair value of securities available for sale and held for investment
Gross Amortized Cost, Available for Sale		37,534
Gross Unrealized Gains, Available for Sale		111
Gross Unrealized Losses, Available for Sale		(473)
Fair Value, Available for Sale		$ 37,172