Supplemental Disclosures for Consolidated Statements of Cash Flows - Reconciliation of Net Income (Loss) to Net Cash Provided by Operating Activities (Details) - USD ($) $ in Thousands	12 Months Ended
	Dec. 31, 2015	Dec. 31, 2014	Dec. 31, 2013
Net income	$ 22,141	$ 5,696	$ 51,989
Adjustments to reconcile net income to net cash (used) provided by operating activities
Depreciation	3,773	3,268	2,776
Net amortization of premiums and discounts on securities	3,920	2,353	3,882
Accretion of purchase accounting loan discount	(5,152)	(750)	0
Other amortization and accretion	(2,791)	494	(172)
Change in loans available for sale, net	(11,920)	1,754	22,189
Provision (recpature) for loan losses, net	2,644	(3,486)	3,188
Deferred tax benefit	0	0	(40,552)
Gain on sale of securities	(161)	(469)	(419)
Gain on sale of loans	(702)	(419)	(455)
Loss on sale or write down of foreclosed assets	239	310	1,295
Loss on branch closures and disposition of equipment	183	4,493	1
Stock based employee benefit expense	127	142	190
Earnings on bank owned lif insurance	(1,426)	(252)	0
Change in interest receivable	(903)	(2,763)	160
Change in interest payable	(682)	847	(27)
Change in prepaid expenses	(1,201)	(591)	4,562
Change in accrued taxes	12,990	4,294	(102)
Other, net	(2,486)	2,954	792
Change in other liabilities	238	2,660	499
Net cash provided (used) by operating activities	22,989	21,946	49,852
Supplemental disclosure of non cash investing activities
Fair value adjustment to securities	(2,256)	8,985	(21,957)
Transfers from loans to other real estate owned	5,255	4,789	5,087
Transfers from loans to loans available for sale	0	0	379
Securities principal receivable recorded in other assets	230	101	159
Transfer from securities held for investment to available for sale	0	0	13,818
Transfer from securities available for sale to held for investment	0	158,781	0
Purchase of securities under trade date accounting	$ 28,343	$ 0	$ 0