FDIC Indemnification Asset (Details) - USD ($) $ in Thousands	12 Months Ended
	Dec. 31, 2015	Dec. 31, 2014	Dec. 31, 2013	Apr. 01, 2015	Dec. 31, 2012
Receivable from FDIC for Loss Share Agreements
Balance at the beginning of period	$ 22,161	$ 86,447	$ 146,171
FDIC indemnification asset recorded for First Federal’s loss share agreements			21,891
Decrease in expected losses on loans	(788)	(2,421)	(3,452)
Additional recoveries on OREO	(5,440)	(4,520)	(2,252)
Reimbursable expenses	789	2,829	5,542
Amortization of discounts and premiums, net	(8,587)	(21,895)	(29,535)
Reimburesements from FDIC	(3,734)	(38,279)	(51,918)
Balance at the end of period	4,401	22,161	86,447
Reimbursements from FDIC	8,600
Carrying value of commercial loans				$ 49,000
Carrying value of other real estate owned	30,554	$ 42,726	$ 64,918	$ 2,200	$ 66,505
Habersham
Receivable from FDIC for Loss Share Agreements
Estimated "true-up" payment to the FDIC at the end of the loss share agreement	0
BankMeridian
Receivable from FDIC for Loss Share Agreements
Estimated "true-up" payment to the FDIC at the end of the loss share agreement	4,200
Plantation
Receivable from FDIC for Loss Share Agreements
Estimated "true-up" payment to the FDIC at the end of the loss share agreement	3,600
CBT
Receivable from FDIC for Loss Share Agreements
Estimated "true-up" payment to the FDIC at the end of the loss share agreement	0
Cape Fear
Receivable from FDIC for Loss Share Agreements
Estimated "true-up" payment to the FDIC at the end of the loss share agreement	$ 0