FDIC Indemnification Asset (Details) (USD $)	12 Months Ended
	Dec. 31, 2013	Dec. 31, 2012
Receivable from FDIC for Loss Share Agreements
Balance at the beginning of the period	$ 146,171,000	$ 262,651,000
Increase (decrease) in expected losses on loans	(3,452,000)	78,000
Additional losses (improvements) on OREO	(2,252,000)	7,467,000
Reimbursable expenses	5,542,000	9,262,000
Amortization of discounts and premiums, net	(29,535,000)	(20,773,000)
Reimbursements from FDIC	(51,918,000)	(112,514,000)
Balance at the end of the period	93,947,000	146,171,000
Reimbursements from FDIC	34,500,000
Subsequent event
Receivable from FDIC for Loss Share Agreements
Reimbursements from FDIC	13,100,000
First Federal
Receivable from FDIC for Loss Share Agreements
FDIC indemnification asset recorded	29,391,000
Habersham
Receivable from FDIC for Loss Share Agreements
Estimated "true-up" payment to the FDIC at the end of the loss share agreement	0
BankMeridian
Receivable from FDIC for Loss Share Agreements
Estimated "true-up" payment to the FDIC at the end of the loss share agreement	3,800,000
Plantation
Receivable from FDIC for Loss Share Agreements
Estimated "true-up" payment to the FDIC at the end of the loss share agreement	3,400,000
CBT
Receivable from FDIC for Loss Share Agreements
Estimated "true-up" payment to the FDIC at the end of the loss share agreement	0
Cape Fear
Receivable from FDIC for Loss Share Agreements
Estimated "true-up" payment to the FDIC at the end of the loss share agreement	0
Cape Fear | The Bank
Receivable from FDIC for Loss Share Agreements
Losses assumed	32,400,000
Cape Fear | Tranche one | The Bank
Receivable from FDIC for Loss Share Agreements
Percentage of eligible losses for specified amount over which the FDIC is obligated to reimburse	80.00%
Cape Fear | Tranche one | Minimum | The Bank
Receivable from FDIC for Loss Share Agreements
Amount of eligible losses over which the FDIC is obligated to reimburse the specified percentage	32,400,000
Cape Fear | Tranche one | Maximum | The Bank
Receivable from FDIC for Loss Share Agreements
Amount of eligible losses over which the FDIC is obligated to reimburse the specified percentage	110,000,000
Cape Fear | Tranche two | The Bank
Receivable from FDIC for Loss Share Agreements
Percentage of eligible losses for specified amount over which the FDIC is obligated to reimburse	95.00%
Cape Fear | Tranche two | Minimum | The Bank
Receivable from FDIC for Loss Share Agreements
Amount of eligible losses over which the FDIC is obligated to reimburse the specified percentage	110,000,000
Plantation | The Bank
Receivable from FDIC for Loss Share Agreements
Number of tranches	3
Plantation | Tranche one | The Bank
Receivable from FDIC for Loss Share Agreements
Percentage of eligible losses for specified amount over which the FDIC is obligated to reimburse	80.00%
Plantation | Tranche one | Maximum | The Bank
Receivable from FDIC for Loss Share Agreements
Amount of eligible losses over which the FDIC is obligated to reimburse the specified percentage	55,000,000
Plantation | Tranche two | Minimum | The Bank
Receivable from FDIC for Loss Share Agreements
Losses absorbed	55,000,000
Plantation | Tranche two | Maximum | The Bank
Receivable from FDIC for Loss Share Agreements
Losses absorbed	65,000,000
Plantation | Tranche three | The Bank
Receivable from FDIC for Loss Share Agreements
Percentage of eligible losses for specified amount over which the FDIC is obligated to reimburse	60.00%
Plantation | Tranche three | Minimum | The Bank
Receivable from FDIC for Loss Share Agreements
Amount of eligible losses over which the FDIC is obligated to reimburse the specified percentage	$ 65,000,000