Consolidated Statements of Operations (USD $) In Thousands, except Per Share data, unless otherwise specified	12 Months Ended
	Dec. 31, 2013	Dec. 31, 2012	Dec. 31, 2011
Interest income:
Interest and fees on loans	$ 45,304	$ 48,832	$ 61,604
Interest on investments	5,436	5,839	4,961
Other investment income	245	208	535
Total interest income	50,985	54,879	67,100
Interest expense:
Interest bearing demand	732	1,051	2,100
Savings	22	23	55
Time	1,045	2,017	5,559
Other borrowings	970	1,908	3,990
Total interest expense	2,769	4,999	11,704
Net Interest Income (Loss) after Provision for Loan Losses
Net interest income	48,216	49,880	55,396
Loan loss provision	1,000	1,100	75,000
Net interest income (loss) after loan loss provision	47,216	48,780	(19,604)
Non-interest income:
Service charges on deposit accounts	3,031	3,244	4,493
Card issuer and merchant services fees, net	3,310	2,632	2,478
Earnings on BOLI	862	1,022	1,213
Mortgage banking income, net	4,261	4,319	513
Other income	2,989	1,874	2,270
Total non-interest income	14,453	13,091	10,967
Non-interest expense:
Salaries and employee benefits	32,651	31,559	31,434
Occupancy	4,931	4,598	4,710
Information Technology and Data Processing	2,488	1,842	1,971
Equipment	1,583	1,547	1,583
Communications	1,496	1,541	1,653
FDIC insurance	1,542	2,519	3,271
OREO	529	1,725	17,936
Professional services	4,249	3,999	4,356
Increase (decrease) in reserve for unfunded commitments	0	(1,110)	609
CDI Impairment	0	0	3,436
Prepayment penalties on FHLB and TLGP borrowings	3,827	0	1,291
Other expenses	7,674	7,621	10,949
Total non-interest expense	60,970	55,841	83,199
Income (loss) before income taxes and extraordinary net gain	699	6,030	(91,836)
Income tax benefit (provision)	50,146	(79)	11,721
Income (Loss) from Operations before Extraordinary Items	50,845	5,951	(80,115)
Extraordinary Gain on Extinguishment of Junior Subordinate Debentures, Net of Income Taxes	0	0	32,839
Net income (loss)	$ 50,845	$ 5,951	$ (47,276)
Basic and diluted income (loss) per share:
Income (loss) before extraordinary net gain	$ 1.08	$ 0.13	$ (1.83)
Extraordinary net gain	$ 0.00	$ 0.00	$ 0.75
Net income (loss) per common share (basic)	$ 1.08	$ 0.13	$ (1.08)
Net income per common share (diluted)	$ 1.07	$ 0.13	$ (1.08)