Consolidated Statements of Income (Loss) (USD $) In Thousands, except Per Share data, unless otherwise specified	12 Months Ended
	Dec. 31, 2013	Dec. 31, 2012	Dec. 31, 2011
INTEREST INCOME:
Interest and fees on loans	$ 67,604	$ 71,956	$ 79,952
Interest on federal funds sold	9	32	28
Interest and dividends on securities:
Taxable	3,556	4,683	5,946
Non-taxable	3,067	2,132	2,272
Interest-bearing deposits	63	169	348
TOTAL INTEREST INCOME	74,299	78,972	88,546
INTEREST EXPENSE
Time deposits of $100 or more	4,041	7,013	10,075
Other time and savings deposits	4,507	8,713	13,800
Borrowed funds	1,693	2,262	2,166
TOTAL INTEREST EXPENSE	10,241	17,988	26,041
NET INTEREST INCOME	64,058	60,984	62,505
PROVISION FOR (RECOVERY OF) LOAN LOSSES	(2,685)	40,374	20,843
NET INTEREST INCOME AFTER PROVISION FOR LOAN LOSSES	66,743	20,610	41,662
NON-INTEREST INCOME:
Service charges on deposit accounts	5,186	5,285	5,342
Other service fees	4,653	3,631	3,363
Net gain (loss) on sale of securities	(2,355)	1,744	2,756
Income on investment in bank-owned life insurance	599	632	657
Mortgage banking activities	7,272	6,252	6,345
(Recovery of) losses for mortgage loans sold	1,437	(392)	(641)
Other than temporary impairment of equity securities	(39)	(50)	(116)
Gain (loss) on sale of loans	575	(3,032)	0
Gain on sale of branch	310	0	0
Write-down of subsidiary held-for-sale	0	(1,019)	0
Other income	766	515	474
TOTAL NON-INTEREST INCOME	18,404	13,566	18,180
NON-INTEREST EXPENSES:
Salaries and employee benefits	30,976	26,313	27,537
Occupancy and equipment expense	7,816	7,039	8,242
Data processing	1,494	1,743	1,536
Amortization of core deposit intangible	680	1,080	1,174
Communications	1,389	1,373	1,658
FDIC assessment	2,030	2,668	3,475
Loan collection fees	739	1,079	1,571
Other professional fees	1,667	2,012	1,380
Net cost of operation of other real estate owned	(602)	13,431	5,068
Loss on sale of premises and equipment	142	131	1,494
Goodwill impairment	0	0	4,944
Other	9,589	9,942	9,464
TOTAL NON-INTEREST EXPENSES	55,920	66,811	67,543
INCOME (LOSS) BEFORE INCOME TAXES	29,227	(32,635)	(7,701)
INCOME TAX EXPENSE (BENEFIT)	10,401	(23,961)	6,694
NET INCOME (LOSS)	18,826	(8,674)	(14,395)
Preferred stock dividend and accretion of preferred stock discount	1,877	3,911	2,987
NET INCOME (LOSS) TO COMMON SHAREHOLDERS	$ 16,949	$ (12,585)	$ (17,382)
NET INCOME (LOSS) PER COMMON SHARE:
Basic	$ 1.19	$ (1.91)	$ (2.86)
Diluted	$ 1.19	$ (1.91)	$ (2.86)
CASH DIVIDENDS PER COMMON SHARE	$ 0.00	$ 0.00	$ 0.00