Loans (Details 8) (USD $) In Thousands, unless otherwise specified	3 Months Ended
	Mar. 31, 2013	Mar. 31, 2012
Change in allowance for loan losses during the period
Balance at the beginning of the period	$ 19,948	$ 18,540
Provision charged to expense	750	1,750
Charge-offs	350	1,010
Recoveries	195	24
Balance, end of period	20,543	19,304
Commercial
Change in allowance for loan losses during the period
Balance at the beginning of the period	6,394	5,545
Provision charged to expense	1,145	432
Charge-offs	150	73
Recoveries	193	18
Balance, end of period	7,582	5,922
Real estate construction
Change in allowance for loan losses during the period
Balance at the beginning of the period	673	1,220
Provision charged to expense	46	365
Recoveries	2	2
Balance, end of period	721	1,587
Land loans
Change in allowance for loan losses during the period
Balance at the beginning of the period	333	613
Provision charged to expense	(14)	(109)
Recoveries	1	4
Balance, end of period	320	508
Real estate other
Change in allowance for loan losses during the period
Balance at the beginning of the period	5,178	6,111
Provision charged to expense	(857)	51
Balance, end of period	4,321	6,162
Factoring and asset based
Change in allowance for loan losses during the period
Balance at the beginning of the period	4,352	2,381
Provision charged to expense	579	1,104
Charge-offs	200	750
Balance, end of period	4,731	2,735
SBA
Change in allowance for loan losses during the period
Balance at the beginning of the period	2,905	2,567
Provision charged to expense	(161)	(80)
Charge-offs		187
Recoveries	(1)
Balance, end of period	2,743	2,300
Other
Change in allowance for loan losses during the period
Balance at the beginning of the period	113	103
Provision charged to expense	12	(13)
Balance, end of period	$ 125	$ 90