Parent Company Financial Information (Details) (USD $) In Thousands, unless otherwise specified	12 Months Ended
	Dec. 31, 2013	Dec. 31, 2012	Dec. 31, 2011	Dec. 31, 2010
Assets [Abstract]
Investment in subsidiary	$ 1,094	$ 1,430
Total Assets	897,669	831,483
Liabilities and stockholders' equity [Abstract]
Stockholders' equity	84,908	92,325	87,702	79,596
Total Liabilities and Stockholders' Equity	897,669	831,483
Statements of Income [Abstract]
Income tax benefit (expense)	(2,854)	(1,723)	(1,132)
Net income	5,384	4,646	2,664
Statement of Cash Flows [Abstract]
Net income	5,384	4,646	2,664
Adjustments to reconcile net income (loss) to net cash provided by operating activities [Abstract]
Net cash provided by operating activities	16,220	14,334	8,133
Cash flows from financing activities [Abstract]
Issuance of preferred stock	0	0	22,847
Dividend on preferred stock	(677)	(1,139)	(953)
Common stock issued and stock based compensation	76	93	105
Net cash provided by financing activities	62,371	43,804	39,405
Net increase in cash and cash equivalents	15,895	21,187	465
Cash and cash equivalents at beginning of year	161,359	140,172	139,707
Cash and cash equivalents at end of year	177,254	161,359	140,172
Parent Company [Member]
Assets [Abstract]
Cash	3,982	4,638
Investment in subsidiary	81,070	87,972
Total Assets	85,052	92,610
Liabilities and stockholders' equity [Abstract]
Other liabilities	144	285
Stockholders' equity	84,908	92,325
Total Liabilities and Stockholders' Equity	85,052	92,610
Statements of Income [Abstract]
Dividends from subsidiary	10,000	0	0
Other operating expenses	(113)	(106)	(101)
Income tax benefit (expense)	46	44	41
Income (loss) before undistributed earnings of subsidiary	9,933	(62)	(60)
Equity in undistributed earnings of subsidiary	(4,549)	4,708	2,724
Net income	5,384	4,646	2,664
Statement of Cash Flows [Abstract]
Net income	5,384	4,646	2,664
Adjustments to reconcile net income (loss) to net cash provided by operating activities [Abstract]
Decrease in other assets	0	0	2
(Decrease) increase in other liabilities	(141)	(54)	228
Equity in undistributed earnings of subsidiary	4,549	(4,708)	(2,724)
Net cash provided by operating activities	9,792	(116)	170
Cash flows from financing activities [Abstract]
Issuance of preferred stock	0	0	22,847
Redemption of preferred stock	(10,000)	0	(17,390)
Redemption of common stock warrants	0	0	(375)
Dividend on preferred stock	(677)	(1,139)	(953)
Common stock issued and stock based compensation	233	208	257
Cash in lieu of fractional shares	(4)	(3)	0
Net cash provided by financing activities	(10,448)	(934)	4,386
Net increase in cash and cash equivalents	(656)	(1,050)	4,556
Cash and cash equivalents at beginning of year	4,638	5,688	1,132
Cash and cash equivalents at end of year	$ 3,982	$ 4,638	$ 5,688