Consolidated Statements of Operations (USD $)	12 Months Ended
	Dec. 31, 2014	Dec. 31, 2013	Dec. 31, 2012
Interest income:
Interest and fees on taxable loans	$ 34,530,000	$ 31,986,000	$ 32,326,000
Interest and fees on tax-exempt loans	339,000	349,000	265,000
Interest and dividends on taxable investment securities	5,053,000	4,435,000	4,899,000
Interest and dividends on tax-exempt investment securities	364,000	250,000	470,000
Interest on federal funds sold and other interest-earning assets	187,000	185,000	300,000
Total interest income	40,473,000	37,205,000	38,260,000
Interest expense:
Demand- interest bearing	888,000	825,000	796,000
Money market and savings	1,929,000	1,786,000	2,718,000
Time deposits	719,000	867,000	1,718,000
Other borrowings	1,108,000	1,112,000	1,134,000
Total interest expense	4,644,000	4,590,000	6,366,000
Net interest income	35,829,000	32,615,000	31,894,000
Provision for loan losses	900,000	4,935,000	1,350,000
Net interest income after provision for loan losses	34,929,000	27,680,000	30,544,000
Non-interest income:
Loan advisory and servicing fees	1,452,000	1,615,000	1,251,000
Gain on sales of SBA loans	4,717,000	5,338,000	5,531,000
Service fees on deposit accounts	1,224,000	1,046,000	922,000
Legal settlements		238,000	155,000
Gain on sale of investment securities	458,000	703,000	737,000
Other-than-temporary impairment	21,000		(35,000)
Portion recognized in other comprehensive income (before taxes)	(28,000)		1,000
Net impairment loss on investment securities	(7,000)	0	(34,000)
Bank owned life insurance income		13,000	73,000
Other non-interest income	173,000	263,000	193,000
Total non-interest income	8,017,000	9,216,000	8,828,000
Non-interest expenses:
Salaries and employee benefits	20,089,000	17,064,000	16,512,000
Occupancy	4,247,000	3,635,000	3,454,000
Depreciation and amortization	2,382,000	2,105,000	2,006,000
Legal	1,290,000	1,878,000	2,966,000
Other real estate owned	1,794,000	3,179,000	763,000
Advertising	597,000	447,000	307,000
Data processing	1,345,000	1,000,000	1,187,000
Insurance	586,000	625,000	654,000
Professional fees	1,468,000	1,420,000	1,106,000
Regulatory assessments and costs	1,065,000	1,257,000	1,367,000
Taxes, other	616,000	557,000	594,000
Legal settlement		1,875,000
Other operating expenses	5,071,000	5,369,000	4,986,000
Total non-interest expense	40,550,000	40,411,000	35,902,000
Income (loss) before benefit for income taxes	2,396,000	(3,515,000)	3,470,000
Benefit for income taxes	(46,000)	(35,000)	(144,000)
Net income (loss)	$ 2,442,000	$ (3,480,000)	$ 3,614,000
Net income (loss) per share:
Basic (in Dollars per share)	$ 0.07	$ (0.13)	$ 0.14
Diluted (in Dollars per share)	$ 0.07	$ (0.13)	$ 0.14