Allowance for Loan and Lease Losses - Changes in Allowance for Loan and Lease Losses (Details) (USD $) In Thousands, unless otherwise specified	3 Months Ended		12 Months Ended
	Mar. 31, 2014	Mar. 31, 2013	Dec. 31, 2013
Allowance for Loan and Lease Losses [Roll Forward]
Beginning balance	$ 13,671	$ 17,261	$ 17,261
Charge-offs	(1,185)	(1,985)	(3,893)
Recoveries	19	364	1,175
(Credit) provision	(639)	(251)	(872)
Ending balance	11,866	15,389	13,671
Ending balance: related to loans individually evaluated for impairment	138	155	886
Ending balance: related to loans collectively evaluated for impairment	11,728	15,234	12,785
LHFI [Abstract]
Ending balance	358,658	358,055	366,481
Ending balance: individually evaluated for impairment	16,898	22,753	18,506
Ending balance: collectively evaluated for impairment	341,760	335,302	347,975
Commercial Real Estate [Member]
Allowance for Loan and Lease Losses [Roll Forward]
Beginning balance	5,498	8,750	8,750
Charge-offs	(350)	(835)	(1,684)
Recoveries	0	104	600
(Credit) provision	(220)	84	(2,168)
Ending balance	4,928	8,103	5,498
Ending balance: related to loans individually evaluated for impairment	0	0	331
Ending balance: related to loans collectively evaluated for impairment	4,928	8,103	5,167
LHFI [Abstract]
Ending balance	150,111	175,397	148,293
Ending balance: individually evaluated for impairment	4,800	8,484	5,325
Ending balance: collectively evaluated for impairment	145,311	166,913	142,968
Construction and Land Development [Member]
Allowance for Loan and Lease Losses [Roll Forward]
Beginning balance	2,316	2,987	2,987
Charge-offs	0	(820)	(820)
Recoveries	0	95	297
(Credit) provision	(53)	(184)	(148)
Ending balance	2,263	2,078	2,316
Ending balance: related to loans individually evaluated for impairment	0	0	0
Ending balance: related to loans collectively evaluated for impairment	2,263	2,078	2,316
LHFI [Abstract]
Ending balance	45,751	34,533	45,261
Ending balance: individually evaluated for impairment	3,446	4,466	3,907
Ending balance: collectively evaluated for impairment	42,305	30,067	41,354
Commercial and Industrial [Member]
Allowance for Loan and Lease Losses [Roll Forward]
Beginning balance	3,006	1,924	1,924
Charge-offs	(452)	(173)	(383)
Recoveries	4	4	17
(Credit) provision	(696)	958	1,448
Ending balance	1,862	2,713	3,006
Ending balance: related to loans individually evaluated for impairment	0	45	452
Ending balance: related to loans collectively evaluated for impairment	1,862	2,668	2,554
LHFI [Abstract]
Ending balance	68,664	67,512	79,589
Ending balance: individually evaluated for impairment	6,630	8,750	8,049
Ending balance: collectively evaluated for impairment	62,034	58,762	71,540
Multi Family [Member]
Allowance for Loan and Lease Losses [Roll Forward]
Beginning balance	402	654	654
Charge-offs	0	0	0
Recoveries	0	0	0
(Credit) provision	(1)	4	(252)
Ending balance	401	658	402
Ending balance: related to loans individually evaluated for impairment	0	0	0
Ending balance: related to loans collectively evaluated for impairment	401	658	402
LHFI [Abstract]
Ending balance	11,876	11,830	11,737
Ending balance: individually evaluated for impairment	0	0	0
Ending balance: collectively evaluated for impairment	11,876	11,830	11,737
Residential Real Estate [Member]
Allowance for Loan and Lease Losses [Roll Forward]
Beginning balance	473	1,098	1,098
Charge-offs	0	0	(46)
Recoveries	2	151	158
(Credit) provision	15	(1,081)	(737)
Ending balance	490	168	473
Ending balance: related to loans individually evaluated for impairment	22	13	19
Ending balance: related to loans collectively evaluated for impairment	468	155	454
LHFI [Abstract]
Ending balance	26,465	6,697	25,535
Ending balance: individually evaluated for impairment	746	431	632
Ending balance: collectively evaluated for impairment	25,719	6,266	24,903
Leases [Member]
Allowance for Loan and Lease Losses [Roll Forward]
Beginning balance	1,223	1,108	1,108
Charge-offs	(118)	(147)	(382)
Recoveries	13	0	29
(Credit) provision	239	100	468
Ending balance	1,357	1,061	1,223
Ending balance: related to loans individually evaluated for impairment	116	70	60
Ending balance: related to loans collectively evaluated for impairment	1,241	991	1,163
LHFI [Abstract]
Ending balance	43,644	38,629	42,524
Ending balance: individually evaluated for impairment	139	71	139
Ending balance: collectively evaluated for impairment	43,505	38,558	42,385
Tax Certificates [Member]
Allowance for Loan and Lease Losses [Roll Forward]
Beginning balance	555	472	472
Charge-offs	(265)	(10)	(578)
Recoveries	0	10	74
(Credit) provision	97	(88)	587
Ending balance	387	384	555
Ending balance: related to loans individually evaluated for impairment	0	27	24
Ending balance: related to loans collectively evaluated for impairment	387	357	531
LHFI [Abstract]
Ending balance	11,182	22,541	12,716
Ending balance: individually evaluated for impairment	1,137	551	454
Ending balance: collectively evaluated for impairment	10,045	21,990	12,262
Consumer [Member]
Allowance for Loan and Lease Losses [Roll Forward]
Beginning balance	15	29	29
Charge-offs	0	0	0
Recoveries	0	0	0
(Credit) provision	2	(11)	(14)
Ending balance	17	18	15
Ending balance: related to loans individually evaluated for impairment	0	0	0
Ending balance: related to loans collectively evaluated for impairment	17	18	15
LHFI [Abstract]
Ending balance	965	916	826
Ending balance: individually evaluated for impairment	0	0	0
Ending balance: collectively evaluated for impairment	965	916	826
Unallocated [Member]
Allowance for Loan and Lease Losses [Roll Forward]
Beginning balance	183	239	239
Charge-offs	0	0	0
Recoveries	0	0	0
(Credit) provision	(22)	(33)	(56)
Ending balance	161	206	183
Ending balance: related to loans individually evaluated for impairment	0	0	0
Ending balance: related to loans collectively evaluated for impairment	161	206	183
LHFI [Abstract]
Ending balance	0	0	0
Ending balance: individually evaluated for impairment	0	0	0
Ending balance: collectively evaluated for impairment	$ 0	$ 0	$ 0