Investment Securities (Details) (USD $)	3 Months Ended
	Mar. 31, 2014		Mar. 31, 2013	Dec. 31, 2013
Carrying value and fair value of available for sale investment securities [Abstract]
Amortized cost	$ 314,172,000			$ 313,852,000
Gross unrealized gains	5,025,000	[1]		4,568,000	[1]
Gross unrealized losses	(6,985,000)	[1]		(9,693,000)	[1]
Fair value	312,212,000			308,727,000
Amortized cost [Abstract]
Within 1 year	0
After 1 but within 5 years	6,421,000
After 5 but within 10 years	36,544,000
After 10 years	43,139,000
Mortgage-backed securities-residential	31,946,000
Collateralized mortgage obligations Issued or guaranteed by U.S. government agencies	188,541,000
Collateralized mortgage obligations Non agency	4,314,000
Total available for sale debt securities	310,905,000
No contractual maturity	3,267,000
Total available for sale securities	314,172,000
Fair value [Abstract]
Within 1 year	0
After 1 but within 5 years	6,416,000
After 5 but within 10 years	35,028,000
After 10 years	40,273,000
Mortgage-backed securities residential	31,538,000
Collateralized mortgage obligations issued or guaranteed by U.S. government agencies	189,786,000
Collateralized mortgage obligations Non-agency	4,342,000
Total available for sale debt securities	307,383,000
No contractual maturity	4,829,000
Total available for sale securities	312,212,000
Proceeds from sales of AFS investment securities	0		12,135,000
Gross realized gains and losses on the sale of securities recognized in earnings [Abstract]
Gross realized gains	0		138,000
Gross realized losses	0		(93,000)
Net realized gains	0		45,000
Available for sale securities, continuous unrealized loss position [Abstract]
Less than 12 months, Fair value	130,302,000			142,071,000
12 months or longer, Fair value	50,823,000			33,664,000
Total, Fair value	181,125,000			175,735,000
Less than 12 months, Gross unrealized losses	(4,228,000)			(7,322,000)
12 months or longer, Gross unrealized losses	(2,757,000)			(2,371,000)
Total, Gross unrealized losses	(6,985,000)			(9,693,000)
Decrease in gross unrealized losses from year to year	2,700,000
Credit related impairment losses on debt securities held for which a portion of OTTI was recognized in other comprehensive income [Roll Forward]
Balance at beginning of year			173,000
Reductions for securities sold during the period (realized)			0
Reductions for securities for which the amount previously recognized in other comprehensive income was recognized in earnings because the Company intends to sell the security			0
Balance at end of year			173,000
U.S. Government Agencies [Member]
Carrying value and fair value of available for sale investment securities [Abstract]
Amortized cost	67,358,000			68,207,000
Gross unrealized gains	0	[1]		0	[1]
Gross unrealized losses	(4,299,000)	[1]		(6,171,000)	[1]
Fair value	63,059,000			62,036,000
Available for sale securities, continuous unrealized loss position [Abstract]
Less than 12 months, Fair value	43,782,000			48,919,000
12 months or longer, Fair value	19,277,000			12,267,000
Total, Fair value	63,059,000			61,186,000
Less than 12 months, Gross unrealized losses	(2,935,000)			(5,035,000)
12 months or longer, Gross unrealized losses	(1,364,000)			(1,136,000)
Total, Gross unrealized losses	(4,299,000)			(6,171,000)
Decrease in gross unrealized losses from year to year	1,900,000
Number of investment securities	21
Number of securities in an unrealized loss position for less than twelve months	14
Number of securities in an unrealized loss position for more than twelve months	7
Mortgage-backed Securities-residential [Member]
Carrying value and fair value of available for sale investment securities [Abstract]
Amortized cost	31,946,000			32,769,000
Gross unrealized gains	227,000	[1]		210,000	[1]
Gross unrealized losses	(635,000)	[1]		(882,000)	[1]
Fair value	31,538,000			32,097,000
Available for sale securities, continuous unrealized loss position [Abstract]
Less than 12 months, Fair value	12,204,000			18,045,000
12 months or longer, Fair value	9,390,000			6,276,000
Total, Fair value	21,594,000			24,321,000
Less than 12 months, Gross unrealized losses	(284,000)			(518,000)
12 months or longer, Gross unrealized losses	(351,000)			(364,000)
Total, Gross unrealized losses	(635,000)			(882,000)
Decrease in gross unrealized losses from year to year	247,000
Number of investment securities	8
Number of securities in an unrealized loss position for less than twelve months	5
Number of securities in an unrealized loss position for more than twelve months	3
Collateralized Mortgage Obligations, Issued or Guaranteed by U.S. Government Agencies [Member]
Carrying value and fair value of available for sale investment securities [Abstract]
Amortized cost	188,541,000			188,194,000
Gross unrealized gains	3,031,000	[1]		2,887,000	[1]
Gross unrealized losses	(1,786,000)	[1]		(1,978,000)	[1]
Fair value	189,786,000			189,103,000
Available for sale securities, continuous unrealized loss position [Abstract]
Less than 12 months, Fair value	71,677,000			67,240,000
12 months or longer, Fair value	16,568,000			9,974,000
Total, Fair value	88,245,000			77,214,000
Less than 12 months, Gross unrealized losses	(953,000)			(1,446,000)
12 months or longer, Gross unrealized losses	(833,000)			(532,000)
Total, Gross unrealized losses	(1,786,000)			(1,978,000)
Decrease in gross unrealized losses from year to year	192,000
Number of investment securities	28
Number of securities in an unrealized loss position for less than twelve months	23
Number of securities in an unrealized loss position for more than twelve months	5
Collateralized Mortgage Obligations, Non-agency [Member]
Carrying value and fair value of available for sale investment securities [Abstract]
Amortized cost	4,314,000			4,454,000
Gross unrealized gains	28,000	[1]		25,000	[1]
Gross unrealized losses	0	[1]		0	[1]
Fair value	4,342,000			4,479,000
Available for sale securities, continuous unrealized loss position [Abstract]
Number of investment securities	2
Corporate Bonds [Member]
Carrying value and fair value of available for sale investment securities [Abstract]
Amortized cost	9,561,000			9,669,000
Gross unrealized gains	61,000	[1]		25,000	[1]
Gross unrealized losses	(67,000)	[1]		(256,000)	[1]
Fair value	9,555,000			9,438,000
Available for sale securities, continuous unrealized loss position [Abstract]
Less than 12 months, Fair value	953,000			4,848,000
12 months or longer, Fair value	980,000			965,000
Total, Fair value	1,933,000			5,813,000
Less than 12 months, Gross unrealized losses	(47,000)			(221,000)
12 months or longer, Gross unrealized losses	(20,000)			(35,000)
Total, Gross unrealized losses	(67,000)			(256,000)
Number of investment securities	2
Number of securities in an unrealized loss position for less than twelve months	1
Number of securities in an unrealized loss position for more than twelve months	1
Number of securities at or above investment grade	2
Number of securities considered not to be other than temporarily impaired	2
Municipal Bonds [Member]
Carrying value and fair value of available for sale investment securities [Abstract]
Amortized cost	9,185,000			7,163,000
Gross unrealized gains	16,000	[1]		0	[1]
Gross unrealized losses	(98,000)	[1]		(263,000)	[1]
Fair value	9,103,000			6,900,000
Available for sale securities, continuous unrealized loss position [Abstract]
Less than 12 months, Fair value	1,686,000			3,019,000
12 months or longer, Fair value	4,269,000			3,881,000
Total, Fair value	5,955,000			6,900,000
Less than 12 months, Gross unrealized losses	(9,000)			(102,000)
12 months or longer, Gross unrealized losses	(89,000)			(161,000)
Total, Gross unrealized losses	(98,000)			(263,000)
Number of investment securities	8
Number of securities in an unrealized loss position for less than twelve months	3
Number of securities in an unrealized loss position for more than twelve months	5
Common Stocks [Member]
Carrying value and fair value of available for sale investment securities [Abstract]
Amortized cost	33,000			33,000
Gross unrealized gains	16,000	[1]		16,000	[1]
Gross unrealized losses	0	[1]		0	[1]
Fair value	49,000			49,000
Available for sale securities, continuous unrealized loss position [Abstract]
Number of investment securities	2
Other Securities [Member]
Carrying value and fair value of available for sale investment securities [Abstract]
Amortized cost	3,234,000			3,363,000
Gross unrealized gains	1,646,000	[1]		1,405,000	[1]
Gross unrealized losses	(100,000)	[1]		(143,000)	[1]
Fair value	4,780,000			4,625,000
Available for sale securities, continuous unrealized loss position [Abstract]
Less than 12 months, Fair value	0			0
12 months or longer, Fair value	339,000			301,000
Total, Fair value	339,000			301,000
Less than 12 months, Gross unrealized losses	0			0
12 months or longer, Gross unrealized losses	(100,000)			(143,000)
Total, Gross unrealized losses	$ (100,000)			$ (143,000)
Number of securities invested in real estate funds	7
Number of securities considered not to be other than temporarily impaired	2
Number of securities recording an impairment charge	2

[1] Accumulated other comprehensive loss