Loans and Allowance for Loan Losses - Allowance for Loan Losses (Details) (USD $)	3 Months Ended		9 Months Ended		12 Months Ended
	Sep. 30, 2012	Sep. 29, 2011	Sep. 30, 2012	Sep. 30, 2011	Jun. 30, 2012	Jun. 30, 2011	Dec. 31, 2011
Financing Receivable, Allowance for Credit Losses [Roll Forward]
Allowance for loan losses, beginning balance	$ 28,797,000	$ 35,652,000	$ 32,848,000	$ 37,752,000	$ 35,652,000
Charge-offs	6,237,000	5,024,000	16,524,000	23,982,000	25,300,000	39,800,000
Recoveries	420,000	1,089,000	2,087,000	2,687,000
Provision	4,251,000	1,956,000	8,820,000	17,216,000
Allowance for loan losses, ending balance	27,231,000		27,231,000	33,673,000	28,797,000	35,652,000
Financing Receivable, Allowance for Credit Loss, Additional Information
Reserves for loans individually evaluated for impairment	3,791,000		3,791,000				4,101,000
Loans individually evaluated for impairment	58,122,000		58,122,000				77,896,000
Reserves for loans collectively evaluated for impairment	23,440,000		23,440,000				28,747,000
Loans collectively evaluated for impairment	1,299,518,000		1,299,518,000				1,372,143,000
Loans sold during period	94,000,000		288,600,000
Impact of change in allowance methodology	1,400,000
Reserve for Off-balance Sheet Activities
Financing Receivable, Allowance for Credit Loss, Additional Information
Reserves	201,000		201,000				221,000
Reserve for Losses on Mortgage Loans Sold
Financing Receivable, Allowance for Credit Loss, Additional Information
Reserves	1,700,000		1,700,000				1,800,000
Charges against reserves	115,238
Provision expense	204,861	18,000	234,576
Real Estate Loans | Construction Loans
Financing Receivable, Allowance for Credit Losses [Roll Forward]
Allowance for loan losses, beginning balance	5,903,000	10,770,000	8,214,000	12,014,000	10,770,000
Charge-offs	3,881,000	1,175,000	7,823,000	9,050,000
Recoveries	45,000	630,000	323,000	1,710,000
Provision	3,675,000	(390,000)	5,028,000	5,161,000
Allowance for loan losses, ending balance	5,742,000		5,742,000	9,835,000
Financing Receivable, Allowance for Credit Loss, Additional Information
Reserves for loans individually evaluated for impairment	1,249,000		1,249,000				570,000
Loans individually evaluated for impairment	18,025,000		18,025,000				29,078,000
Reserves for loans collectively evaluated for impairment	4,493,000		4,493,000				7,644,000
Loans collectively evaluated for impairment	129,383,000		129,383,000				173,725,000
Real Estate Loans | Owner Occupied
Financing Receivable, Allowance for Credit Losses [Roll Forward]
Allowance for loan losses, beginning balance	5,630,000	6,701,000	5,792,000	5,958,000	6,701,000
Charge-offs	146,000	546,000	2,019,000	3,382,000
Recoveries	2,000	3,000	126,000	109,000
Provision	(943,000)	(506,000)	644,000	2,967,000
Allowance for loan losses, ending balance	4,543,000		4,543,000	5,652,000
Financing Receivable, Allowance for Credit Loss, Additional Information
Reserves for loans individually evaluated for impairment	279,000		279,000				585,000
Loans individually evaluated for impairment	19,268,000		19,268,000				24,045,000
Reserves for loans collectively evaluated for impairment	4,264,000		4,264,000				5,207,000
Loans collectively evaluated for impairment	327,640,000		327,640,000				324,886,000
Real Estate Loans | Non-owner Occupied
Financing Receivable, Allowance for Credit Losses [Roll Forward]
Allowance for loan losses, beginning balance	3,582,000	4,280,000	4,668,000	7,150,000	4,280,000
Charge-offs	697,000	1,101,000	2,461,000	3,434,000
Recoveries	256,000	0	289,000	52,000
Provision	1,096,000	707,000	1,741,000	118,000
Allowance for loan losses, ending balance	4,237,000		4,237,000	3,886,000
Financing Receivable, Allowance for Credit Loss, Additional Information
Reserves for loans individually evaluated for impairment	476,000		476,000				143,000
Loans individually evaluated for impairment	7,535,000		7,535,000				9,614,000
Reserves for loans collectively evaluated for impairment	3,761,000		3,761,000				4,525,000
Loans collectively evaluated for impairment	223,319,000		223,319,000				233,213,000
Real Estate Loans | One to Four Family
Financing Receivable, Allowance for Credit Losses [Roll Forward]
Allowance for loan losses, beginning balance	3,063,000	4,514,000	3,726,000	3,706,000	4,514,000
Charge-offs	57,000	584,000	625,000	2,364,000
Recoveries	4,000	84,000	153,000	161,000
Provision	(83,000)	223,000	(327,000)	2,734,000
Allowance for loan losses, ending balance	2,927,000		2,927,000	4,237,000
Financing Receivable, Allowance for Credit Loss, Additional Information
Reserves for loans individually evaluated for impairment	155,000		155,000				45,000
Loans individually evaluated for impairment	1,871,000		1,871,000				4,341,000
Reserves for loans collectively evaluated for impairment	2,773,000		2,773,000				3,681,000
Loans collectively evaluated for impairment	172,841,000		172,841,000				174,706,000
Real Estate Loans | Multifamily
Financing Receivable, Allowance for Credit Losses [Roll Forward]
Allowance for loan losses, beginning balance	563,000	526,000	805,000	424,000	526,000
Charge-offs	0	0	213,000	11,000
Recoveries	0	0	0	0
Provision	5,000	48,000	(24,000)	161,000
Allowance for loan losses, ending balance	568,000		568,000	574,000
Financing Receivable, Allowance for Credit Loss, Additional Information
Reserves for loans individually evaluated for impairment	0		0				135,000
Loans individually evaluated for impairment	701,000		701,000				595,000
Reserves for loans collectively evaluated for impairment	567,000		567,000				670,000
Loans collectively evaluated for impairment	37,286,000		37,286,000				39,286,000
Real Estate Loans | Home Equity Lines of Credit
Financing Receivable, Allowance for Credit Losses [Roll Forward]
Allowance for loan losses, beginning balance	4,399,000	4,012,000	3,310,000	3,298,000	4,012,000
Charge-offs	372,000	973,000	1,073,000	2,659,000
Recoveries	29,000	17,000	50,000	97,000
Provision	(162,000)	867,000	1,607,000	3,187,000
Allowance for loan losses, ending balance	3,894,000		3,894,000	3,923,000
Financing Receivable, Allowance for Credit Loss, Additional Information
Reserves for loans individually evaluated for impairment	0		0				112,000
Loans individually evaluated for impairment	1,706,000		1,706,000				928,000
Reserves for loans collectively evaluated for impairment	3,894,000		3,894,000				3,198,000
Loans collectively evaluated for impairment	194,783,000		194,783,000				200,292,000
Commercial
Financing Receivable, Allowance for Credit Losses [Roll Forward]
Allowance for loan losses, beginning balance	4,715,000	4,020,000	5,712,000	4,335,000	4,020,000
Charge-offs	771,000	489,000	1,789,000	2,513,000
Recoveries	63,000	278,000	829,000	389,000
Provision	624,000	976,000	(121,000)	2,574,000
Allowance for loan losses, ending balance	4,631,000		4,631,000	4,785,000
Financing Receivable, Allowance for Credit Loss, Additional Information
Reserves for loans individually evaluated for impairment	1,632,000		1,632,000				2,511,000
Loans individually evaluated for impairment	9,016,000		9,016,000				9,295,000
Reserves for loans collectively evaluated for impairment	2,999,000		2,999,000				3,201,000
Loans collectively evaluated for impairment	159,494,000		159,494,000				167,752,000
Consumer and other
Financing Receivable, Allowance for Credit Losses [Roll Forward]
Allowance for loan losses, beginning balance	942,000	829,000	621,000	867,000	829,000
Charge-offs	313,000	156,000	521,000	569,000
Recoveries	21,000	77,000	317,000	169,000
Provision	39,000	31,000	272,000	314,000
Allowance for loan losses, ending balance	689,000		689,000	781,000
Financing Receivable, Allowance for Credit Loss, Additional Information
Reserves for loans individually evaluated for impairment	0		0				0
Loans individually evaluated for impairment	0		0				0
Reserves for loans collectively evaluated for impairment	689,000		689,000				621,000
Loans collectively evaluated for impairment	$ 54,772,000		$ 54,772,000				$ 58,283,000