Loans and Allowance for Loan Losses - Allowance for Loan Losses (Details) (USD $)	3 Months Ended		6 Months Ended		12 Months Ended
	Jun. 30, 2012	Jun. 30, 2011	Jun. 30, 2012	Jun. 30, 2011	Jun. 30, 2012	Jun. 30, 2011	Dec. 31, 2011
Financing Receivable, Allowance for Credit Losses [Roll Forward]
Allowance for loan losses, beginning balance	$ 30,062,000	$ 35,860,000	$ 32,848,000	$ 37,752,000	$ 35,652,000
Charge-offs	4,907,000	11,878,000	10,287,000	18,959,000	25,300,000	39,800,000
Recoveries	1,424,000	1,277,000	1,667,000	1,599,000
Provision	2,218,000	10,393,000	4,569,000	15,260,000
Allowance for loan losses, ending balance	28,797,000	35,652,000	28,797,000	35,652,000	28,797,000	35,652,000
Financing Receivable, Allowance for Credit Loss, Additional Information
Reserves for loans individually evaluated for impairment	3,749,000		3,749,000		3,749,000		4,101,000
Loans individually evaluated for impairment	68,334,000		68,334,000		68,334,000		77,896,000
Reserves for loans collectively evaluated for impairment	25,048,000		25,048,000		25,048,000		28,747,000
Loans collectively evaluated for impairment	1,302,906,000		1,302,906,000		1,302,906,000		1,372,143,000
Loans sold during period	69,300,000		134,800,000
Impact of change in allowance methodology	1,400,000
Reserve for Off-balance Sheet Activities
Financing Receivable, Allowance for Credit Loss, Additional Information
Reserves	238,000		238,000		238,000		221,000
Reserve for Losses on Mortgage Loans Sold
Financing Receivable, Allowance for Credit Loss, Additional Information
Reserves	1,600,000		1,600,000		1,600,000		1,800,000
Charges against reserves	208,539	32,000
Provision expense	29,715	14,000	138,701
Real Estate Loans | Construction Loans
Financing Receivable, Allowance for Credit Losses [Roll Forward]
Allowance for loan losses, beginning balance	7,957,000	9,592,000	8,214,000	12,014,000
Charge-offs	2,114,000	4,097,000	3,942,000	7,875,000
Recoveries	170,000	1,008,000	278,000	1,081,000
Provision	(110,000)	4,267,000	1,353,000	5,550,000
Allowance for loan losses, ending balance	5,903,000	10,770,000	5,903,000	10,770,000	5,903,000	10,770,000
Financing Receivable, Allowance for Credit Loss, Additional Information
Reserves for loans individually evaluated for impairment	325,000		325,000		325,000		570,000
Loans individually evaluated for impairment	20,612,000		20,612,000		20,612,000		29,078,000
Reserves for loans collectively evaluated for impairment	5,578,000		5,578,000		5,578,000		7,644,000
Loans collectively evaluated for impairment	169,228,000		169,228,000		169,228,000		173,725,000
Real Estate Loans | Owner Occupied
Financing Receivable, Allowance for Credit Losses [Roll Forward]
Allowance for loan losses, beginning balance	5,334,000	7,234,000	5,792,000	5,958,000
Charge-offs	1,190,000	2,098,000	1,873,000	2,837,000
Recoveries	116,000	30,000	124,000	106,000
Provision	1,370,000	1,535,000	1,587,000	3,474,000
Allowance for loan losses, ending balance	5,630,000	6,701,000	5,630,000	6,701,000	5,630,000	6,701,000
Financing Receivable, Allowance for Credit Loss, Additional Information
Reserves for loans individually evaluated for impairment	996,000		996,000		996,000		585,000
Loans individually evaluated for impairment	21,953,000		21,953,000		21,953,000		24,045,000
Reserves for loans collectively evaluated for impairment	4,633,000		4,633,000		4,633,000		5,207,000
Loans collectively evaluated for impairment	313,154,000		313,154,000		313,154,000		324,886,000
Real Estate Loans | Non-owner Occupied
Financing Receivable, Allowance for Credit Losses [Roll Forward]
Allowance for loan losses, beginning balance	3,726,000	5,269,000	4,668,000	7,150,000
Charge-offs	452,000	1,355,000	1,764,000	2,332,000
Recoveries	33,000	52,000	33,000	52,000
Provision	275,000	314,000	645,000	(590,000)
Allowance for loan losses, ending balance	3,582,000	4,280,000	3,582,000	4,280,000	3,582,000	4,280,000
Financing Receivable, Allowance for Credit Loss, Additional Information
Reserves for loans individually evaluated for impairment	97,000		97,000		97,000		143,000
Loans individually evaluated for impairment	6,561,000		6,561,000		6,561,000		9,614,000
Reserves for loans collectively evaluated for impairment	3,485,000		3,485,000		3,485,000		4,525,000
Loans collectively evaluated for impairment	209,729,000		209,729,000		209,729,000		233,213,000
Real Estate Loans | One to Four Family
Financing Receivable, Allowance for Credit Losses [Roll Forward]
Allowance for loan losses, beginning balance	3,353,000	4,323,000	3,726,000	3,706,000
Charge-offs	193,000	1,239,000	568,000	1,780,000
Recoveries	137,000	46,000	148,000	76,000
Provision	(234,000)	1,384,000	(243,000)	2,512,000
Allowance for loan losses, ending balance	3,063,000	4,514,000	3,063,000	4,514,000	3,063,000	4,514,000
Financing Receivable, Allowance for Credit Loss, Additional Information
Reserves for loans individually evaluated for impairment	222,000		222,000		222,000		45,000
Loans individually evaluated for impairment	7,168,000		7,168,000		7,168,000		4,341,000
Reserves for loans collectively evaluated for impairment	2,841,000		2,841,000		2,841,000		3,681,000
Loans collectively evaluated for impairment	167,306,000		167,306,000		167,306,000		174,706,000
Real Estate Loans | Multifamily
Financing Receivable, Allowance for Credit Losses [Roll Forward]
Allowance for loan losses, beginning balance	506,000	486,000	805,000	424,000
Charge-offs	11,000	0	213,000	12,000
Recoveries	0	0	0	0
Provision	68,000	40,000	(29,000)	114,000
Allowance for loan losses, ending balance	563,000	526,000	563,000	526,000	563,000	526,000
Financing Receivable, Allowance for Credit Loss, Additional Information
Reserves for loans individually evaluated for impairment	0		0		0		135,000
Loans individually evaluated for impairment	707,000		707,000		707,000		595,000
Reserves for loans collectively evaluated for impairment	563,000		563,000		563,000		670,000
Loans collectively evaluated for impairment	35,993,000		35,993,000		35,993,000		39,286,000
Real Estate Loans | Home Equity Lines of Credit
Financing Receivable, Allowance for Credit Losses [Roll Forward]
Allowance for loan losses, beginning balance	3,099,000	3,144,000	3,310,000	3,298,000
Charge-offs	229,000	1,154,000	701,000	1,686,000
Recoveries	11,000	42,000	21,000	81,000
Provision	1,518,000	1,980,000	1,769,000	2,319,000
Allowance for loan losses, ending balance	4,399,000	4,012,000	4,399,000	4,012,000	4,399,000	4,012,000
Financing Receivable, Allowance for Credit Loss, Additional Information
Reserves for loans individually evaluated for impairment	47,000		47,000		47,000		112,000
Loans individually evaluated for impairment	1,351,000		1,351,000		1,351,000		928,000
Reserves for loans collectively evaluated for impairment	4,352,000		4,352,000		4,352,000		3,198,000
Loans collectively evaluated for impairment	195,196,000		195,196,000		195,196,000		200,292,000
Commercial
Financing Receivable, Allowance for Credit Losses [Roll Forward]
Allowance for loan losses, beginning balance	5,514,000	4,939,000	5,712,000	4,335,000
Charge-offs	593,000	1,806,000	1,018,000	2,024,000
Recoveries	681,000	43,000	766,000	111,000
Provision	(887,000)	844,000	(745,000)	1,598,000
Allowance for loan losses, ending balance	4,715,000	4,020,000	4,715,000	4,020,000	4,715,000	4,020,000
Financing Receivable, Allowance for Credit Loss, Additional Information
Reserves for loans individually evaluated for impairment	1,838,000		1,838,000		1,838,000		2,511,000
Loans individually evaluated for impairment	9,758,000		9,758,000		9,758,000		9,295,000
Reserves for loans collectively evaluated for impairment	2,878,000		2,878,000		2,878,000		3,201,000
Loans collectively evaluated for impairment	158,082,000		158,082,000		158,082,000		167,752,000
Consumer and other
Financing Receivable, Allowance for Credit Losses [Roll Forward]
Allowance for loan losses, beginning balance	573,000	873,000	621,000	867,000
Charge-offs	125,000	129,000	208,000	413,000
Recoveries	276,000	56,000	297,000	92,000
Provision	218,000	29,000	232,000	283,000
Allowance for loan losses, ending balance	942,000	829,000	942,000	829,000	942,000	829,000
Financing Receivable, Allowance for Credit Loss, Additional Information
Reserves for loans individually evaluated for impairment	224,000		224,000		224,000		0
Loans individually evaluated for impairment	224,000		224,000		224,000		0
Reserves for loans collectively evaluated for impairment	718,000		718,000		718,000		621,000
Loans collectively evaluated for impairment	$ 54,218,000		$ 54,218,000		$ 54,218,000		$ 58,283,000