Loans and Allowance for Loan Losses - Allowance for Loan Losses (Details) (USD $)	3 Months Ended									12 Months Ended					12 Months Ended			3 Months Ended	12 Months Ended			3 Months Ended	12 Months Ended			3 Months Ended	12 Months Ended			3 Months Ended	12 Months Ended			3 Months Ended	12 Months Ended			3 Months Ended	12 Months Ended			3 Months Ended	12 Months Ended			3 Months Ended	12 Months Ended
	Dec. 31, 2013	Dec. 31, 2012	Sep. 30, 2012	Jun. 30, 2012	Mar. 31, 2012	Dec. 31, 2011	Sep. 30, 2011	Jun. 30, 2011	Mar. 31, 2011	Dec. 31, 2013	Dec. 31, 2012	Dec. 31, 2011	Dec. 31, 2013 Reserve for Off-balance Sheet Activities	Dec. 31, 2012 Reserve for Off-balance Sheet Activities	Dec. 31, 2013 Reserve for Losses on Mortgage Loans Sold	Dec. 31, 2012 Reserve for Losses on Mortgage Loans Sold	Dec. 31, 2011 Reserve for Losses on Mortgage Loans Sold	Dec. 31, 2012 Real Estate Loans Construction Loans	Dec. 31, 2013 Real Estate Loans Construction Loans	Dec. 31, 2012 Real Estate Loans Construction Loans	Dec. 31, 2010 Real Estate Loans Construction Loans	Dec. 31, 2012 Real Estate Loans Owner Occupied Commercial	Dec. 31, 2013 Real Estate Loans Owner Occupied Commercial	Dec. 31, 2012 Real Estate Loans Owner Occupied Commercial	Dec. 31, 2010 Real Estate Loans Owner Occupied Commercial	Dec. 31, 2012 Real Estate Loans Non-owner Occupied	Dec. 31, 2013 Real Estate Loans Non-owner Occupied	Dec. 31, 2012 Real Estate Loans Non-owner Occupied	Dec. 31, 2010 Real Estate Loans Non-owner Occupied	Dec. 31, 2012 Real Estate Loans One to Four Family	Dec. 31, 2013 Real Estate Loans One to Four Family	Dec. 31, 2012 Real Estate Loans One to Four Family	Dec. 31, 2010 Real Estate Loans One to Four Family	Dec. 31, 2012 Real Estate Loans Multifamily	Dec. 31, 2013 Real Estate Loans Multifamily	Dec. 31, 2012 Real Estate Loans Multifamily	Dec. 31, 2010 Real Estate Loans Multifamily	Dec. 31, 2012 Real Estate Loans Home Equity Lines of Credit	Dec. 31, 2013 Real Estate Loans Home Equity Lines of Credit	Dec. 31, 2012 Real Estate Loans Home Equity Lines of Credit	Dec. 31, 2010 Real Estate Loans Home Equity Lines of Credit	Dec. 31, 2012 Commercial	Dec. 31, 2013 Commercial	Dec. 31, 2012 Commercial	Dec. 31, 2010 Commercial	Dec. 31, 2012 Consumer and other	Dec. 31, 2013 Consumer and other	Dec. 31, 2012 Consumer and other	Dec. 31, 2010 Consumer and other
Receivables [Abstract]
Provision for loan losses	$ (3,017,000)		$ 40,000	$ 55,000	$ 237,000	$ 31,554,000	$ 4,251,000	$ 2,218,000	$ 2,351,000	$ (2,685,000)	$ 40,374,000	$ 20,843,000
Financing Receivable, Allowance for Credit Losses
Financing Receivable, Allowance for Credit Losses, Write-downs		49,000,000								7,872,000	51,756,000								2,104,000	17,825,000			383,000	7,334,000			224,000	9,830,000			871,000	3,512,000			0	347,000			1,037,000	4,360,000			2,923,000	7,796,000			330,000	752,000
Financing Receivable, Allowance for Credit Losses, Writedowns, Other Problem Loans		10,999,000																1,601,000				1,599,000				2,267,000				291,000				0				1,309,000				3,908,000				24,000
Financing Receivable, Allowance for Credit Losses [Roll Forward]
Allowance for loan losses, beginning balance									32,848,000	25,149,000		32,848,000							4,269,000		8,214,000		4,374,000		5,792,000		3,935,000		4,668,000		3,191,000		3,726,000		594,000		805,000		3,822,000		3,310,000		4,291,000		5,712,000		673,000		621,000
Recoveries										3,471,000	3,683,000								1,007,000	1,138,000			120,000	165,000			134,000	664,000			253,000	215,000			0	0			568,000	186,000			1,287,000	956,000			102,000	359,000
Provision		11,000,000								(2,685,000)	40,374,000								(3,000)	12,742,000			(1,022,000)	5,751,000			(1,764,000)	8,433,000			(205,000)	2,762,000			(158,000)	136,000			(697,000)	4,686,000			1,078,000	5,419,000			86,000	445,000
Allowance for loan losses, ending balance	18,063,000	25,149,000								18,063,000	25,149,000							4,269,000	3,169,000	4,269,000	8,214,000	4,374,000	3,089,000	4,374,000	5,792,000	3,935,000	2,081,000	3,935,000	4,668,000	3,191,000	2,368,000	3,191,000	3,726,000	594,000	436,000	594,000	805,000	3,822,000	2,656,000	3,822,000	3,310,000	4,291,000	3,733,000	4,291,000	5,712,000	673,000	531,000	673,000	621,000
Financing Receivable, Allowance for Credit Loss, Additional Information
Reserves for loans individually evaluated for impairment	557,000	1,397,000								557,000	1,397,000							191,000	40,000	191,000		189,000	201,000	189,000		89,000	50,000	89,000		0	161,000	0		0	0	0		0	0	0		928,000	105,000	928,000		0	0	0
Loans individually evaluated for impairment	24,526,000	33,554,000								24,526,000	33,554,000							8,170,000	3,016,000	8,170,000		11,524,000	10,576,000	11,524,000		5,537,000	5,114,000	5,537,000		1,435,000	992,000	1,435,000		530,000	0	530,000		705,000	425,000	705,000		5,653,000	4,403,000	5,653,000		0	0	0
Reserves for loans collectively evaluated for impairment	17,506,000	23,752,000								17,506,000	23,752,000							4,078,000	3,129,000	4,078,000		4,185,000	2,888,000	4,185,000		3,846,000	2,031,000	3,846,000		3,191,000	2,207,000	3,191,000		594,000	436,000	594,000		3,822,000	2,656,000	3,822,000		3,363,000	3,628,000	3,363,000		673,000	531,000	673,000
Loans collectively evaluated for impairment	1,333,241,000	1,274,818,000								1,333,241,000	1,274,818,000							123,811,000	128,019,000	123,811,000		331,438,000	372,190,000	331,438,000		205,952,000	191,812,000	205,952,000		167,176,000	173,080,000	167,176,000		34,807,000	41,713,000	34,807,000		191,183,000	193,720,000	191,183,000		168,787,000	181,040,000	168,787,000		51,664,000	51,667,000	51,664,000
Reserves													116,309	182,489	216,687	1,700,000	1,800,000
Charges against reserves															83,198	488,012	809,282
Provision expense															1,400,000	392,103	640,893
Loans sold during period		43,000,000								254,500,000
Valuation Allowance and Reserves, Factors Changing Estimate																1.16%	1.37%
Valuation Allowances and Reserves, Adjustments															$ 125,000