Loans and Allowance for Loan Losses - Impaired Loans (Details) (USD $)	3 Months Ended		6 Months Ended
	Jun. 30, 2014	Jun. 30, 2013	Jun. 30, 2014	Jun. 30, 2013	Dec. 31, 2013
Impaired Loans
Impaired financing receivable collectively evaluated for impairment	$ 7,800,000		$ 7,800,000		$ 6,200,000
Threshold for individual impairment evaluation			250,000
Impaired loans with a related allowance for loan losses, related allowance	1,995,000		1,995,000		557,000
Total impaired loans, recorded investment	22,188,000		22,188,000		24,526,000
Total impaired loans, unpaid principal balance	23,233,000		23,233,000		25,911,000
Total impaired loans, average recorded investment	21,915,000	32,326,000	22,125,000	25,914,000
Total impaired loans, interest income recognized	159,000	463,000	348,000	221,000
Allowance for loan losses	16,449,000		16,449,000		18,063,000	[1]
Outstanding balance of purchased impaired loans	88,763		88,763		90,741
Impaired financing receivables, Allowance for credit losses, collectively evaluated for impairment	2,400,000		2,400,000		2,100,000
Allowance for Purchased Impaired Loans Excess Credit Discount
Impaired Loans
Allowance for loan losses	24,270		24,270		3,567
Commercial
Impaired Loans
Impaired loans without a related allowance for loan losses, recorded investment	769,000		769,000		4,095,000
Impaired loans without a related allowance for loan losses, unpaid principal balance	769,000		769,000		4,502,000
Impaired loans without a related allowance for loan losses, average recorded investment	771,000	4,433,000	1,296,000	3,317,000
Impaired loans without a related allowance for loan losses, interest income recognized	12,000	79,000	23,000	40,000
Impaired loans with a related allowance for loan losses, recorded investment	1,617,000		1,617,000		308,000
Impaired loans with a related allowance for loan losses, unpaid principal balance	1,617,000		1,617,000		313,000
Impaired loans with a related allowance for loan losses, related allowance	908,000		908,000		105,000
Impaired loans with a related allowance for loan losses, average recorded investment	1,290,000	1,193,000	894,000	1,193,000
Impaired loans with a related allowance for loan losses, interest income recognized	12,000	0	12,000	0
Total impaired loans, recorded investment	2,386,000		2,386,000		4,403,000
Total impaired loans, unpaid principal balance	2,386,000		2,386,000		4,815,000
Total impaired loans, average recorded investment	2,061,000	5,626,000	2,190,000	4,510,000
Total impaired loans, interest income recognized	24,000	79,000	35,000	40,000
Consumer and other
Impaired Loans
Impaired loans without a related allowance for loan losses, recorded investment	0		0		0
Impaired loans without a related allowance for loan losses, unpaid principal balance	0		0		0
Impaired loans without a related allowance for loan losses, average recorded investment	0	0	0	1,000
Impaired loans without a related allowance for loan losses, interest income recognized	0	0	0	0
Impaired loans with a related allowance for loan losses, recorded investment	0		0		0
Impaired loans with a related allowance for loan losses, unpaid principal balance	0		0		0
Impaired loans with a related allowance for loan losses, related allowance	0		0		0
Impaired loans with a related allowance for loan losses, average recorded investment	0	0	0	0
Impaired loans with a related allowance for loan losses, interest income recognized	0	0	0	0
Total impaired loans, recorded investment	0		0		0
Total impaired loans, unpaid principal balance	0		0		0
Total impaired loans, average recorded investment	0	0	0	1,000
Total impaired loans, interest income recognized	0	0	0	0
Construction Loans
Impaired Loans
Impaired loans without a related allowance for loan losses, recorded investment	1,154,000		1,154,000		2,816,000
Impaired loans without a related allowance for loan losses, unpaid principal balance	1,252,000		1,252,000		2,997,000
Impaired loans without a related allowance for loan losses, average recorded investment	1,628,000	5,665,000	2,671,000	3,938,000
Impaired loans without a related allowance for loan losses, interest income recognized	8,000	60,000	36,000	36,000
Impaired loans with a related allowance for loan losses, recorded investment	826,000		826,000		200,000
Impaired loans with a related allowance for loan losses, unpaid principal balance	937,000		937,000		215,000
Impaired loans with a related allowance for loan losses, related allowance	191,000		191,000		40,000
Impaired loans with a related allowance for loan losses, average recorded investment	669,000	1,613,000	383,000	846,000
Impaired loans with a related allowance for loan losses, interest income recognized	0	25,000	0	3,000
Total impaired loans, recorded investment	1,980,000		1,980,000		3,016,000
Total impaired loans, unpaid principal balance	2,189,000		2,189,000		3,212,000
Total impaired loans, average recorded investment	2,297,000	7,278,000	3,054,000	4,784,000
Total impaired loans, interest income recognized	8,000	85,000	36,000	39,000
Owner Occupied
Impaired Loans
Impaired loans without a related allowance for loan losses, recorded investment	7,725,000		7,725,000		5,445,000
Impaired loans without a related allowance for loan losses, unpaid principal balance	8,386,000		8,386,000		5,981,000
Impaired loans without a related allowance for loan losses, average recorded investment	7,861,000	3,282,000	7,392,000	2,443,000
Impaired loans without a related allowance for loan losses, interest income recognized	40,000	49,000	110,000	24,000
Impaired loans with a related allowance for loan losses, recorded investment	5,581,000		5,581,000		5,131,000
Impaired loans with a related allowance for loan losses, unpaid principal balance	5,582,000		5,582,000		5,172,000
Impaired loans with a related allowance for loan losses, related allowance	134,000		134,000		201,000
Impaired loans with a related allowance for loan losses, average recorded investment	5,474,000	7,679,000	5,274,000	7,655,000
Impaired loans with a related allowance for loan losses, interest income recognized	65,000	116,000	126,000	58,000
Total impaired loans, recorded investment	13,306,000		13,306,000		10,576,000
Total impaired loans, unpaid principal balance	13,968,000		13,968,000		11,153,000
Total impaired loans, average recorded investment	13,335,000	10,961,000	12,666,000	10,098,000
Total impaired loans, interest income recognized	105,000	165,000	236,000	82,000
Non-owner Occupied
Impaired Loans
Impaired loans without a related allowance for loan losses, recorded investment	1,470,000		1,470,000		4,469,000
Impaired loans without a related allowance for loan losses, unpaid principal balance	1,541,000		1,541,000		4,537,000
Impaired loans without a related allowance for loan losses, average recorded investment	1,534,000	5,417,000	1,868,000	3,907,000
Impaired loans without a related allowance for loan losses, interest income recognized	4,000	113,000	8,000	53,000
Impaired loans with a related allowance for loan losses, recorded investment	642,000		642,000		645,000
Impaired loans with a related allowance for loan losses, unpaid principal balance	642,000		642,000		645,000
Impaired loans with a related allowance for loan losses, related allowance	42,000		42,000		50,000
Impaired loans with a related allowance for loan losses, average recorded investment	643,000	464,000	643,000	426,000
Impaired loans with a related allowance for loan losses, interest income recognized	8,000	2,000	16,000	0
Total impaired loans, recorded investment	2,112,000		2,112,000		5,114,000
Total impaired loans, unpaid principal balance	2,183,000		2,183,000		5,182,000
Total impaired loans, average recorded investment	2,177,000	5,881,000	2,511,000	4,333,000
Total impaired loans, interest income recognized	12,000	115,000	24,000	53,000
One to Four Family
Impaired Loans
Impaired loans without a related allowance for loan losses, recorded investment	606,000		606,000		661,000
Impaired loans without a related allowance for loan losses, unpaid principal balance	683,000		683,000		731,000
Impaired loans without a related allowance for loan losses, average recorded investment	611,000	710,000	641,000	514,000
Impaired loans without a related allowance for loan losses, interest income recognized	7,000	14,000	14,000	7,000
Impaired loans with a related allowance for loan losses, recorded investment	1,798,000		1,798,000		331,000
Impaired loans with a related allowance for loan losses, unpaid principal balance	1,824,000		1,824,000		341,000
Impaired loans with a related allowance for loan losses, related allowance	720,000		720,000		161,000
Impaired loans with a related allowance for loan losses, average recorded investment	1,434,000	401,000	960,000	396,000
Impaired loans with a related allowance for loan losses, interest income recognized	3,000	3,000	3,000	0
Total impaired loans, recorded investment	2,404,000		2,404,000		992,000
Total impaired loans, unpaid principal balance	2,507,000		2,507,000		1,072,000
Total impaired loans, average recorded investment	2,045,000	1,111,000	1,601,000	910,000
Total impaired loans, interest income recognized	10,000	17,000	17,000	7,000
Multifamily
Impaired Loans
Impaired loans without a related allowance for loan losses, recorded investment	0		0		0
Impaired loans without a related allowance for loan losses, unpaid principal balance	0		0		0
Impaired loans without a related allowance for loan losses, average recorded investment	0	258,000	37,000	192,000
Impaired loans without a related allowance for loan losses, interest income recognized	0	0	0	0
Impaired loans with a related allowance for loan losses, recorded investment	0		0		0
Impaired loans with a related allowance for loan losses, unpaid principal balance	0		0		0
Impaired loans with a related allowance for loan losses, related allowance	0		0		0
Impaired loans with a related allowance for loan losses, average recorded investment	0	0	0	0
Impaired loans with a related allowance for loan losses, interest income recognized	0	0	0	0
Total impaired loans, recorded investment	0		0		0
Total impaired loans, unpaid principal balance	0		0		0
Total impaired loans, average recorded investment	0	258,000	37,000	192,000
Total impaired loans, interest income recognized	0	0	0	0
Home Equity Lines of Credit
Impaired Loans
Impaired loans without a related allowance for loan losses, recorded investment	0		0		425,000
Impaired loans without a related allowance for loan losses, unpaid principal balance	0		0		477,000
Impaired loans without a related allowance for loan losses, average recorded investment	0	468,000	66,000	345,000
Impaired loans without a related allowance for loan losses, interest income recognized	0	0	0	0
Impaired loans with a related allowance for loan losses, recorded investment	0		0		0
Impaired loans with a related allowance for loan losses, unpaid principal balance	0		0		0
Impaired loans with a related allowance for loan losses, related allowance	0		0		0
Impaired loans with a related allowance for loan losses, average recorded investment	0	743,000	0	741,000
Impaired loans with a related allowance for loan losses, interest income recognized	0	2,000	0	0
Total impaired loans, recorded investment	0		0		425,000
Total impaired loans, unpaid principal balance	0		0		477,000
Total impaired loans, average recorded investment	0	1,211,000	66,000	1,086,000
Total impaired loans, interest income recognized	$ 0	$ 2,000	$ 0	$ 0

[1] Derived from audited consolidated financial statements