Loans and Credit Quality - Impaired Loans (Details) - USD ($) $ in Thousands	12 Months Ended
	Dec. 31, 2016	Dec. 31, 2015	Dec. 31, 2014
Financing Receivable, Impaired [Line Items]
Impaired loan, no related allowance, recorded investment	$ 29,435	$ 33,579
Impaired loan, no related allowance, unpaid principal balance	30,131	33,992
Impaired loan, related allowance, recorded investment	9,221	1,061
Impaired loan, related allowance, unpaid principal balance	9,221	1,061
Related allowance	614	344
Recorded investment	38,656	34,640
Unpaid principal balance	39,352	35,053
Impaired loan, no related allowance, average recorded investment	38,089	24,249
Impaired loan, no related allowance, interest income on accrual method	2,354	1,140
Impaired loan, related allowance, average recorded investment	4,510	1,748
Impaired loan, related allowance, interest income on accrual method	0	0
Impaired loan, average recorded investment	42,599	25,997	$ 10,500
Impaired loan, interest income on accrual method	2,354	1,140	$ 762
Commercial | Asset Quality Rating - Special Mention
Financing Receivable, Impaired [Line Items]
Impaired loan, no related allowance, recorded investment	223	157
Impaired loan, no related allowance, unpaid principal balance	223	157
Related allowance	0	0
Recorded investment	223	157
Unpaid principal balance	223	157
Impaired loan, no related allowance, average recorded investment	170	163
Impaired loan, no related allowance, interest income on accrual method	13	13
Impaired loan, average recorded investment	170	163
Impaired loan, interest income on accrual method	13	13
Commercial | Asset Quality Rating - Substandard
Financing Receivable, Impaired [Line Items]
Impaired loan, no related allowance, recorded investment	9,213	13,333
Impaired loan, no related allowance, unpaid principal balance	9,893	13,746
Impaired loan, related allowance, recorded investment	9,221	1,061
Impaired loan, related allowance, unpaid principal balance	9,221	1,061
Related allowance	614	344
Recorded investment	18,434	14,394
Unpaid principal balance	19,114	14,807
Impaired loan, no related allowance, average recorded investment	15,522	9,996
Impaired loan, no related allowance, interest income on accrual method	840	378
Impaired loan, related allowance, average recorded investment	2,451	1,668
Impaired loan, related allowance, interest income on accrual method	0	0
Impaired loan, average recorded investment	17,973	11,664
Impaired loan, interest income on accrual method	840	378
Real estate construction one-to-four family | Asset Quality Rating - Substandard
Financing Receivable, Impaired [Line Items]
Impaired loan, no related allowance, average recorded investment	988	0
Impaired loan, no related allowance, interest income on accrual method	0	0
Impaired loan, related allowance, average recorded investment	1,986	0
Impaired loan, related allowance, interest income on accrual method	0	0
Impaired loan, average recorded investment	2,974	0
Impaired loan, interest income on accrual method	0	0
Real estate construction other | Asset Quality Rating - Pass
Financing Receivable, Impaired [Line Items]
Impaired loan, no related allowance, average recorded investment	0	557
Impaired loan, no related allowance, interest income on accrual method	0	72
Impaired loan, average recorded investment	0	557
Impaired loan, interest income on accrual method	0	72
Real estate construction other | Asset Quality Rating - Substandard
Financing Receivable, Impaired [Line Items]
Impaired loan, no related allowance, recorded investment	0
Impaired loan, no related allowance, unpaid principal balance	0
Related allowance	0
Recorded investment	0
Unpaid principal balance	0
Impaired loan, no related allowance, average recorded investment	1,431	1,431
Impaired loan, no related allowance, interest income on accrual method	0	0
Impaired loan, average recorded investment	1,431	1,431
Impaired loan, interest income on accrual method	0	0
Real estate term owner occupied | Asset Quality Rating - Pass
Financing Receivable, Impaired [Line Items]
Impaired loan, no related allowance, recorded investment	252	753
Impaired loan, no related allowance, unpaid principal balance	252	753
Related allowance	0	0
Recorded investment	252	753
Unpaid principal balance	252	753
Impaired loan, no related allowance, average recorded investment	378	696
Impaired loan, no related allowance, interest income on accrual method	34	63
Impaired loan, average recorded investment	378	696
Impaired loan, interest income on accrual method	34	63
Real estate term owner occupied | Asset Quality Rating - Special Mention
Financing Receivable, Impaired [Line Items]
Impaired loan, no related allowance, average recorded investment	0	67
Impaired loan, no related allowance, interest income on accrual method	0	5
Impaired loan, average recorded investment	0	67
Impaired loan, interest income on accrual method	0	5
Real estate term owner occupied | Asset Quality Rating - Substandard
Financing Receivable, Impaired [Line Items]
Impaired loan, no related allowance, recorded investment	16,694	16,476
Impaired loan, no related allowance, unpaid principal balance	16,694	16,476
Related allowance	0	0
Recorded investment	16,694	16,476
Unpaid principal balance	16,694	16,476
Impaired loan, no related allowance, average recorded investment	16,551	6,941
Impaired loan, no related allowance, interest income on accrual method	1,202	355
Impaired loan, average recorded investment	16,551	6,941
Impaired loan, interest income on accrual method	1,202	355
Real estate term non-owner occupied | Asset Quality Rating - Pass
Financing Receivable, Impaired [Line Items]
Impaired loan, no related allowance, recorded investment	391	473
Impaired loan, no related allowance, unpaid principal balance	391	473
Related allowance	0	0
Recorded investment	391	473
Unpaid principal balance	391	473
Impaired loan, no related allowance, average recorded investment	442	521
Impaired loan, no related allowance, interest income on accrual method	91	75
Impaired loan, average recorded investment	442	521
Impaired loan, interest income on accrual method	91	75
Real estate term non-owner occupied | Asset Quality Rating - Special Mention
Financing Receivable, Impaired [Line Items]
Impaired loan, no related allowance, average recorded investment	0	1,080
Impaired loan, no related allowance, interest income on accrual method	0	97
Impaired loan, average recorded investment	0	1,080
Impaired loan, interest income on accrual method	0	97
Real estate term non-owner occupied | Asset Quality Rating - Substandard
Financing Receivable, Impaired [Line Items]
Impaired loan, no related allowance, recorded investment	693	352
Impaired loan, no related allowance, unpaid principal balance	693	352
Related allowance	0	0
Recorded investment	693	352
Unpaid principal balance	693	352
Impaired loan, no related allowance, average recorded investment	599	1,141
Impaired loan, no related allowance, interest income on accrual method	28	0
Impaired loan, average recorded investment	599	1,141
Impaired loan, interest income on accrual method	28	0
Real estate term other | Asset Quality Rating - Pass
Financing Receivable, Impaired [Line Items]
Impaired loan, no related allowance, recorded investment	633	699
Impaired loan, no related allowance, unpaid principal balance	633	699
Related allowance	0	0
Recorded investment	633	699
Unpaid principal balance	633	699
Impaired loan, no related allowance, average recorded investment	627	179
Impaired loan, no related allowance, interest income on accrual method	58	13
Impaired loan, average recorded investment	627	179
Impaired loan, interest income on accrual method	58	13
Real estate term other | Asset Quality Rating - Special Mention
Financing Receivable, Impaired [Line Items]
Impaired loan, no related allowance, recorded investment		91
Impaired loan, no related allowance, unpaid principal balance		91
Related allowance		0
Recorded investment		91
Unpaid principal balance		91
Impaired loan, no related allowance, average recorded investment	43	68
Impaired loan, no related allowance, interest income on accrual method	4	6
Impaired loan, average recorded investment	43	68
Impaired loan, interest income on accrual method	4	6
Real estate term other | Asset Quality Rating - Substandard
Financing Receivable, Impaired [Line Items]
Impaired loan, no related allowance, recorded investment	669	697
Impaired loan, no related allowance, unpaid principal balance	669	697
Related allowance	0	0
Recorded investment	669	697
Unpaid principal balance	669	697
Impaired loan, no related allowance, average recorded investment	683	816
Impaired loan, no related allowance, interest income on accrual method	55	50
Impaired loan, related allowance, average recorded investment	0	70
Impaired loan, related allowance, interest income on accrual method	0	0
Impaired loan, average recorded investment	683	886
Impaired loan, interest income on accrual method	55	50
Consumer secured by 1st deeds of trust | Asset Quality Rating - Pass
Financing Receivable, Impaired [Line Items]
Impaired loan, no related allowance, recorded investment		76
Impaired loan, no related allowance, unpaid principal balance		76
Related allowance		0
Recorded investment		76
Unpaid principal balance		76
Impaired loan, no related allowance, average recorded investment	38	79
Impaired loan, no related allowance, interest income on accrual method	2	4
Impaired loan, average recorded investment	38	79
Impaired loan, interest income on accrual method	2	4
Consumer secured by 1st deeds of trust | Asset Quality Rating - Special Mention
Financing Receivable, Impaired [Line Items]
Impaired loan, no related allowance, recorded investment	143
Impaired loan, no related allowance, unpaid principal balance	143
Related allowance	0
Recorded investment	143
Unpaid principal balance	143
Impaired loan, no related allowance, average recorded investment	36	0
Impaired loan, no related allowance, interest income on accrual method	3	0
Impaired loan, average recorded investment	36	0
Impaired loan, interest income on accrual method	3	0
Consumer secured by 1st deeds of trust | Asset Quality Rating - Substandard
Financing Receivable, Impaired [Line Items]
Impaired loan, no related allowance, recorded investment	472	472
Impaired loan, no related allowance, unpaid principal balance	488	472
Related allowance	0	0
Recorded investment	472	472
Unpaid principal balance	488	472
Impaired loan, no related allowance, average recorded investment	568	514
Impaired loan, no related allowance, interest income on accrual method	24	9
Impaired loan, related allowance, average recorded investment	73	0
Impaired loan, related allowance, interest income on accrual method	0	0
Impaired loan, average recorded investment	641	514
Impaired loan, interest income on accrual method	24	9
Consumer other | Asset Quality Rating - Substandard
Financing Receivable, Impaired [Line Items]
Impaired loan, no related allowance, recorded investment	52
Impaired loan, no related allowance, unpaid principal balance	52
Related allowance	0
Recorded investment	52
Unpaid principal balance	52
Impaired loan, no related allowance, average recorded investment	13	0
Impaired loan, no related allowance, interest income on accrual method	0	0
Impaired loan, related allowance, average recorded investment	0	10
Impaired loan, related allowance, interest income on accrual method	0	0
Impaired loan, average recorded investment	13	10
Impaired loan, interest income on accrual method	$ 0	$ 0