Allowance for Loan Losses - Balance of the Allowance Segregated by Segment and Class (Details) - USD ($) $ in Thousands	Mar. 31, 2019	Dec. 31, 2018	Mar. 31, 2018	Dec. 31, 2017
Financing Receivable, Allowance for Credit Losses [Line Items]
Individually evaluated for impairment	$ 305		$ 372
Collectively evaluated for impairment	19,904		20,077
Allowance for credit losses	20,209	$ 19,519	20,449	$ 21,461
AQR Substandard
Financing Receivable, Allowance for Credit Losses [Line Items]
Individually evaluated for impairment	305	14
Collectively evaluated for impairment	17	37
AQR Pass
Financing Receivable, Allowance for Credit Losses [Line Items]
Collectively evaluated for impairment	17,575	16,005
AQR Special Mention
Financing Receivable, Allowance for Credit Losses [Line Items]
Collectively evaluated for impairment	790	828
Unallocated
Financing Receivable, Allowance for Credit Losses [Line Items]
Collectively evaluated for impairment	1,522	2,635
Commercial
Financing Receivable, Allowance for Credit Losses [Line Items]
Individually evaluated for impairment	212		370
Collectively evaluated for impairment	6,266		5,314
Allowance for credit losses	6,478	5,660	5,684	6,172
Commercial | AQR Substandard
Financing Receivable, Allowance for Credit Losses [Line Items]
Individually evaluated for impairment	212	14
Collectively evaluated for impairment	2	3
Commercial | AQR Pass
Financing Receivable, Allowance for Credit Losses [Line Items]
Collectively evaluated for impairment	6,127	5,522
Commercial | AQR Special Mention
Financing Receivable, Allowance for Credit Losses [Line Items]
Collectively evaluated for impairment	137	121
Commercial | Unallocated
Financing Receivable, Allowance for Credit Losses [Line Items]
Collectively evaluated for impairment	0	0
Real estate construction one-to-four family
Financing Receivable, Allowance for Credit Losses [Line Items]
Individually evaluated for impairment	0		0
Collectively evaluated for impairment	659		608
Allowance for credit losses	659	675	608	629
Real estate construction one-to-four family | AQR Substandard
Financing Receivable, Allowance for Credit Losses [Line Items]
Individually evaluated for impairment	0	0
Collectively evaluated for impairment	0	0
Real estate construction one-to-four family | AQR Pass
Financing Receivable, Allowance for Credit Losses [Line Items]
Collectively evaluated for impairment	659	615
Real estate construction one-to-four family | AQR Special Mention
Financing Receivable, Allowance for Credit Losses [Line Items]
Collectively evaluated for impairment	0	60
Real estate construction one-to-four family | Unallocated
Financing Receivable, Allowance for Credit Losses [Line Items]
Collectively evaluated for impairment	0	0
Real estate construction other
Financing Receivable, Allowance for Credit Losses [Line Items]
Individually evaluated for impairment	0		0
Collectively evaluated for impairment	1,367		1,069
Allowance for credit losses	1,367	1,275	1,069	1,566
Real estate construction other | AQR Substandard
Financing Receivable, Allowance for Credit Losses [Line Items]
Individually evaluated for impairment	0	0
Collectively evaluated for impairment	0	0
Real estate construction other | AQR Pass
Financing Receivable, Allowance for Credit Losses [Line Items]
Collectively evaluated for impairment	1,367	1,275
Real estate construction other | AQR Special Mention
Financing Receivable, Allowance for Credit Losses [Line Items]
Collectively evaluated for impairment	0	0
Real estate construction other | Unallocated
Financing Receivable, Allowance for Credit Losses [Line Items]
Collectively evaluated for impairment	0	0
Real estate term owner occupied
Financing Receivable, Allowance for Credit Losses [Line Items]
Individually evaluated for impairment	0		0
Collectively evaluated for impairment	2,320		2,174
Allowance for credit losses	2,320	2,027	2,174	2,194
Real estate term owner occupied | AQR Substandard
Financing Receivable, Allowance for Credit Losses [Line Items]
Individually evaluated for impairment	0	0
Collectively evaluated for impairment	0	0
Real estate term owner occupied | AQR Pass
Financing Receivable, Allowance for Credit Losses [Line Items]
Collectively evaluated for impairment	2,248	1,958
Real estate term owner occupied | AQR Special Mention
Financing Receivable, Allowance for Credit Losses [Line Items]
Collectively evaluated for impairment	72	69
Real estate term owner occupied | Unallocated
Financing Receivable, Allowance for Credit Losses [Line Items]
Collectively evaluated for impairment	0	0
Real estate term non-owner occupied
Financing Receivable, Allowance for Credit Losses [Line Items]
Individually evaluated for impairment	0		0
Collectively evaluated for impairment	6,122		6,379
Allowance for credit losses	6,122	5,799	6,379	6,043
Real estate term non-owner occupied | AQR Substandard
Financing Receivable, Allowance for Credit Losses [Line Items]
Individually evaluated for impairment	0	0
Collectively evaluated for impairment	0	0
Real estate term non-owner occupied | AQR Pass
Financing Receivable, Allowance for Credit Losses [Line Items]
Collectively evaluated for impairment	5,541	5,236
Real estate term non-owner occupied | AQR Special Mention
Financing Receivable, Allowance for Credit Losses [Line Items]
Collectively evaluated for impairment	581	563
Real estate term non-owner occupied | Unallocated
Financing Receivable, Allowance for Credit Losses [Line Items]
Collectively evaluated for impairment	0	0
Real estate term other
Financing Receivable, Allowance for Credit Losses [Line Items]
Individually evaluated for impairment	54		0
Collectively evaluated for impairment	790		1,025
Allowance for credit losses	844	716	1,025	725
Real estate term other | AQR Substandard
Financing Receivable, Allowance for Credit Losses [Line Items]
Individually evaluated for impairment	54	0
Collectively evaluated for impairment	0	21
Real estate term other | AQR Pass
Financing Receivable, Allowance for Credit Losses [Line Items]
Collectively evaluated for impairment	790	683
Real estate term other | AQR Special Mention
Financing Receivable, Allowance for Credit Losses [Line Items]
Collectively evaluated for impairment	0	12
Real estate term other | Unallocated
Financing Receivable, Allowance for Credit Losses [Line Items]
Collectively evaluated for impairment	0	0
Consumer secured by 1st deeds of trust
Financing Receivable, Allowance for Credit Losses [Line Items]
Individually evaluated for impairment	39		2
Collectively evaluated for impairment	357		303
Allowance for credit losses	396	306	305	315
Consumer secured by 1st deeds of trust | AQR Substandard
Financing Receivable, Allowance for Credit Losses [Line Items]
Individually evaluated for impairment	39	0
Collectively evaluated for impairment	0	0
Consumer secured by 1st deeds of trust | AQR Pass
Financing Receivable, Allowance for Credit Losses [Line Items]
Collectively evaluated for impairment	357	303
Consumer secured by 1st deeds of trust | AQR Special Mention
Financing Receivable, Allowance for Credit Losses [Line Items]
Collectively evaluated for impairment	0	3
Consumer secured by 1st deeds of trust | Unallocated
Financing Receivable, Allowance for Credit Losses [Line Items]
Collectively evaluated for impairment	0	0
Consumer other
Financing Receivable, Allowance for Credit Losses [Line Items]
Individually evaluated for impairment	0		0
Collectively evaluated for impairment	501		315
Allowance for credit losses	501	426	315	307
Consumer other | AQR Substandard
Financing Receivable, Allowance for Credit Losses [Line Items]
Individually evaluated for impairment	0	0
Collectively evaluated for impairment	15	13
Consumer other | AQR Pass
Financing Receivable, Allowance for Credit Losses [Line Items]
Collectively evaluated for impairment	486	413
Consumer other | AQR Special Mention
Financing Receivable, Allowance for Credit Losses [Line Items]
Collectively evaluated for impairment	0	0
Consumer other | Unallocated
Financing Receivable, Allowance for Credit Losses [Line Items]
Collectively evaluated for impairment	0	0
Unallocated
Financing Receivable, Allowance for Credit Losses [Line Items]
Individually evaluated for impairment	0		0
Collectively evaluated for impairment	1,522		2,890
Allowance for credit losses	1,522	2,635	$ 2,890	$ 3,510
Unallocated | AQR Substandard
Financing Receivable, Allowance for Credit Losses [Line Items]
Individually evaluated for impairment	0	0
Collectively evaluated for impairment	0	0
Unallocated | AQR Pass
Financing Receivable, Allowance for Credit Losses [Line Items]
Collectively evaluated for impairment	0	0
Unallocated | AQR Special Mention
Financing Receivable, Allowance for Credit Losses [Line Items]
Collectively evaluated for impairment	0	0
Unallocated | Unallocated
Financing Receivable, Allowance for Credit Losses [Line Items]
Collectively evaluated for impairment	$ 1,522	$ 2,635