Income Taxes - Deferred Tax Assets and Liabilities (Details) - USD ($) $ in Thousands	Dec. 31, 2019	Dec. 31, 2018
Deferred tax assets:
Defined postretirement benefit obligation	$ 9,901	$ 7,877
Allowance for loan losses	7,231	7,697
Federal net operating loss carryforwards	3,662	5,093
Accrued expenses	2,562	1,939
Lease accounting	1,647
Stock compensation	1,636	1,244
California net operating loss carryforwards	1,489	2,128
State income taxes	954	1,117
Premises and equipment	695	642
Split-dollar life insurance benefit plan	75	80
Nonaccrual interest	61	55
Tax credit carryforwards	57	71
Securities available-for-sale		2,184
Other	654	716
Total deferred tax assets	30,624	30,843
Deferred tax liabilities:
Intangible liabilities	(1,321)	(1,671)
Loan fees	(1,842)	(1,089)
Prepaid expenses	(289)	(554)
I/O strips	(144)	(163)
Securities available-for-sale	(772)
Lease accounting	(1,647)
FHLB stock	(177)	(174)
Other	(130)	(103)
Total deferred tax liabilities	(6,322)	(3,754)
Net deferred tax assets	$ 24,302	$ 27,089