Income Taxes - Deferred Tax Components (Details) - USD ($) $ in Thousands		12 Months Ended
	Jan. 01, 2018	Dec. 31, 2019	Dec. 31, 2018	Dec. 31, 2017
Deferred tax assets:
Premises and equipment		$ 547	$ 1,037
Hawaii franchise tax		479	468
Unfunded pension liability		925	691
Allowance for loan losses		722	703
Impaired asset write-down			765
Employee benefit plans		2,692	2,601
Equity incentive plan		350	399
Unrealized losses on securities available-for-sale			32
Deferred compensation		413	480
Net lease liability		161
Other		8	159
Total		6,297	7,335
Deferred tax liabilities:
Deferred loan costs		3,078	2,844
FHLB stock dividends		126	126
Prepaid expense		155	169
Unrealized gain on securities available-for-sale		185
Premium on loans sold		134	60
Total		3,678	3,199
Net deferred tax assets		$ 2,619	$ 4,136
Federal statutory corporate tax rate (as a percent)	21.00%	21.00%	21.00%	35.00%
Valuation allowance for deferred tax assets		$ 0	$ 0