Regulations and Supervision - Actual Capital Amounts and Ratios (Details) $ in Thousands	Dec. 31, 2020 USD ($)	Dec. 31, 2019 USD ($)
Banking Regulation, Total Capital [Abstract]
Total Capital (to risk-weighted assets), Actual Amount	$ 720,755	$ 666,106
Total Capital (to risk-weighted assets), Actual Ratio	0.144	0.135
Total Capital (to risk-weighted assets), Minimum Capital Required, Amount	$ 525,755	$ 517,039
Total Capital (to risk-weighted assets), Minimum Capital Required, Ratio	0.105	0.105
Total Capital (to risk-weighted assets), Required to be Well Capitalized, Amount	$ 500,719	$ 492,418
Total Capital (to risk-weighted assets), Required to be Well Capitalized, Ratio	0.100	0.100
Common Equity Tier 1 Capital (To Risk-weighted Assets)
Common Equity Tier 1 Capital (to risk-weighted assets), Actual Amount	$ 654,144	$ 607,290
Common Equity Tier 1 Capital (to risk-weighted assets), Actual Ratio	0.131	0.123
Common Equity Tier 1 Capital (to risk-weighted assets), Minimum Capital Required, Amount	$ 350,503	$ 344,692
Common Equity Tier 1 Capital (to risk-weighted assets), Minimum Capital Required, Ratio	7.00%	7.00%
Common Equity Tier 1 Capital (to risk-weighted assets), Required to be Well Capitalized, Amount	$ 325,467	$ 320,072
Common Equity Tier 1 Capital (to risk-weighted assets), Required to be Well Capitalized, Ratio	6.50%	6.50%
Tier 1 Capital (To Risk-weighted Assets)
Tier 1 Capital (to risk-weighted assets), Actual Amount	$ 667,364	$ 624,325
Tier 1 Capital (to risk-weighted assets), Actual Ratio	0.133	0.127
Tier 1 Capital (to risk-weighted assets), Minimum Capital Required, Amount	$ 425,611	$ 418,555
Tier 1 Capital (to risk-weighted assets), Minimum Capital Required, Ratio	0.085	0.085
Tier 1 Capital (to risk-weighted assets), Required to be Well Capitalized, Amount	$ 400,575	$ 393,934
Tier 1 Capital (to risk-weighted assets), Required to be Well Capitalized, Ratio	0.080	0.080
Tier 1 Leverage Capital (To Average Assets)
Tier 1 Capital (to average assets), Actual Amount	$ 667,364	$ 624,325
Tier 1 Capital (to average assets), Actual Ratio	0.088	0.096
Tier 1 Capital (to average assets), Minimum Capital Required, Amount	$ 305,083	$ 259,903
Tier 1 Capital (to average assets), Minimum Capital Required, Ratio	0.040	0.040
Tier 1 Capital (to average assets), Required to be Well Capitalized, Amount	$ 381,354	$ 324,879
Tier 1 Capital (to average assets), Required to be Well Capitalized, Ratio	0.050	0.050
Castile
Banking Regulation, Total Capital [Abstract]
Total Capital (to risk-weighted assets), Actual Amount	$ 157,514	$ 145,669
Total Capital (to risk-weighted assets), Actual Ratio	0.126	0.122
Total Capital (to risk-weighted assets), Minimum Capital Required, Amount	$ 131,034	$ 125,493
Total Capital (to risk-weighted assets), Minimum Capital Required, Ratio	0.105	0.105
Total Capital (to risk-weighted assets), Required to be Well Capitalized, Amount	$ 124,795	$ 119,517
Total Capital (to risk-weighted assets), Required to be Well Capitalized, Ratio	0.100	0.100
Common Equity Tier 1 Capital (To Risk-weighted Assets)
Common Equity Tier 1 Capital (to risk-weighted assets), Actual Amount	$ 144,448	$ 136,342
Common Equity Tier 1 Capital (to risk-weighted assets), Actual Ratio	0.116	0.114
Common Equity Tier 1 Capital (to risk-weighted assets), Minimum Capital Required, Amount	$ 87,356	$ 83,662
Common Equity Tier 1 Capital (to risk-weighted assets), Minimum Capital Required, Ratio	7.00%	7.00%
Common Equity Tier 1 Capital (to risk-weighted assets), Required to be Well Capitalized, Amount	$ 81,117	$ 77,686
Common Equity Tier 1 Capital (to risk-weighted assets), Required to be Well Capitalized, Ratio	6.50%	6.50%
Tier 1 Capital (To Risk-weighted Assets)
Tier 1 Capital (to risk-weighted assets), Actual Amount	$ 144,448	$ 136,342
Tier 1 Capital (to risk-weighted assets), Actual Ratio	0.116	0.114
Tier 1 Capital (to risk-weighted assets), Minimum Capital Required, Amount	$ 106,076	$ 101,589
Tier 1 Capital (to risk-weighted assets), Minimum Capital Required, Ratio	0.085	0.085
Tier 1 Capital (to risk-weighted assets), Required to be Well Capitalized, Amount	$ 99,836	$ 95,614
Tier 1 Capital (to risk-weighted assets), Required to be Well Capitalized, Ratio	0.080	0.080
Tier 1 Leverage Capital (To Average Assets)
Tier 1 Capital (to average assets), Actual Amount	$ 144,448	$ 136,342
Tier 1 Capital (to average assets), Actual Ratio	0.081	0.092
Tier 1 Capital (to average assets), Minimum Capital Required, Amount	$ 71,605	$ 59,239
Tier 1 Capital (to average assets), Minimum Capital Required, Ratio	0.040	0.040
Tier 1 Capital (to average assets), Required to be Well Capitalized, Amount	$ 89,507	$ 74,049
Tier 1 Capital (to average assets), Required to be Well Capitalized, Ratio	0.050	0.050
Mahopac
Banking Regulation, Total Capital [Abstract]
Total Capital (to risk-weighted assets), Actual Amount	$ 133,969	$ 128,238
Total Capital (to risk-weighted assets), Actual Ratio	0.130	0.130
Total Capital (to risk-weighted assets), Minimum Capital Required, Amount	$ 108,129	$ 103,800
Total Capital (to risk-weighted assets), Minimum Capital Required, Ratio	0.105	0.105
Total Capital (to risk-weighted assets), Required to be Well Capitalized, Amount	$ 102,980	$ 98,857
Total Capital (to risk-weighted assets), Required to be Well Capitalized, Ratio	0.100	0.100
Common Equity Tier 1 Capital (To Risk-weighted Assets)
Common Equity Tier 1 Capital (to risk-weighted assets), Actual Amount	$ 122,393	$ 116,953
Common Equity Tier 1 Capital (to risk-weighted assets), Actual Ratio	0.119	0.118
Common Equity Tier 1 Capital (to risk-weighted assets), Minimum Capital Required, Amount	$ 72,086	$ 69,200
Common Equity Tier 1 Capital (to risk-weighted assets), Minimum Capital Required, Ratio	7.00%	7.00%
Common Equity Tier 1 Capital (to risk-weighted assets), Required to be Well Capitalized, Amount	$ 66,937	$ 64,257
Common Equity Tier 1 Capital (to risk-weighted assets), Required to be Well Capitalized, Ratio	6.50%	6.50%
Tier 1 Capital (To Risk-weighted Assets)
Tier 1 Capital (to risk-weighted assets), Actual Amount	$ 122,393	$ 116,953
Tier 1 Capital (to risk-weighted assets), Actual Ratio	0.119	0.118
Tier 1 Capital (to risk-weighted assets), Minimum Capital Required, Amount	$ 87,533	$ 84,029
Tier 1 Capital (to risk-weighted assets), Minimum Capital Required, Ratio	0.085	0.085
Tier 1 Capital (to risk-weighted assets), Required to be Well Capitalized, Amount	$ 82,384	$ 79,086
Tier 1 Capital (to risk-weighted assets), Required to be Well Capitalized, Ratio	0.080	0.080
Tier 1 Leverage Capital (To Average Assets)
Tier 1 Capital (to average assets), Actual Amount	$ 122,393	$ 116,953
Tier 1 Capital (to average assets), Actual Ratio	0.082	0.089
Tier 1 Capital (to average assets), Minimum Capital Required, Amount	$ 59,742	$ 52,469
Tier 1 Capital (to average assets), Minimum Capital Required, Ratio	0.040	0.040
Tier 1 Capital (to average assets), Required to be Well Capitalized, Amount	$ 74,678	$ 65,586
Tier 1 Capital (to average assets), Required to be Well Capitalized, Ratio	0.050	0.050
VIST
Banking Regulation, Total Capital [Abstract]
Total Capital (to risk-weighted assets), Actual Amount	$ 175,931	$ 161,559
Total Capital (to risk-weighted assets), Actual Ratio	0.137	0.122
Total Capital (to risk-weighted assets), Minimum Capital Required, Amount	$ 134,615	$ 139,235
Total Capital (to risk-weighted assets), Minimum Capital Required, Ratio	0.105	0.105
Total Capital (to risk-weighted assets), Required to be Well Capitalized, Amount	$ 128,205	$ 132,605
Total Capital (to risk-weighted assets), Required to be Well Capitalized, Ratio	0.100	0.100
Common Equity Tier 1 Capital (To Risk-weighted Assets)
Common Equity Tier 1 Capital (to risk-weighted assets), Actual Amount	$ 163,895	$ 152,550
Common Equity Tier 1 Capital (to risk-weighted assets), Actual Ratio	0.128	0.115
Common Equity Tier 1 Capital (to risk-weighted assets), Minimum Capital Required, Amount	$ 89,743	$ 92,823
Common Equity Tier 1 Capital (to risk-weighted assets), Minimum Capital Required, Ratio	7.00%	7.00%
Common Equity Tier 1 Capital (to risk-weighted assets), Required to be Well Capitalized, Amount	$ 83,333	$ 86,193
Common Equity Tier 1 Capital (to risk-weighted assets), Required to be Well Capitalized, Ratio	6.50%	6.50%
Tier 1 Capital (To Risk-weighted Assets)
Tier 1 Capital (to risk-weighted assets), Actual Amount	$ 163,895	$ 152,550
Tier 1 Capital (to risk-weighted assets), Actual Ratio	0.128	0.115
Tier 1 Capital (to risk-weighted assets), Minimum Capital Required, Amount	$ 108,794	$ 112,714
Tier 1 Capital (to risk-weighted assets), Minimum Capital Required, Ratio	0.085	0.085
Tier 1 Capital (to risk-weighted assets), Required to be Well Capitalized, Amount	$ 102,564	$ 106,084
Tier 1 Capital (to risk-weighted assets), Required to be Well Capitalized, Ratio	0.080	0.080
Tier 1 Leverage Capital (To Average Assets)
Tier 1 Capital (to average assets), Actual Amount	$ 163,895	$ 152,550
Tier 1 Capital (to average assets), Actual Ratio	0.084	0.087
Tier 1 Capital (to average assets), Minimum Capital Required, Amount	$ 77,874	$ 70,228
Tier 1 Capital (to average assets), Minimum Capital Required, Ratio	0.040	0.040
Tier 1 Capital (to average assets), Required to be Well Capitalized, Amount	$ 83,333	$ 87,785
Tier 1 Capital (to average assets), Required to be Well Capitalized, Ratio	0.050	0.050
Subsidiaries
Banking Regulation, Total Capital [Abstract]
Total Capital (to risk-weighted assets), Actual Amount	$ 210,756	$ 197,660
Total Capital (to risk-weighted assets), Actual Ratio	0.147	0.141
Total Capital (to risk-weighted assets), Minimum Capital Required, Amount	$ 150,482	$ 147,405
Total Capital (to risk-weighted assets), Minimum Capital Required, Ratio	0.105	0.105
Total Capital (to risk-weighted assets), Required to be Well Capitalized, Amount	$ 143,316	$ 140,386
Total Capital (to risk-weighted assets), Required to be Well Capitalized, Ratio	0.100	0.100
Common Equity Tier 1 Capital (To Risk-weighted Assets)
Common Equity Tier 1 Capital (to risk-weighted assets), Actual Amount	$ 196,522	$ 185,499
Common Equity Tier 1 Capital (to risk-weighted assets), Actual Ratio	0.137	0.132
Common Equity Tier 1 Capital (to risk-weighted assets), Minimum Capital Required, Amount	$ 100,321	$ 98,270
Common Equity Tier 1 Capital (to risk-weighted assets), Minimum Capital Required, Ratio	7.00%	7.00%
Common Equity Tier 1 Capital (to risk-weighted assets), Required to be Well Capitalized, Amount	$ 93,156	$ 91,251
Common Equity Tier 1 Capital (to risk-weighted assets), Required to be Well Capitalized, Ratio	6.50%	6.50%
Tier 1 Capital (To Risk-weighted Assets)
Tier 1 Capital (to risk-weighted assets), Actual Amount	$ 196,522	$ 185,499
Tier 1 Capital (to risk-weighted assets), Actual Ratio	0.137	0.132
Tier 1 Capital (to risk-weighted assets), Minimum Capital Required, Amount	$ 121,819	$ 119,328
Tier 1 Capital (to risk-weighted assets), Minimum Capital Required, Ratio	0.085	0.085
Tier 1 Capital (to risk-weighted assets), Required to be Well Capitalized, Amount	$ 114,653	$ 112,309
Tier 1 Capital (to risk-weighted assets), Required to be Well Capitalized, Ratio	0.080	0.080
Tier 1 Leverage Capital (To Average Assets)
Tier 1 Capital (to average assets), Actual Amount	$ 196,522	$ 185,499
Tier 1 Capital (to average assets), Actual Ratio	0.082	0.092
Tier 1 Capital (to average assets), Minimum Capital Required, Amount	$ 95,691	$ 81,001
Tier 1 Capital (to average assets), Minimum Capital Required, Ratio	0.040	0.040
Tier 1 Capital (to average assets), Required to be Well Capitalized, Amount	$ 119,614	$ 101,252
Tier 1 Capital (to average assets), Required to be Well Capitalized, Ratio	0.050	0.050