Loan Portfolio and Allowance for Credit Losses, Allowance for Credit Losses (Details) - USD ($) $ in Thousands		3 Months Ended
		Mar. 31, 2022	Mar. 31, 2021
Allowance for loan losses by portfolio segment [Roll Forward]
Balance at beginning of period		$ 44,267	$ 49,595
Loans charged off [Abstract]
Total loan charge offs		47	95
Recoveries of loans previously charged off [Abstract]
Total recoveries		105	141
Net loans (recoveries) charged off		(58)	(46)
(Credit) provision for credit losses		(500)	350
Balance at end of period		46,178
Balance at end of period			49,991
New York and Other States [Member]
Loans charged off [Abstract]
Total loan charge offs	[1]	46	93
Recoveries of loans previously charged off [Abstract]
Total recoveries	[1]	105	141
Florida [Member]
Loans charged off [Abstract]
Total loan charge offs		1	2
Recoveries of loans previously charged off [Abstract]
Total recoveries		0	0
Commercial [Member]
Allowance for loan losses by portfolio segment [Roll Forward]
Balance at beginning of period		3,135	4,140
Loans charged off [Abstract]
Total loan charge offs		36	0
Recoveries of loans previously charged off [Abstract]
Total recoveries		0	32
Net loans (recoveries) charged off		36	(32)
(Credit) provision for credit losses		64	(120)
Balance at end of period		2,177
Balance at end of period			4,052
Commercial [Member] | New York and Other States [Member]
Loans charged off [Abstract]
Total loan charge offs	[1]	36	0
Recoveries of loans previously charged off [Abstract]
Total recoveries	[1]	0	32
Commercial [Member] | Florida [Member]
Loans charged off [Abstract]
Total loan charge offs		0	0
Recoveries of loans previously charged off [Abstract]
Total recoveries		0	0
Real Estate Mortgage - 1 to 4 Family [Member]
Allowance for loan losses by portfolio segment [Roll Forward]
Balance at beginning of period		40,689	44,950
Loans charged off [Abstract]
Total loan charge offs		0	86
Recoveries of loans previously charged off [Abstract]
Total recoveries		97	88
Net loans (recoveries) charged off		(97)	(2)
(Credit) provision for credit losses		(572)	555
Balance at end of period		43,931
Balance at end of period			45,507
Real Estate Mortgage - 1 to 4 Family [Member] | New York and Other States [Member]
Loans charged off [Abstract]
Total loan charge offs	[1]	0	86
Recoveries of loans previously charged off [Abstract]
Total recoveries	[1]	97	88
Real Estate Mortgage - 1 to 4 Family [Member] | Florida [Member]
Loans charged off [Abstract]
Total loan charge offs		0	0
Recoveries of loans previously charged off [Abstract]
Total recoveries		0	0
Installment [Member]
Allowance for loan losses by portfolio segment [Roll Forward]
Balance at beginning of period		443	505
Loans charged off [Abstract]
Total loan charge offs		11	9
Recoveries of loans previously charged off [Abstract]
Total recoveries		8	21
Net loans (recoveries) charged off		3	(12)
(Credit) provision for credit losses		8	(85)
Balance at end of period		70
Balance at end of period			432
Installment [Member] | New York and Other States [Member]
Loans charged off [Abstract]
Total loan charge offs	[1]	10	7
Recoveries of loans previously charged off [Abstract]
Total recoveries	[1]	8	21
Installment [Member] | Florida [Member]
Loans charged off [Abstract]
Total loan charge offs		1	2
Recoveries of loans previously charged off [Abstract]
Total recoveries		0	$ 0
ASU 2016-13 [Member]
Allowance for loan losses by portfolio segment [Roll Forward]
Balance at beginning of period		46,620
ASU 2016-13 [Member] | Commercial [Member]
Allowance for loan losses by portfolio segment [Roll Forward]
Balance at beginning of period		2,149
ASU 2016-13 [Member] | Real Estate Mortgage - 1 to 4 Family [Member]
Allowance for loan losses by portfolio segment [Roll Forward]
Balance at beginning of period		44,406
ASU 2016-13 [Member] | Installment [Member]
Allowance for loan losses by portfolio segment [Roll Forward]
Balance at beginning of period		65
ASU 2016-13 [Member] | Impact of Adoption [Member]
Allowance for loan losses by portfolio segment [Roll Forward]
Balance at beginning of period		2,353
ASU 2016-13 [Member] | Impact of Adoption [Member] | Commercial [Member]
Allowance for loan losses by portfolio segment [Roll Forward]
Balance at beginning of period		(986)
ASU 2016-13 [Member] | Impact of Adoption [Member] | Real Estate Mortgage - 1 to 4 Family [Member]
Allowance for loan losses by portfolio segment [Roll Forward]
Balance at beginning of period		3,717
ASU 2016-13 [Member] | Impact of Adoption [Member] | Installment [Member]
Allowance for loan losses by portfolio segment [Roll Forward]
Balance at beginning of period		$ (378)

[1] Includes New York, New Jersey, Vermont and Massachusetts.