Loans and Allowance for Loan Losses, Allowance for Loan Losses (Details) - USD ($) $ in Thousands		12 Months Ended
		Dec. 31, 2019	Dec. 31, 2018	Dec. 31, 2017
Allowance for loan losses by portfolio segment [Roll Forward]
Balance at beginning of period		$ 44,766	$ 44,170	$ 43,890
Loans charged off [Abstract]
Total loan charge offs		1,207	1,203	2,511
Recoveries of loans previously charged off [Abstract]
Total recoveries		599	399	791
Net loans charged off (recoveries)		608	804	1,720
Provision (recoveries) for loan losses		159	1,400	2,000
Balance at end of period		44,317	44,766	44,170
New York and Other States [Member]
Loans charged off [Abstract]
Total loan charge offs	[1]	1,130	1,170	2,325
Recoveries of loans previously charged off [Abstract]
Total recoveries	[1]	563	390	718
Florida [Member]
Loans charged off [Abstract]
Total loan charge offs		77	33	186
Recoveries of loans previously charged off [Abstract]
Total recoveries		36	9	73
Commercial [Member]
Allowance for loan losses by portfolio segment [Roll Forward]
Balance at beginning of period		4,048	4,324	4,929
Loans charged off [Abstract]
Total loan charge offs		20	100	72
Recoveries of loans previously charged off [Abstract]
Total recoveries		46	10	96
Net loans charged off (recoveries)		(26)	90	(24)
Provision (recoveries) for loan losses		(75)	(186)	(629)
Balance at end of period		3,999	4,048	4,324
Commercial [Member] | New York and Other States [Member]
Loans charged off [Abstract]
Total loan charge offs	[1]	20	100	72
Recoveries of loans previously charged off [Abstract]
Total recoveries	[1]	46	10	96
Commercial [Member] | Florida [Member]
Loans charged off [Abstract]
Total loan charge offs		0	0	0
Recoveries of loans previously charged off [Abstract]
Total recoveries		0	0	0
Real Estate Mortgage - 1 to 4 Family [Member]
Allowance for loan losses by portfolio segment [Roll Forward]
Balance at beginning of period		39,772	39,077	38,231
Loans charged off [Abstract]
Total loan charge offs		974	846	2,220
Recoveries of loans previously charged off [Abstract]
Total recoveries		532	351	669
Net loans charged off (recoveries)		442	495	1,551
Provision (recoveries) for loan losses		418	1,190	2,397
Balance at end of period		39,748	39,772	39,077
Real Estate Mortgage - 1 to 4 Family [Member] | New York and Other States [Member]
Loans charged off [Abstract]
Total loan charge offs	[1]	945	846	2,053
Recoveries of loans previously charged off [Abstract]
Total recoveries	[1]	496	348	596
Real Estate Mortgage - 1 to 4 Family [Member] | Florida [Member]
Loans charged off [Abstract]
Total loan charge offs		29	0	167
Recoveries of loans previously charged off [Abstract]
Total recoveries		36	3	73
Installment [Member]
Allowance for loan losses by portfolio segment [Roll Forward]
Balance at beginning of period		946	769	730
Loans charged off [Abstract]
Total loan charge offs		213	257	219
Recoveries of loans previously charged off [Abstract]
Total recoveries		21	38	26
Net loans charged off (recoveries)		192	219	193
Provision (recoveries) for loan losses		(184)	396	232
Balance at end of period		570	946	769
Installment [Member] | New York and Other States [Member]
Loans charged off [Abstract]
Total loan charge offs	[1]	165	224	200
Recoveries of loans previously charged off [Abstract]
Total recoveries	[1]	21	32	26
Installment [Member] | Florida [Member]
Loans charged off [Abstract]
Total loan charge offs		48	33	19
Recoveries of loans previously charged off [Abstract]
Total recoveries		$ 0	$ 6	$ 0

[1] Includes New York, New Jersey, Vermont and Massachusetts.