Loans and Allowance for Loan Losses, Allowance for Loan Losses (Details) - USD ($) $ in Thousands		3 Months Ended		6 Months Ended
		Jun. 30, 2020	Jun. 30, 2019	Jun. 30, 2020	Jun. 30, 2019
Allowance for loan losses by portfolio segment [Roll Forward]
Balance at beginning of period		$ 46,155	$ 44,671	$ 44,317	$ 44,766
Loans charged off [Abstract]
Total loan charge offs		71	254	291	742
Recoveries of loans previously charged off [Abstract]
Total recoveries		60	289	118	382
Net loans (recoveries) charged off		11	(35)	173	360
(Credit) provision for loan losses		2,000	(341)	4,000	(41)
Balance at end of period		48,144	44,365	48,144	44,365
New York and Other States [Member]
Loans charged off [Abstract]
Total loan charge offs	[1]	71	254	272	682
Recoveries of loans previously charged off [Abstract]
Total recoveries	[1]	60	264	116	347
Florida [Member]
Loans charged off [Abstract]
Total loan charge offs		0	0	19	60
Recoveries of loans previously charged off [Abstract]
Total recoveries		0	25	2	35
Commercial [Member]
Allowance for loan losses by portfolio segment [Roll Forward]
Balance at beginning of period		3,960	3,734	3,999	4,048
Loans charged off [Abstract]
Total loan charge offs		0	0	3	7
Recoveries of loans previously charged off [Abstract]
Total recoveries		6	1	8	4
Net loans (recoveries) charged off		(6)	(1)	(5)	3
(Credit) provision for loan losses		400	178	362	(132)
Balance at end of period		4,366	3,913	4,366	3,913
Commercial [Member] | New York and Other States [Member]
Loans charged off [Abstract]
Total loan charge offs	[1]	0	0	3	7
Recoveries of loans previously charged off [Abstract]
Total recoveries	[1]	6	1	8	4
Commercial [Member] | Florida [Member]
Loans charged off [Abstract]
Total loan charge offs		0	0	0	0
Recoveries of loans previously charged off [Abstract]
Total recoveries		0	0	0	0
Real Estate Mortgage - 1 to 4 Family [Member]
Allowance for loan losses by portfolio segment [Roll Forward]
Balance at beginning of period		41,643	39,985	39,748	39,772
Loans charged off [Abstract]
Total loan charge offs		22	205	213	626
Recoveries of loans previously charged off [Abstract]
Total recoveries		49	284	102	368
Net loans (recoveries) charged off		(27)	(79)	111	258
(Credit) provision for loan losses		1,604	(101)	3,637	449
Balance at end of period		43,274	39,963	43,274	39,963
Real Estate Mortgage - 1 to 4 Family [Member] | New York and Other States [Member]
Loans charged off [Abstract]
Total loan charge offs	[1]	22	205	213	597
Recoveries of loans previously charged off [Abstract]
Total recoveries	[1]	49	259	100	333
Real Estate Mortgage - 1 to 4 Family [Member] | Florida [Member]
Loans charged off [Abstract]
Total loan charge offs		0	0	0	29
Recoveries of loans previously charged off [Abstract]
Total recoveries		0	25	2	35
Installment [Member]
Allowance for loan losses by portfolio segment [Roll Forward]
Balance at beginning of period		552	952	570	946
Loans charged off [Abstract]
Total loan charge offs		49	49	75	109
Recoveries of loans previously charged off [Abstract]
Total recoveries		5	4	8	10
Net loans (recoveries) charged off		44	45	67	99
(Credit) provision for loan losses		(4)	(418)	1	(358)
Balance at end of period		504	489	504	489
Installment [Member] | New York and Other States [Member]
Loans charged off [Abstract]
Total loan charge offs	[1]	49	49	56	78
Recoveries of loans previously charged off [Abstract]
Total recoveries	[1]	5	4	8	10
Installment [Member] | Florida [Member]
Loans charged off [Abstract]
Total loan charge offs		0	0	19	31
Recoveries of loans previously charged off [Abstract]
Total recoveries		$ 0	$ 0	$ 0	$ 0

[1] Includes New York, New Jersey, Vermont and Massachusetts.