Loans and Allowance for Loan Losses, Allowance for Loan Losses (Details) - USD ($) $ in Thousands		3 Months Ended
		Mar. 31, 2018	Mar. 31, 2017
Allowance for loan losses by portfolio segment [Roll Forward]
Balance at beginning of period		$ 44,170	$ 43,890
Loans charged off [Abstract]
Total loan charge offs		205	629
Recoveries of loans previously charged off [Abstract]
Total recoveries		115	187
Net loans charged off (recoveries)		90	442
Provision for loan losses		300	600
Balance at end of period		44,379	44,048
New York and Other States [Member]
Loans charged off [Abstract]
Total loan charge offs	[1]	202	543
Recoveries of loans previously charged off [Abstract]
Total recoveries		115	187	[1]
Florida [Member]
Loans charged off [Abstract]
Total loan charge offs		3	86
Recoveries of loans previously charged off [Abstract]
Total recoveries		0	0
Commercial [Member]
Allowance for loan losses by portfolio segment [Roll Forward]
Balance at beginning of period		4,324	4,929
Loans charged off [Abstract]
Total loan charge offs		0	72
Recoveries of loans previously charged off [Abstract]
Total recoveries		6	8
Net loans charged off (recoveries)		(6)	64
Provision for loan losses		(75)	(55)
Balance at end of period		4,255	4,810
Commercial [Member] | New York and Other States [Member]
Loans charged off [Abstract]
Total loan charge offs	[1]	0	72
Recoveries of loans previously charged off [Abstract]
Total recoveries		6	8	[1]
Commercial [Member] | Florida [Member]
Loans charged off [Abstract]
Total loan charge offs		0	0
Recoveries of loans previously charged off [Abstract]
Total recoveries		0	0
Real Estate Mortgage - 1 to 4 Family [Member]
Allowance for loan losses by portfolio segment [Roll Forward]
Balance at beginning of period		39,077	38,231
Loans charged off [Abstract]
Total loan charge offs		131	514
Recoveries of loans previously charged off [Abstract]
Total recoveries		103	169
Net loans charged off (recoveries)		28	345
Provision for loan losses		310	695
Balance at end of period		39,359	38,581
Real Estate Mortgage - 1 to 4 Family [Member] | New York and Other States [Member]
Loans charged off [Abstract]
Total loan charge offs	[1]	131	430
Recoveries of loans previously charged off [Abstract]
Total recoveries		103	169	[1]
Real Estate Mortgage - 1 to 4 Family [Member] | Florida [Member]
Loans charged off [Abstract]
Total loan charge offs		0	84
Recoveries of loans previously charged off [Abstract]
Total recoveries		0	0
Installment [Member]
Allowance for loan losses by portfolio segment [Roll Forward]
Balance at beginning of period		769	730
Loans charged off [Abstract]
Total loan charge offs		74	43
Recoveries of loans previously charged off [Abstract]
Total recoveries		6	10
Net loans charged off (recoveries)		68	33
Provision for loan losses		64	(40)
Balance at end of period		765	657
Installment [Member] | New York and Other States [Member]
Loans charged off [Abstract]
Total loan charge offs	[1]	71	41
Recoveries of loans previously charged off [Abstract]
Total recoveries		6	10	[1]
Installment [Member] | Florida [Member]
Loans charged off [Abstract]
Total loan charge offs		3	2
Recoveries of loans previously charged off [Abstract]
Total recoveries		$ 0	$ 0

[1] Includes New York, New Jersey, Vermont and Massachusetts