Loans and Allowance for Loan Losses, Allowance for Loan Losses (Details) - USD ($) $ in Thousands		12 Months Ended
		Dec. 31, 2018	Dec. 31, 2017	Dec. 31, 2016
Allowance for loan losses by portfolio segment [Roll Forward]
Balance at beginning of period		$ 44,170	$ 43,890	$ 44,762
Loans charged off [Abstract]
Total loan charge offs		1,203	2,511	4,710
Recoveries of loans previously charged off [Abstract]
Total recoveries		399	791	888
Net loans charged off (recoveries)		804	1,720	3,822
Provision (recoveries) for loan losses		1,400	2,000	2,950
Balance at end of period		44,766	44,170	43,890
New York and Other States [Member]
Loans charged off [Abstract]
Total loan charge offs	[1]	1,170	2,325	4,545
Recoveries of loans previously charged off [Abstract]
Total recoveries	[1]	390	718	884
Florida [Member]
Loans charged off [Abstract]
Total loan charge offs		33	186	165
Recoveries of loans previously charged off [Abstract]
Total recoveries		9	73	4
Commercial [Member]
Allowance for loan losses by portfolio segment [Roll Forward]
Balance at beginning of period		4,324	4,929	4,491
Loans charged off [Abstract]
Total loan charge offs		100	72	795
Recoveries of loans previously charged off [Abstract]
Total recoveries		10	96	207
Net loans charged off (recoveries)		90	(24)	588
Provision (recoveries) for loan losses		(186)	(629)	1,026
Balance at end of period		4,048	4,324	4,929
Commercial [Member] | New York and Other States [Member]
Loans charged off [Abstract]
Total loan charge offs	[1]	100	72	795
Recoveries of loans previously charged off [Abstract]
Total recoveries	[1]	10	96	207
Commercial [Member] | Florida [Member]
Loans charged off [Abstract]
Total loan charge offs		0	0	0
Recoveries of loans previously charged off [Abstract]
Total recoveries		0	0	0
Real Estate Mortgage - 1 to 4 Family [Member]
Allowance for loan losses by portfolio segment [Roll Forward]
Balance at beginning of period		39,077	38,231	39,753
Loans charged off [Abstract]
Total loan charge offs		846	2,220	3,573
Recoveries of loans previously charged off [Abstract]
Total recoveries		351	669	617
Net loans charged off (recoveries)		495	1,551	2,956
Provision (recoveries) for loan losses		1,190	2,397	1,434
Balance at end of period		39,772	39,077	38,231
Real Estate Mortgage - 1 to 4 Family [Member] | New York and Other States [Member]
Loans charged off [Abstract]
Total loan charge offs	[1]	846	2,053	3,447
Recoveries of loans previously charged off [Abstract]
Total recoveries	[1]	348	596	613
Real Estate Mortgage - 1 to 4 Family [Member] | Florida [Member]
Loans charged off [Abstract]
Total loan charge offs		0	167	126
Recoveries of loans previously charged off [Abstract]
Total recoveries		3	73	4
Installment [Member]
Allowance for loan losses by portfolio segment [Roll Forward]
Balance at beginning of period		769	730	518
Loans charged off [Abstract]
Total loan charge offs		257	219	342
Recoveries of loans previously charged off [Abstract]
Total recoveries		38	26	64
Net loans charged off (recoveries)		219	193	278
Provision (recoveries) for loan losses		396	232	490
Balance at end of period		946	769	730
Installment [Member] | New York and Other States [Member]
Loans charged off [Abstract]
Total loan charge offs	[1]	224	200	303
Recoveries of loans previously charged off [Abstract]
Total recoveries	[1]	32	26	64
Installment [Member] | Florida [Member]
Loans charged off [Abstract]
Total loan charge offs		33	19	39
Recoveries of loans previously charged off [Abstract]
Total recoveries		$ 6	$ 0	$ 0

[1] Includes New York, New Jersey, Vermont, and Massachusetts.