Loans and Allowance for Loan Losses (Details) - USD ($) $ in Thousands		Mar. 31, 2018	Dec. 31, 2017	Mar. 31, 2017	Dec. 31, 2016
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		$ 3,666,975	$ 3,636,407
Less: Allowance for loan losses		44,379	44,170	$ 44,048	$ 43,890
Net loans		3,622,596	3,592,237
New York and Other States [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	[1]	2,811,286	2,795,993
Florida [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		855,689	840,414
Real Estate Construction Loans [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		28,400	30,900
Residential Borrowers [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		20,400	21,100
Commercial Borrowers Residential Purpose [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		8,000	9,800
Commercial [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		185,129	186,207
Less: Allowance for loan losses		4,255	4,324	4,810	4,929
Commercial [Member] | New York and Other States [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		172,816	172,974
Commercial [Member] | Florida [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		12,313	13,233
Commercial [Member] | Commercial Real Estate [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		161,066	161,892
Commercial [Member] | Commercial Real Estate [Member] | New York and Other States [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	[1]	149,357	149,368
Commercial [Member] | Commercial Real Estate [Member] | Florida [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		11,709	12,524
Commercial [Member] | Other [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		24,063	24,315
Commercial [Member] | Other [Member] | New York and Other States [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	[1]	23,459	23,606
Commercial [Member] | Other [Member] | Florida [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		604	709
Real Estate Mortgage - 1 to 4 Family [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		3,473,433	3,441,437
Less: Allowance for loan losses		39,359	39,077	38,581	38,231
Real Estate Mortgage - 1 to 4 Family [Member] | First Mortgages [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		3,089,514	3,052,077
Real Estate Mortgage - 1 to 4 Family [Member] | First Mortgages [Member] | New York and Other States [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	[1]	2,305,510	2,286,148
Real Estate Mortgage - 1 to 4 Family [Member] | First Mortgages [Member] | Florida [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		784,004	765,929
Real Estate Mortgage - 1 to 4 Family [Member] | Home Equity Loans [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		82,034	80,444
Real Estate Mortgage - 1 to 4 Family [Member] | Home Equity Loans [Member] | New York and Other States [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	[1]	67,841	66,455
Real Estate Mortgage - 1 to 4 Family [Member] | Home Equity Loans [Member] | Florida [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		14,193	13,989
Real Estate Mortgage - 1 to 4 Family [Member] | Home Equity Lines of Credit [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		301,885	308,916
Real Estate Mortgage - 1 to 4 Family [Member] | Home Equity Lines of Credit [Member] | New York and Other States [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	[1]	257,714	263,275
Real Estate Mortgage - 1 to 4 Family [Member] | Home Equity Lines of Credit [Member] | Florida [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		44,171	45,641
Installment [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		8,413	8,763
Less: Allowance for loan losses		765	769	$ 657	$ 730
Installment [Member] | New York and Other States [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	[1]	7,405	7,141
Installment [Member] | Florida [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		$ 1,008	$ 1,622

[1] Includes New York, New Jersey, Vermont and Massachusetts.