Loans and Allowance for Loan Losses (Details) - USD ($) $ in Thousands		Sep. 30, 2016	Jun. 30, 2016	Dec. 31, 2015	Sep. 30, 2015	Jun. 30, 2015	Dec. 31, 2014
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		$ 3,387,631		$ 3,293,304
Less: Allowance for loan losses		43,950	$ 44,064	44,762	$ 45,149	$ 45,571	$ 46,327
Net loans		3,343,681		3,248,542
Real Estate Construction Loans [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		23,500		26,600
Residential Borrowers [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		14,000		16,000
Commercial Borrowers Residential Purpose [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		9,500		10,600
Commercial [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		189,795		203,415
Less: Allowance for loan losses		5,198	5,046	4,491	4,056	4,022	4,071
Commercial [Member] | Commercial Real Estate [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		167,252		175,873
Commercial [Member] | Other [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		22,543		27,542
Real Estate Mortgage - 1 to 4 Family [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		3,189,321		3,080,498
Less: Allowance for loan losses		38,067	38,589	39,753	40,647	41,087	42,088
Real Estate Mortgage - 1 to 4 Family [Member] | First Mortgages [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		2,775,536		2,660,672
Real Estate Mortgage - 1 to 4 Family [Member] | Home Equity Loans [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		70,340		60,501
Real Estate Mortgage - 1 to 4 Family [Member] | Home Equity Lines of Credit [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		343,445		359,325
Installment [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		8,515		9,391
Less: Allowance for loan losses		685	$ 429	518	$ 446	$ 462	$ 168
New York and Other States [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	[1]	2,675,893		2,650,787
New York and Other States [Member] | Commercial [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		177,290		188,414
New York and Other States [Member] | Commercial [Member] | Commercial Real Estate [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	[1]	154,747		160,965
New York and Other States [Member] | Commercial [Member] | Other [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	[1]	22,543		27,449
New York and Other States [Member] | Real Estate Mortgage - 1 to 4 Family [Member] | First Mortgages [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	[1]	2,138,783		2,093,957
New York and Other States [Member] | Real Estate Mortgage - 1 to 4 Family [Member] | Home Equity Loans [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	[1]	59,512		52,251
New York and Other States [Member] | Real Estate Mortgage - 1 to 4 Family [Member] | Home Equity Lines of Credit [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	[1]	293,459		308,165
New York and Other States [Member] | Installment [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	[1]	6,849		8,000
Florida [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		711,738		642,517
Florida [Member] | Commercial [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		12,505		15,001
Florida [Member] | Commercial [Member] | Commercial Real Estate [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		12,505		14,908
Florida [Member] | Commercial [Member] | Other [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		0		93
Florida [Member] | Real Estate Mortgage - 1 to 4 Family [Member] | First Mortgages [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		636,753		566,715
Florida [Member] | Real Estate Mortgage - 1 to 4 Family [Member] | Home Equity Loans [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		10,828		8,250
Florida [Member] | Real Estate Mortgage - 1 to 4 Family [Member] | Home Equity Lines of Credit [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		49,986		51,160
Florida [Member] | Installment [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		$ 1,666		$ 1,391

[1] Includes New York, New Jersey, Vermont and Massachusetts.