Loans and Allowance for Loan Losses (Details) - USD ($) $ in Thousands		12 Months Ended
		Dec. 31, 2015	Dec. 31, 2014	Dec. 31, 2013	Dec. 31, 2012
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		$ 3,293,304	$ 3,158,332
Less: Allowance for loan losses		44,762	46,327	$ 47,714	$ 47,927
Net loans		3,248,542	3,112,005
Due from officers		7,600	8,300
New loans		2,600
Repayments of loans		3,300
Real Estate Construction Loans [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		26,600	38,500
Residential Borrowers [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		16,000	17,600
Commercial Borrowers Residential Purpose [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		10,600	20,900
Commercial [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		203,415	223,382
Less: Allowance for loan losses		4,491	4,071	4,019	3,771
Commercial [Member] | Commercial Real Estate [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		175,873	194,124
Commercial [Member] | Other [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		27,542	29,258
Real Estate Mortgage - 1 to 4 Family [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		3,080,498	2,927,356
Less: Allowance for loan losses		39,753	42,088	43,597	44,069
Real Estate Mortgage - 1 to 4 Family [Member] | First Mortgages [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		2,660,672	2,517,567
Real Estate Mortgage - 1 to 4 Family [Member] | Home Equity Loans [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		60,501	57,655
Real Estate Mortgage - 1 to 4 Family [Member] | Home Equity Lines of Credit [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		359,325	352,134
Installment [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		9,391	7,594
Less: Allowance for loan losses		518	168	$ 98	$ 87
New York and Other States [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	[1]	2,650,787	2,612,379
New York and Other States [Member] | Commercial [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		188,414	203,988
New York and Other States [Member] | Commercial [Member] | Commercial Real Estate [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	[1]	160,965	174,788
New York and Other States [Member] | Commercial [Member] | Other [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	[1]	27,449	29,200
New York and Other States [Member] | Real Estate Mortgage - 1 to 4 Family [Member] | First Mortgages [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	[1]	2,093,957	2,041,140
New York and Other States [Member] | Real Estate Mortgage - 1 to 4 Family [Member] | Home Equity Loans [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	[1]	52,251	51,713
New York and Other States [Member] | Real Estate Mortgage - 1 to 4 Family [Member] | Home Equity Lines of Credit [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	[1]	308,165	308,764
New York and Other States [Member] | Installment [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	[1]	8,000	6,774
Florida [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		642,517	545,953
Florida [Member] | Commercial [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		15,001	19,394
Florida [Member] | Commercial [Member] | Commercial Real Estate [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		14,908	19,336
Florida [Member] | Commercial [Member] | Other [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		93	58
Florida [Member] | Real Estate Mortgage - 1 to 4 Family [Member] | First Mortgages [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		566,715	476,427
Florida [Member] | Real Estate Mortgage - 1 to 4 Family [Member] | Home Equity Loans [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		8,250	5,942
Florida [Member] | Real Estate Mortgage - 1 to 4 Family [Member] | Home Equity Lines of Credit [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		51,160	43,370
Florida [Member] | Installment [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		$ 1,391	$ 820

[1] Includes New York, New Jersey, Vermont, and Massachusetts.