Loans and Allowance for Loan Losses (Details) (USD $) In Thousands, unless otherwise specified	Mar. 31, 2015		Dec. 31, 2014		Mar. 31, 2014	Dec. 31, 2013
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	$ 3,193,625		$ 3,158,332
Less: Allowance for loan losses	45,944		46,327		47,035	47,714
Net loans	3,147,681		3,112,005
Commercial [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	212,145		223,382
Less: Allowance for loan losses	4,024		4,071		3,840	4,019
Commercial Real Estate [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	187,146		194,124
Other [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	24,999		29,258
Real Estate Mortgage - 1 to 4 Family [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	2,973,477		2,927,356
Less: Allowance for loan losses	41,529		42,088		43,091	43,597
First Mortgages [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	2,561,775		2,517,567
Home Equity Loans [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	59,150		57,655
Home Equity Lines of Credit [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	352,552		352,134
Installment [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	8,003		7,594
Less: Allowance for loan losses	391		168		104	98
Real Estate Construction Loans [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	32,800		38,500
Residential Borrowers [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	17,100		17,600
Commercial Borrowers Residential Purpose [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	15,700		20,900
New York and Other States [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	2,622,550	[1]	2,612,379	[1]
New York and Other States [Member] | Commercial [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	196,128		203,988
New York and Other States [Member] | Commercial Real Estate [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	171,178	[1]	174,788	[1]
New York and Other States [Member] | Other [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	24,950	[1]	29,200	[1]
New York and Other States [Member] | First Mortgages [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	2,061,142	[1]	2,041,140	[1]
New York and Other States [Member] | Home Equity Loans [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	51,853	[1]	51,713	[1]
New York and Other States [Member] | Home Equity Lines of Credit [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	306,196	[1]	308,764	[1]
New York and Other States [Member] | Installment [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	7,231	[1]	6,774	[1]
Florida [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	571,075		545,953
Florida [Member] | Commercial [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	16,017		19,394
Florida [Member] | Commercial Real Estate [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	15,968		19,336
Florida [Member] | Other [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	49		58
Florida [Member] | First Mortgages [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	500,633		476,427
Florida [Member] | Home Equity Loans [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	7,297		5,942
Florida [Member] | Home Equity Lines of Credit [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	46,356		43,370
Florida [Member] | Installment [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	$ 772		$ 820

[1] Includes New York, New Jersey, Vermont and Massachusetts.