Loans and Allowance for Loan Losses (Details) (USD $) In Thousands, unless otherwise specified	Mar. 31, 2014		Dec. 31, 2013
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	$ 2,941,002		$ 2,908,809
Less: Allowance for loan losses	47,035		47,714
Net loans	2,893,967		2,861,095
Commercial [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	220,443		223,481
Commercial [Member] | Commercial Real Estate [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	190,323		191,126
Commercial [Member] | Other [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	30,120		32,355
Real Estate Mortgage - 1 to 4 Family [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	2,714,845		2,679,433
Real Estate Mortgage - 1 to 4 Family [Member] | First Mortgages [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	2,322,422		2,287,808
Real Estate Mortgage - 1 to 4 Family [Member] | Home Equity Loans [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	52,452		51,136
Real Estate Mortgage - 1 to 4 Family [Member] | Home Equity Lines of Credit [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	339,971		340,489
Installment [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	5,714		5,895
Real Estate Construction Loans [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	35,200		35,400
Residential Borrowers [Member] | First Mortgages [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	15,700		13,900
Commercial Borrowers Residential Purpose [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	19,500		21,500
New York and Other States [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	2,483,028	[1]	2,468,322	[1]
New York and Other States [Member] | Commercial [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	198,273	[1]	202,045	[1]
New York and Other States [Member] | Commercial [Member] | Commercial Real Estate [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	168,226	[1]	169,722	[1]
New York and Other States [Member] | Commercial [Member] | Other [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	30,047	[1]	32,323	[1]
New York and Other States [Member] | Real Estate Mortgage - 1 to 4 Family [Member] | First Mortgages [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	1,928,369	[1]	1,909,447	[1]
New York and Other States [Member] | Real Estate Mortgage - 1 to 4 Family [Member] | Home Equity Loans [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	48,249	[1]	47,494	[1]
New York and Other States [Member] | Real Estate Mortgage - 1 to 4 Family [Member] | Home Equity Lines of Credit [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	303,023	[1]	304,044	[1]
New York and Other States [Member] | Installment [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	5,114	[1]	5,292	[1]
Florida [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	457,974		440,487
Florida [Member] | Commercial [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	22,170		21,436
Florida [Member] | Commercial [Member] | Commercial Real Estate [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	22,097		21,404
Florida [Member] | Commercial [Member] | Other [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	73		32
Florida [Member] | Real Estate Mortgage - 1 to 4 Family [Member] | First Mortgages [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	394,053		378,361
Florida [Member] | Real Estate Mortgage - 1 to 4 Family [Member] | Home Equity Loans [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	4,203		3,642
Florida [Member] | Real Estate Mortgage - 1 to 4 Family [Member] | Home Equity Lines of Credit [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	36,948		36,445
Florida [Member] | Installment [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	$ 600		$ 603

[1] Includes New York, New Jersey, Vermont and Massachusetts.