Loans and Allowance for Loan Losses (Details) (USD $) In Thousands, unless otherwise specified	Jun. 30, 2014		Dec. 31, 2013
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	$ 3,006,150		$ 2,908,809
Less: Allowance for loan losses	46,935		47,714
Net loans	2,959,215		2,861,095
New York and Other States [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	2,517,744	[1]	2,468,322	[1]
Florida [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	488,406		440,487
Commercial [Member] | New York and Other States [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	200,679		202,045
Commercial [Member] | Florida [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	21,976		21,436
Commercial Real Estate [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	189,723		191,126
Commercial Real Estate [Member] | New York and Other States [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	167,815		169,722
Commercial Real Estate [Member] | Florida [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	21,908		21,404
Other [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	32,932		32,355
Other [Member] | New York and Other States [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	32,864		32,323
Other [Member] | Florida [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	68		32
First Mortgages [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	2,383,232		2,287,808
First Mortgages [Member] | New York and Other States [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	1,960,668		1,909,447
First Mortgages [Member] | Florida [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	422,564		378,361
Home Equity Loans [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	54,268		51,136
Home Equity Loans [Member] | New York and Other States [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	49,695		47,494
Home Equity Loans [Member] | Florida [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	4,573		3,642
Home Equity Lines of Credit [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	339,897		340,489
Home Equity Lines of Credit [Member] | New York and Other States [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	301,236		304,044
Home Equity Lines of Credit [Member] | Florida [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	38,661		36,445
Installment [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	6,098		5,895
Installment [Member] | New York and Other States [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	5,466		5,292
Installment [Member] | Florida [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	632		603
Real Estate Construction Loans [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	34,300		35,400
Residential Borrowers [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	15,400		13,900
Commercial Borrowers Residential Purpose [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	$ 18,900		$ 21,500

[1] Includes New York, New Jersey, Vermont and Massachusetts.