Loans and Allowance for Loan Losses (Details) (USD $) In Thousands, unless otherwise specified	Sep. 30, 2014		Dec. 31, 2013
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	$ 3,083,029		$ 2,908,809
Less: Allowance for loan losses	46,512		47,714
Net loans	3,036,517		2,861,095
New York and Other States [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	2,566,489	[1]	2,468,322	[1]
Florida [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	516,540		440,487
Commercial [Member] | New York and Other States [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	200,052		202,045
Commercial [Member] | Florida [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	19,773		21,436
Commercial Real Estate [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	192,169		191,126
Commercial Real Estate [Member] | New York and Other States [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	172,457	[1]	169,722	[1]
Commercial Real Estate [Member] | Florida [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	19,712		21,404
Other [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	27,656		32,355
Other [Member] | New York and Other States [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	27,595	[1]	32,323	[1]
Other [Member] | Florida [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	61		32
First Mortgages [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	2,453,430		2,287,808
First Mortgages [Member] | New York and Other States [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	2,002,935	[1]	1,909,447	[1]
First Mortgages [Member] | Florida [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	450,495		378,361
Home Equity Loans [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	56,721		51,136
Home Equity Loans [Member] | New York and Other States [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	51,430	[1]	47,494	[1]
Home Equity Loans [Member] | Florida [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	5,291		3,642
Home Equity Lines of Credit [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	346,496		340,489
Home Equity Lines of Credit [Member] | New York and Other States [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	306,138	[1]	304,044	[1]
Home Equity Lines of Credit [Member] | Florida [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	40,358		36,445
Installment [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	6,557		5,895
Installment [Member] | New York and Other States [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	5,934	[1]	5,292	[1]
Installment [Member] | Florida [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	623		603
Real Estate Construction Loans [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	35,800		35,400
Residential Borrowers [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	14,300		13,900
Commercial Borrowers Residential Purpose [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	$ 21,500		$ 21,500

[1] Includes New York, New Jersey, Vermont and Massachusetts.