Loans and Allowance for Loan Losses, Based on Impairment Review Method (Details) (USD $) In Thousands, unless otherwise specified	Dec. 31, 2013		Dec. 31, 2012		Dec. 31, 2011	Dec. 31, 2010
Allowance for loan losses [Abstract]
Individually evaluated for impairment	$ 0		$ 0
Collectively evaluated for impairment	47,714		47,927
Total ending allowance balance	47,714		47,927		48,717	41,911
Loans [Abstract]
Individually evaluated for impairment	29,340		26,073
Collectively evaluated for impairment	2,879,469		2,658,660
Total ending loans balance	2,908,809		2,684,733
Commercial [Member]
Allowance for loan losses [Abstract]
Individually evaluated for impairment	0		0
Collectively evaluated for impairment	4,019		3,771
Total ending allowance balance	4,019		3,771		4,140	4,228
Loans [Abstract]
Individually evaluated for impairment	8,082		9,333
Collectively evaluated for impairment	215,399		210,244
Total ending loans balance	223,481		219,577
Real Estate Mortgage - 1 to 4 Family [Member]
Allowance for loan losses [Abstract]
Individually evaluated for impairment	0		0
Collectively evaluated for impairment	43,597		44,069
Total ending allowance balance	43,597		44,069		44,479	37,447
Loans [Abstract]
Individually evaluated for impairment	21,258		16,740
Collectively evaluated for impairment	2,658,175		2,443,837
Total ending loans balance	2,679,433		2,460,577
Installment [Member]
Allowance for loan losses [Abstract]
Individually evaluated for impairment	0		0
Collectively evaluated for impairment	98		87
Total ending allowance balance	98		87		98	236
Loans [Abstract]
Individually evaluated for impairment	0		0
Collectively evaluated for impairment	5,895		4,579
Total ending loans balance	5,895		4,579
New York and Other States [Member]
Loans [Abstract]
Total ending loans balance	2,468,322	[1]	2,355,966	[1]
New York and Other States [Member] | Commercial [Member]
Loans [Abstract]
Total ending loans balance	202,045		200,630
New York and Other States [Member] | Installment [Member]
Loans [Abstract]
Total ending loans balance	5,292	[1]	4,183	[1]
Florida [Member]
Loans [Abstract]
Total ending loans balance	440,487		328,767
Florida [Member] | Commercial [Member]
Loans [Abstract]
Total ending loans balance	21,436		18,947
Florida [Member] | Installment [Member]
Loans [Abstract]
Total ending loans balance	$ 603		$ 396

[1] Includes New York, New Jersey, Vermont, and Massachusetts.