Loans and Allowance for Loan Losses, Past Due (Details) (USD $) In Thousands, unless otherwise specified	12 Months Ended
	Dec. 31, 2013		Dec. 31, 2012
Aging of the recorded investment in past due loans [Abstract]
30-59 Days Past Due	$ 7,262		$ 8,115
60-89 Days Past Due	3,831		3,322
90+ Days Past Due	28,128		36,609
Total 30+ days Past Due	39,221		48,046
Current	2,869,588		2,636,687
Total loans	2,908,809		2,684,733
Loans that are 90 days past due and still accruing interest	0		0
Number of days past due	90 days
Commercial [Member]
Aging of the recorded investment in past due loans [Abstract]
Total loans	223,481		219,577
Commercial [Member] | Commercial Real Estate [Member]
Aging of the recorded investment in past due loans [Abstract]
30-59 Days Past Due	583		0
60-89 Days Past Due	1,426		0
90+ Days Past Due	3,379		5,923
Total 30+ days Past Due	5,388		5,923
Current	185,738		180,208
Total loans	191,126		186,131
Commercial [Member] | Other [Member]
Aging of the recorded investment in past due loans [Abstract]
30-59 Days Past Due	209		0
60-89 Days Past Due	0		0
90+ Days Past Due	123		4
Total 30+ days Past Due	332		4
Current	32,023		33,442
Total loans	32,355		33,446
Real Estate Mortgage - 1 to 4 Family [Member]
Aging of the recorded investment in past due loans [Abstract]
Total loans	2,679,433		2,460,577
Real Estate Mortgage - 1 to 4 Family [Member] | First Mortgages [Member]
Aging of the recorded investment in past due loans [Abstract]
30-59 Days Past Due	5,216		7,226
60-89 Days Past Due	2,042		2,700
90+ Days Past Due	21,853		26,731
Total 30+ days Past Due	29,111		36,657
Current	2,258,697		2,053,321
Total loans	2,287,808		2,089,978
Real Estate Mortgage - 1 to 4 Family [Member] | Home Equity Loans [Member]
Aging of the recorded investment in past due loans [Abstract]
30-59 Days Past Due	46		177
60-89 Days Past Due	18		216
90+ Days Past Due	552		464
Total 30+ days Past Due	616		857
Current	50,520		35,833
Total loans	51,136		36,690
Real Estate Mortgage - 1 to 4 Family [Member] | Home Equity Lines of Credit [Member]
Aging of the recorded investment in past due loans [Abstract]
30-59 Days Past Due	1,123		663
60-89 Days Past Due	331		379
90+ Days Past Due	2,144		3,486
Total 30+ days Past Due	3,598		4,528
Current	336,891		329,381
Total loans	340,489		333,909
Installment [Member]
Aging of the recorded investment in past due loans [Abstract]
30-59 Days Past Due	85		49
60-89 Days Past Due	14		27
90+ Days Past Due	77		1
Total 30+ days Past Due	176		77
Current	5,719		4,502
Total loans	5,895		4,579
New York and Other States [Member]
Aging of the recorded investment in past due loans [Abstract]
30-59 Days Past Due	6,601		7,185
60-89 Days Past Due	3,829		2,841
90+ Days Past Due	23,652		28,078
Total 30+ days Past Due	34,082		38,104
Current	2,434,240		2,317,862
Total loans	2,468,322	[1]	2,355,966	[1]
New York and Other States [Member] | Commercial [Member]
Aging of the recorded investment in past due loans [Abstract]
Total loans	202,045		200,630
New York and Other States [Member] | Commercial [Member] | Commercial Real Estate [Member]
Aging of the recorded investment in past due loans [Abstract]
30-59 Days Past Due	583		0
60-89 Days Past Due	1,426		0
90+ Days Past Due	3,379		3,225
Total 30+ days Past Due	5,388		3,225
Current	164,334		164,024
Total loans	169,722	[1]	167,249	[1]
New York and Other States [Member] | Commercial [Member] | Other [Member]
Aging of the recorded investment in past due loans [Abstract]
30-59 Days Past Due	209		0
60-89 Days Past Due	0		0
90+ Days Past Due	123		4
Total 30+ days Past Due	332		4
Current	31,991		33,377
Total loans	32,323	[1]	33,381	[1]
New York and Other States [Member] | Real Estate Mortgage - 1 to 4 Family [Member] | First Mortgages [Member]
Aging of the recorded investment in past due loans [Abstract]
30-59 Days Past Due	4,664		6,364
60-89 Days Past Due	2,042		2,248
90+ Days Past Due	17,624		21,341
Total 30+ days Past Due	24,330		29,953
Current	1,885,117		1,784,261
Total loans	1,909,447	[1]	1,814,214	[1]
New York and Other States [Member] | Real Estate Mortgage - 1 to 4 Family [Member] | Home Equity Loans [Member]
Aging of the recorded investment in past due loans [Abstract]
30-59 Days Past Due	46		177
60-89 Days Past Due	18		216
90+ Days Past Due	552		464
Total 30+ days Past Due	616		857
Current	46,878		34,744
Total loans	47,494	[1]	35,601	[1]
New York and Other States [Member] | Real Estate Mortgage - 1 to 4 Family [Member] | Home Equity Lines of Credit [Member]
Aging of the recorded investment in past due loans [Abstract]
30-59 Days Past Due	1,014		604
60-89 Days Past Due	331		350
90+ Days Past Due	1,897		3,044
Total 30+ days Past Due	3,242		3,998
Current	300,802		297,340
Total loans	304,044	[1]	301,338	[1]
New York and Other States [Member] | Installment [Member]
Aging of the recorded investment in past due loans [Abstract]
30-59 Days Past Due	85		40
60-89 Days Past Due	12		27
90+ Days Past Due	77		0
Total 30+ days Past Due	174		67
Current	5,118		4,116
Total loans	5,292	[1]	4,183	[1]
Florida [Member]
Aging of the recorded investment in past due loans [Abstract]
30-59 Days Past Due	661		930
60-89 Days Past Due	2		481
90+ Days Past Due	4,476		8,531
Total 30+ days Past Due	5,139		9,942
Current	435,348		318,825
Total loans	440,487		328,767
Florida [Member] | Commercial [Member]
Aging of the recorded investment in past due loans [Abstract]
Total loans	21,436		18,947
Florida [Member] | Commercial [Member] | Commercial Real Estate [Member]
Aging of the recorded investment in past due loans [Abstract]
30-59 Days Past Due	0		0
60-89 Days Past Due	0		0
90+ Days Past Due	0		2,698
Total 30+ days Past Due	0		2,698
Current	21,404		16,184
Total loans	21,404		18,882
Florida [Member] | Commercial [Member] | Other [Member]
Aging of the recorded investment in past due loans [Abstract]
30-59 Days Past Due	0		0
60-89 Days Past Due	0		0
90+ Days Past Due	0		0
Total 30+ days Past Due	0		0
Current	32		65
Total loans	32		65
Florida [Member] | Real Estate Mortgage - 1 to 4 Family [Member] | First Mortgages [Member]
Aging of the recorded investment in past due loans [Abstract]
30-59 Days Past Due	552		862
60-89 Days Past Due	0		452
90+ Days Past Due	4,229		5,390
Total 30+ days Past Due	4,781		6,704
Current	373,580		269,060
Total loans	378,361		275,764
Florida [Member] | Real Estate Mortgage - 1 to 4 Family [Member] | Home Equity Loans [Member]
Aging of the recorded investment in past due loans [Abstract]
30-59 Days Past Due	0		0
60-89 Days Past Due	0		0
90+ Days Past Due	0		0
Total 30+ days Past Due	0		0
Current	3,642		1,089
Total loans	3,642		1,089
Florida [Member] | Real Estate Mortgage - 1 to 4 Family [Member] | Home Equity Lines of Credit [Member]
Aging of the recorded investment in past due loans [Abstract]
30-59 Days Past Due	109		59
60-89 Days Past Due	0		29
90+ Days Past Due	247		442
Total 30+ days Past Due	356		530
Current	36,089		32,041
Total loans	36,445		32,571
Florida [Member] | Installment [Member]
Aging of the recorded investment in past due loans [Abstract]
30-59 Days Past Due	0		9
60-89 Days Past Due	2		0
90+ Days Past Due	0		1
Total 30+ days Past Due	2		10
Current	601		386
Total loans	$ 603		$ 396

[1] Includes New York, New Jersey, Vermont, and Massachusetts.