Loans and Allowance for Loan Losses, Allowance for Loan Losses (Details) (USD $) In Thousands, unless otherwise specified	12 Months Ended
	Dec. 31, 2012		Dec. 31, 2011		Dec. 31, 2010
Allowance for loan losses by portfolio segment [Roll Forward]
Balance at beginning of period	$ 48,717		$ 41,911		$ 37,591
Loans charged off [Abstract]
Total loan chargeoffs	13,479		12,558		19,868
Recoveries of loans previously charged off [Abstract]
Total recoveries	689		614		988
Net loans charged off	12,790		11,944		18,880
Provision for loan losses	12,000		18,750		23,200
Balance at end of period	47,927		48,717		41,911
Allowance for loan losses [Abstract]
Individually evaluated for impairment	0		0
Collectively evaluated for impairment	47,927		48,717
Total ending allowance balance	47,927		48,717		41,911
Loans [Abstract]
Individually evaluated for impairment	26,073		13,711
Collectively evaluated for impairment	2,658,660		2,507,592
Total ending loans balance	2,684,733		2,521,303
Commercial [Member]
Recoveries of loans previously charged off [Abstract]
Balance at end of period	3,771		4,140
Allowance for loan losses [Abstract]
Individually evaluated for impairment	0		0
Collectively evaluated for impairment	3,771		4,140
Total ending allowance balance	3,771		4,140
Loans [Abstract]
Individually evaluated for impairment	9,333		9,981
Collectively evaluated for impairment	210,244		238,182
Total ending loans balance	219,577		248,163
Real Estate Mortgage - 1 to 4 Family [Member]
Recoveries of loans previously charged off [Abstract]
Balance at end of period	44,069		44,479
Allowance for loan losses [Abstract]
Individually evaluated for impairment	0		0
Collectively evaluated for impairment	44,069		44,479
Total ending allowance balance	44,069		44,479
Loans [Abstract]
Individually evaluated for impairment	16,740		3,730
Collectively evaluated for impairment	2,443,837		2,265,259
Total ending loans balance	2,460,577		2,268,989
Installment [Member]
Recoveries of loans previously charged off [Abstract]
Balance at end of period	87		98
Allowance for loan losses [Abstract]
Individually evaluated for impairment	0		0
Collectively evaluated for impairment	87		98
Total ending allowance balance	87		98
Loans [Abstract]
Individually evaluated for impairment	0		0
Collectively evaluated for impairment	4,579		4,151
Total ending loans balance	4,579		4,151
New York and Other States [Member]
Loans [Abstract]
Total ending loans balance	2,355,966	[1]	2,289,884	[1]
New York and Other States [Member] | Commercial [Member]
Loans charged off [Abstract]
Total loan chargeoffs	1,307		171		141
Recoveries of loans previously charged off [Abstract]
Total recoveries	112		55		34
Loans [Abstract]
Total ending loans balance	200,630		222,835
New York and Other States [Member] | Real Estate Mortgage - 1 to 4 Family [Member]
Loans charged off [Abstract]
Total loan chargeoffs	6,919		4,315		3,754
Recoveries of loans previously charged off [Abstract]
Total recoveries	375		477		700
New York and Other States [Member] | Installment [Member]
Loans charged off [Abstract]
Total loan chargeoffs	128		80		150
Recoveries of loans previously charged off [Abstract]
Total recoveries	49		43		95
Loans [Abstract]
Total ending loans balance	4,183	[1]	4,078	[1]
Florida [Member]
Loans [Abstract]
Total ending loans balance	328,767		231,419
Florida [Member] | Commercial [Member]
Loans charged off [Abstract]
Total loan chargeoffs	1,192		1,000		4,940
Recoveries of loans previously charged off [Abstract]
Total recoveries	26		4		69
Loans [Abstract]
Total ending loans balance	18,947		25,328
Florida [Member] | Real Estate Mortgage - 1 to 4 Family [Member]
Loans charged off [Abstract]
Total loan chargeoffs	3,920		6,990		10,878
Recoveries of loans previously charged off [Abstract]
Total recoveries	127		34		89
Florida [Member] | Installment [Member]
Loans charged off [Abstract]
Total loan chargeoffs	13		2		5
Recoveries of loans previously charged off [Abstract]
Total recoveries	0		1		1
Loans [Abstract]
Total ending loans balance	$ 396		$ 73

[1] Includes New York, New Jersey, Vermont, and Massachusetts