Loans and Allowance for Loan Losses, Allowance for Loan Losses and Recorded Investment in Loans Based on Impairment Method (Details) (USD $) In Thousands, unless otherwise specified	Sep. 30, 2013		Jun. 30, 2013	Dec. 31, 2012		Sep. 30, 2012	Jun. 30, 2012	Dec. 31, 2011
Allowance for loan losses [Abstract]
Individually evaluated for impairment	$ 0			$ 0
Collectively evaluated for impairment	47,722			47,927
Total ending allowance balance	47,722		47,589	47,927		47,364	48,018	48,717
Loans [Abstract]
Individually evaluated for impairment	26,049			26,073
Collectively evaluated for impairment	2,808,920			2,658,660
Total ending loans balance	2,834,969			2,684,733
Commercial [Member]
Allowance for loan losses [Abstract]
Individually evaluated for impairment	0			0
Collectively evaluated for impairment	3,864			3,771
Total ending allowance balance	3,864		3,719	3,771		3,860	3,792	4,140
Loans [Abstract]
Individually evaluated for impairment	5,436			9,333
Collectively evaluated for impairment	207,397			210,244
Total ending loans balance	212,833			219,577
Real Estate Mortgage - 1 to 4 Family [Member]
Allowance for loan losses [Abstract]
Individually evaluated for impairment	0			0
Collectively evaluated for impairment	43,750			44,069
Total ending allowance balance	43,750		43,766	44,069		43,426	44,147	44,479
Loans [Abstract]
Individually evaluated for impairment	20,613			16,740
Collectively evaluated for impairment	2,595,629			2,443,837
Total ending loans balance	2,616,242			2,460,577
Installment [Member]
Allowance for loan losses [Abstract]
Individually evaluated for impairment	0			0
Collectively evaluated for impairment	108			87
Total ending allowance balance	108		104	87		78	79	98
Loans [Abstract]
Individually evaluated for impairment	0			0
Collectively evaluated for impairment	5,894			4,579
Total ending loans balance	5,894			4,579
New York and Other States [Member]
Loans [Abstract]
Total ending loans balance	2,416,857	[1]		2,355,966	[1]
New York and Other States [Member] | Commercial [Member]
Loans [Abstract]
Total ending loans balance	191,436			200,630
New York and Other States [Member] | Installment [Member]
Loans [Abstract]
Total ending loans balance	5,208	[1]		4,183	[1]
Florida [Member]
Loans [Abstract]
Total ending loans balance	418,112			328,767
Florida [Member] | Commercial [Member]
Loans [Abstract]
Total ending loans balance	21,397			18,947
Florida [Member] | Installment [Member]
Loans [Abstract]
Total ending loans balance	$ 686			$ 396

[1] Includes New York, New Jersey, Vermont and Massachusetts.