Loans and Allowance for Loan Losses, Risk Category of Loans (Details) (USD $)	Dec. 31, 2013		Dec. 31, 2012
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	$ 2,908,809,000		$ 2,684,733,000
Impaired loans included in classified loans	8,100,000		9,300,000
Commercial [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	223,481,000		219,577,000
Commercial [Member] | Commercial Real Estate [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	191,126,000		186,131,000
Commercial [Member] | Other [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	32,355,000		33,446,000
New York and Other States [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	2,468,322,000	[1]	2,355,966,000	[1]
New York and Other States [Member] | Commercial [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	202,045,000		200,630,000
New York and Other States [Member] | Commercial [Member] | Pass [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	190,715,000		189,048,000
New York and Other States [Member] | Commercial [Member] | Classified [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	11,330,000		11,582,000
New York and Other States [Member] | Commercial [Member] | Commercial Real Estate [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	169,722,000	[1]	167,249,000	[1]
New York and Other States [Member] | Commercial [Member] | Commercial Real Estate [Member] | Pass [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	159,024,000		155,981,000
New York and Other States [Member] | Commercial [Member] | Commercial Real Estate [Member] | Classified [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	10,698,000		11,268,000
New York and Other States [Member] | Commercial [Member] | Other [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	32,323,000	[1]	33,381,000	[1]
New York and Other States [Member] | Commercial [Member] | Other [Member] | Pass [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	31,691,000		33,067,000
New York and Other States [Member] | Commercial [Member] | Other [Member] | Classified [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	632,000		314,000
Florida [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	440,487,000		328,767,000
Florida [Member] | Commercial [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	21,436,000		18,947,000
Florida [Member] | Commercial [Member] | Pass [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	20,306,000		12,519,000
Florida [Member] | Commercial [Member] | Classified [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	1,130,000		6,428,000
Florida [Member] | Commercial [Member] | Commercial Real Estate [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	21,404,000		18,882,000
Florida [Member] | Commercial [Member] | Commercial Real Estate [Member] | Pass [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	20,274,000		12,454,000
Florida [Member] | Commercial [Member] | Commercial Real Estate [Member] | Classified [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	1,130,000		6,428,000
Florida [Member] | Commercial [Member] | Other [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	32,000		65,000
Florida [Member] | Commercial [Member] | Other [Member] | Pass [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	32,000		65,000
Florida [Member] | Commercial [Member] | Other [Member] | Classified [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	$ 0		$ 0

[1] Includes New York, New Jersey, Vermont, and Massachusetts.