Loans and Allowance for Loan Losses, Risk Category of Loans (Details) (USD $)	Jun. 30, 2014		Dec. 31, 2013
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	$ 3,006,150,000		$ 2,908,809,000
Impaired loans included in classified loans	5,600,000		8,100,000
Commercial Real Estate [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	189,723,000		191,126,000
Other [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	32,932,000		32,355,000
New York and Other States [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	2,517,744,000	[1]	2,468,322,000	[1]
New York and Other States [Member] | Commercial [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	200,679,000		202,045,000
New York and Other States [Member] | Commercial [Member] | Pass [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	191,797,000		190,715,000
New York and Other States [Member] | Commercial [Member] | Classified [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	8,882,000		11,330,000
New York and Other States [Member] | Commercial Real Estate [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	167,815,000		169,722,000
New York and Other States [Member] | Commercial Real Estate [Member] | Pass [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	159,539,000		159,024,000
New York and Other States [Member] | Commercial Real Estate [Member] | Classified [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	8,276,000		10,698,000
New York and Other States [Member] | Other [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	32,864,000		32,323,000
New York and Other States [Member] | Other [Member] | Pass [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	32,258,000		31,691,000
New York and Other States [Member] | Other [Member] | Classified [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	606,000		632,000
Florida [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	488,406,000		440,487,000
Florida [Member] | Commercial [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	21,976,000		21,436,000
Florida [Member] | Commercial [Member] | Pass [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	21,459,000		20,306,000
Florida [Member] | Commercial [Member] | Classified [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	517,000		1,130,000
Florida [Member] | Commercial Real Estate [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	21,908,000		21,404,000
Florida [Member] | Commercial Real Estate [Member] | Pass [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	21,391,000		20,274,000
Florida [Member] | Commercial Real Estate [Member] | Classified [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	517,000		1,130,000
Florida [Member] | Other [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	68,000		32,000
Florida [Member] | Other [Member] | Pass [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	68,000		32,000
Florida [Member] | Other [Member] | Classified [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	$ 0		$ 0

[1] Includes New York, New Jersey, Vermont and Massachusetts.