Loans and Allowance for Loan Losses, Risk Category of Loans (Details) (USD $)	Sep. 30, 2014		Dec. 31, 2013
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	$ 3,083,029,000		$ 2,908,809,000
Impaired loans included in classified loans	5,000,000		8,100,000
Commercial Real Estate [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	192,169,000		191,126,000
Other [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	27,656,000		32,355,000
New York and Other States [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	2,566,489,000	[1]	2,468,322,000	[1]
New York and Other States [Member] | Commercial [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	200,052,000		202,045,000
New York and Other States [Member] | Commercial [Member] | Pass [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	190,227,000		190,715,000
New York and Other States [Member] | Commercial [Member] | Classified [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	9,825,000		11,330,000
New York and Other States [Member] | Commercial Real Estate [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	172,457,000	[1]	169,722,000	[1]
New York and Other States [Member] | Commercial Real Estate [Member] | Pass [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	163,137,000		159,024,000
New York and Other States [Member] | Commercial Real Estate [Member] | Classified [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	9,320,000		10,698,000
New York and Other States [Member] | Other [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	27,595,000	[1]	32,323,000	[1]
New York and Other States [Member] | Other [Member] | Pass [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	27,090,000		31,691,000
New York and Other States [Member] | Other [Member] | Classified [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	505,000		632,000
Florida [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	516,540,000		440,487,000
Florida [Member] | Commercial [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	19,773,000		21,436,000
Florida [Member] | Commercial [Member] | Pass [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	19,256,000		20,306,000
Florida [Member] | Commercial [Member] | Classified [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	517,000		1,130,000
Florida [Member] | Commercial Real Estate [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	19,712,000		21,404,000
Florida [Member] | Commercial Real Estate [Member] | Pass [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	19,195,000		20,274,000
Florida [Member] | Commercial Real Estate [Member] | Classified [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	517,000		1,130,000
Florida [Member] | Other [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	61,000		32,000
Florida [Member] | Other [Member] | Pass [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	61,000		32,000
Florida [Member] | Other [Member] | Classified [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	$ 0		$ 0

[1] Includes New York, New Jersey, Vermont and Massachusetts.