Loans and Allowance for Loan Losses, Risk Category of Loans (Details) - USD ($) $ in Thousands		Sep. 30, 2015	Dec. 31, 2014
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		$ 3,282,005	$ 3,158,332
Impaired loans included in classified loans		3,700	4,100
Commercial [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		208,794	223,382
Commercial [Member] | Pass [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		193,705	210,660
Commercial [Member] | Classified [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		15,089	12,722
Commercial [Member] | Commercial Real Estate [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		185,919	194,124
Commercial [Member] | Commercial Real Estate [Member] | Pass [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		171,570	181,925
Commercial [Member] | Commercial Real Estate [Member] | Classified [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		14,349	12,199
Commercial [Member] | Other [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		22,875	29,258
Commercial [Member] | Other [Member] | Pass [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		22,135	28,735
Commercial [Member] | Other [Member] | Classified [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		740	523
New York and Other States [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	[1]	2,654,151	2,612,379
New York and Other States [Member] | Commercial [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		192,961	203,988
New York and Other States [Member] | Commercial [Member] | Pass [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		177,872	191,266
New York and Other States [Member] | Commercial [Member] | Classified [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		15,089	12,722
New York and Other States [Member] | Commercial [Member] | Commercial Real Estate [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	[1]	170,185	174,788
New York and Other States [Member] | Commercial [Member] | Commercial Real Estate [Member] | Pass [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		155,836	162,589
New York and Other States [Member] | Commercial [Member] | Commercial Real Estate [Member] | Classified [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		14,349	12,199
New York and Other States [Member] | Commercial [Member] | Other [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	[1]	22,776	29,200
New York and Other States [Member] | Commercial [Member] | Other [Member] | Pass [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		22,036	28,677
New York and Other States [Member] | Commercial [Member] | Other [Member] | Classified [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		740	523
Florida [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		627,854	545,953
Florida [Member] | Commercial [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		15,833	19,394
Florida [Member] | Commercial [Member] | Pass [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		15,833	19,394
Florida [Member] | Commercial [Member] | Classified [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		0	0
Florida [Member] | Commercial [Member] | Commercial Real Estate [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		15,734	19,336
Florida [Member] | Commercial [Member] | Commercial Real Estate [Member] | Pass [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		15,734	19,336
Florida [Member] | Commercial [Member] | Commercial Real Estate [Member] | Classified [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		0	0
Florida [Member] | Commercial [Member] | Other [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		99	58
Florida [Member] | Commercial [Member] | Other [Member] | Pass [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		99	58
Florida [Member] | Commercial [Member] | Other [Member] | Classified [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		$ 0	$ 0

[1] Includes New York, New Jersey, Vermont and Massachusetts.