Loans and Allowance for Loan Losses, Risk Category of Loans (Details) (USD $)	12 Months Ended
	Dec. 31, 2014		Dec. 31, 2013
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	$ 3,158,332,000		$ 2,908,809,000
Impaired loans included in classified loans	4,100,000		8,100,000
Non-homogeneous loans over	150,000
Internal grading of non-homogeneous loans over	250,000
Commercial [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	223,382,000		223,481,000
Commercial [Member] | Pass [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	210,660,000		211,021,000
Commercial [Member] | Classified [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	12,722,000		12,460,000
Commercial Real Estate [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	194,124,000		191,126,000
Commercial Real Estate [Member] | Pass [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	181,925,000		179,298,000
Commercial Real Estate [Member] | Classified [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	12,199,000		11,828,000
Other [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	29,258,000		32,355,000
Other [Member] | Pass [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	28,735,000		31,723,000
Other [Member] | Classified [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	523,000		632,000
New York and Other States [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	2,612,379,000	[1]	2,468,322,000	[1]
New York and Other States [Member] | Commercial [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	203,988,000		202,045,000
New York and Other States [Member] | Commercial [Member] | Pass [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	191,266,000		190,715,000
New York and Other States [Member] | Commercial [Member] | Classified [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	12,722,000		11,330,000
New York and Other States [Member] | Commercial Real Estate [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	174,788,000	[1]	169,722,000	[1]
New York and Other States [Member] | Commercial Real Estate [Member] | Pass [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	162,589,000		159,024,000
New York and Other States [Member] | Commercial Real Estate [Member] | Classified [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	12,199,000		10,698,000
New York and Other States [Member] | Other [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	29,200,000	[1]	32,323,000	[1]
New York and Other States [Member] | Other [Member] | Pass [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	28,677,000		31,691,000
New York and Other States [Member] | Other [Member] | Classified [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	523,000		632,000
Florida [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	545,953,000		440,487,000
Florida [Member] | Commercial [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	19,394,000		21,436,000
Florida [Member] | Commercial [Member] | Pass [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	19,394,000		20,306,000
Florida [Member] | Commercial [Member] | Classified [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	0		1,130,000
Florida [Member] | Commercial Real Estate [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	19,336,000		21,404,000
Florida [Member] | Commercial Real Estate [Member] | Pass [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	19,336,000		20,274,000
Florida [Member] | Commercial Real Estate [Member] | Classified [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	0		1,130,000
Florida [Member] | Other [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	58,000		32,000
Florida [Member] | Other [Member] | Pass [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	58,000		32,000
Florida [Member] | Other [Member] | Classified [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	$ 0		$ 0

[1] Includes New York, New Jersey, Vermont, and Massachusetts.