Loans and Allowance for Loan Losses, Risk Category of Loans (Details) - USD ($) $ in Thousands		Sep. 30, 2017	Dec. 31, 2016
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		$ 3,578,282	$ 3,430,586
Non-accrual loans included in classified loans		1,200	1,800
New York and Other States [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	[1]	2,763,629	2,692,335
Florida [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		814,653	738,251
Commercial [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		187,281	191,194
Commercial [Member] | New York and Other States [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		173,982	178,905
Commercial [Member] | Florida [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		13,299	12,289
Commercial [Member] | Pass [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		175,843	174,407
Commercial [Member] | Pass [Member] | New York and Other States [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		162,544	162,118
Commercial [Member] | Pass [Member] | Florida [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		13,299	12,289
Commercial [Member] | Classified [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		11,438	16,787
Commercial [Member] | Classified [Member] | New York and Other States [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		11,438	16,787
Commercial [Member] | Classified [Member] | Florida [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		0	0
Commercial [Member] | Commercial Real Estate [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		165,208	163,609
Commercial [Member] | Commercial Real Estate [Member] | New York and Other States [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	[1]	152,228	151,366
Commercial [Member] | Commercial Real Estate [Member] | Florida [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		12,980	12,243
Commercial [Member] | Commercial Real Estate [Member] | Pass [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		155,370	148,919
Commercial [Member] | Commercial Real Estate [Member] | Pass [Member] | New York and Other States [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		142,390	136,676
Commercial [Member] | Commercial Real Estate [Member] | Pass [Member] | Florida [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		12,980	12,243
Commercial [Member] | Commercial Real Estate [Member] | Classified [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		9,838	14,690
Commercial [Member] | Commercial Real Estate [Member] | Classified [Member] | New York and Other States [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		9,838	14,690
Commercial [Member] | Commercial Real Estate [Member] | Classified [Member] | Florida [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		0	0
Commercial [Member] | Other [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		22,073	27,585
Commercial [Member] | Other [Member] | New York and Other States [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	[1]	21,754	27,539
Commercial [Member] | Other [Member] | Florida [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		319	46
Commercial [Member] | Other [Member] | Pass [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		20,473	25,488
Commercial [Member] | Other [Member] | Pass [Member] | New York and Other States [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		20,154	25,442
Commercial [Member] | Other [Member] | Pass [Member] | Florida [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		319	46
Commercial [Member] | Other [Member] | Classified [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		1,600	2,097
Commercial [Member] | Other [Member] | Classified [Member] | New York and Other States [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		1,600	2,097
Commercial [Member] | Other [Member] | Classified [Member] | Florida [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		$ 0	$ 0

[1] Includes New York, New Jersey, Vermont and Massachusetts.