Loans and Allowance for Loan Losses, Risk Category of Loans (Details) - USD ($) $ in Thousands		Mar. 31, 2018	Dec. 31, 2017
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		$ 3,666,975	$ 3,636,407
Non-accrual loans included in classified loans		1,900	2,200
New York and Other States [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	[1]	2,811,286	2,795,993
Florida [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		855,689	840,414
Commercial [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		185,129	186,207
Commercial [Member] | New York and Other States [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		172,816	172,974
Commercial [Member] | Florida [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		12,313	13,233
Commercial [Member] | Pass [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		176,618	175,857
Commercial [Member] | Pass [Member] | New York and Other States [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		164,305	162,742
Commercial [Member] | Pass [Member] | Florida [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		12,313	13,115
Commercial [Member] | Classified [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		8,511	10,350
Commercial [Member] | Classified [Member] | New York and Other States [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		8,511	10,232
Commercial [Member] | Classified [Member] | Florida [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		0	118
Commercial [Member] | Commercial Real Estate [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		161,066	161,892
Commercial [Member] | Commercial Real Estate [Member] | New York and Other States [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	[1]	149,357	149,368
Commercial [Member] | Commercial Real Estate [Member] | Florida [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		11,709	12,524
Commercial [Member] | Commercial Real Estate [Member] | Pass [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		154,172	153,212
Commercial [Member] | Commercial Real Estate [Member] | Pass [Member] | New York and Other States [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		142,463	140,806
Commercial [Member] | Commercial Real Estate [Member] | Pass [Member] | Florida [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		11,709	12,406
Commercial [Member] | Commercial Real Estate [Member] | Classified [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		6,894	8,680
Commercial [Member] | Commercial Real Estate [Member] | Classified [Member] | New York and Other States [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		6,894	8,562
Commercial [Member] | Commercial Real Estate [Member] | Classified [Member] | Florida [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		0	118
Commercial [Member] | Other [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		24,063	24,315
Commercial [Member] | Other [Member] | New York and Other States [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	[1]	23,459	23,606
Commercial [Member] | Other [Member] | Florida [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		604	709
Commercial [Member] | Other [Member] | Pass [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		22,446	22,645
Commercial [Member] | Other [Member] | Pass [Member] | New York and Other States [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		21,842	21,936
Commercial [Member] | Other [Member] | Pass [Member] | Florida [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		604	709
Commercial [Member] | Other [Member] | Classified [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		1,617	1,670
Commercial [Member] | Other [Member] | Classified [Member] | New York and Other States [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		1,617	1,670
Commercial [Member] | Other [Member] | Classified [Member] | Florida [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		$ 0	$ 0

[1] Includes New York, New Jersey, Vermont and Massachusetts.