Loans and Allowance for Loan Losses, Risk Category of Loans (Details) - USD ($) $ in Thousands		Dec. 31, 2017	Dec. 31, 2016
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		$ 3,636,407	$ 3,430,586
Impaired loans included in classified loans		1,500	1,800
New York and Other States [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	[1]	2,795,993	2,692,335
Florida [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		840,414	738,251
Commercial [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		186,207	191,194
Commercial [Member] | New York and Other States [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		172,974	178,905
Commercial [Member] | Florida [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		13,233	12,289
Commercial [Member] | Pass [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		175,857	174,407
Commercial [Member] | Pass [Member] | New York and Other States [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		162,742	162,118
Commercial [Member] | Pass [Member] | Florida [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		13,115	12,289
Commercial [Member] | Classified [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		10,350	16,787
Commercial [Member] | Classified [Member] | New York and Other States [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		10,232	16,787
Commercial [Member] | Classified [Member] | Florida [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		118	0
Commercial [Member] | Commercial Real Estate [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		161,892	163,609
Commercial [Member] | Commercial Real Estate [Member] | New York and Other States [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	[1]	149,368	151,366
Commercial [Member] | Commercial Real Estate [Member] | Florida [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		12,524	12,243
Commercial [Member] | Commercial Real Estate [Member] | Pass [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		153,212	148,919
Commercial [Member] | Commercial Real Estate [Member] | Pass [Member] | New York and Other States [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		140,806	136,676
Commercial [Member] | Commercial Real Estate [Member] | Pass [Member] | Florida [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		12,406	12,243
Commercial [Member] | Commercial Real Estate [Member] | Classified [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		8,680	14,690
Commercial [Member] | Commercial Real Estate [Member] | Classified [Member] | New York and Other States [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		8,562	14,690
Commercial [Member] | Commercial Real Estate [Member] | Classified [Member] | Florida [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		118	0
Commercial [Member] | Other [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		24,315	27,585
Commercial [Member] | Other [Member] | New York and Other States [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	[1]	23,606	27,539
Commercial [Member] | Other [Member] | Florida [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		709	46
Commercial [Member] | Other [Member] | Pass [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		22,645	25,488
Commercial [Member] | Other [Member] | Pass [Member] | New York and Other States [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		21,936	25,442
Commercial [Member] | Other [Member] | Pass [Member] | Florida [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		709	46
Commercial [Member] | Other [Member] | Classified [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		1,670	2,097
Commercial [Member] | Other [Member] | Classified [Member] | New York and Other States [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		1,670	2,097
Commercial [Member] | Other [Member] | Classified [Member] | Florida [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		$ 0	$ 0

[1] Includes New York, New Jersey, Vermont, and Massachusetts.