Loans and Allowance for Loan Losses, Risk Category of Loans (Details) $ in Thousands		Dec. 31, 2020 USD ($) Loan	Dec. 31, 2019 USD ($)
Loans and Leases Receivable, Gross, Carrying Amount [Abstract]
Total loans, net		$ 4,244,470	$ 4,062,196
Impaired loans included in classified loans		796	816
New York and Other States [Member]
Loans and Leases Receivable, Gross, Carrying Amount [Abstract]
Total loans, net	[1]	3,061,352	3,022,936
Florida [Member]
Loans and Leases Receivable, Gross, Carrying Amount [Abstract]
Total loans, net		1,183,118	1,039,260
PPP Loans [Member]
Loans and Leases Receivable, Gross, Carrying Amount [Abstract]
Total loans, net		$ 29,000
Number of loans granted | Loan		663
Amount of loans granted		$ 46,000
Number of loans remain outstanding | Loan		514
Commercial [Member]
Loans and Leases Receivable, Gross, Carrying Amount [Abstract]
Total loans, net		$ 212,492	199,499
Commercial [Member] | New York and Other States [Member]
Loans and Leases Receivable, Gross, Carrying Amount [Abstract]
Total loans, net	[2]	193,707	181,512
Commercial [Member] | Florida [Member]
Loans and Leases Receivable, Gross, Carrying Amount [Abstract]
Total loans, net		18,785	17,987
Commercial [Member] | Pass [Member]
Loans and Leases Receivable, Gross, Carrying Amount [Abstract]
Total loans, net		208,474	193,651
Commercial [Member] | Pass [Member] | New York and Other States [Member]
Loans and Leases Receivable, Gross, Carrying Amount [Abstract]
Total loans, net	[2]	190,263	175,664
Commercial [Member] | Pass [Member] | Florida [Member]
Loans and Leases Receivable, Gross, Carrying Amount [Abstract]
Total loans, net		18,211	17,987
Commercial [Member] | Classified [Member]
Loans and Leases Receivable, Gross, Carrying Amount [Abstract]
Total loans, net		4,018	5,848
Commercial [Member] | Classified [Member] | New York and Other States [Member]
Loans and Leases Receivable, Gross, Carrying Amount [Abstract]
Total loans, net	[2]	3,444	5,848
Commercial [Member] | Classified [Member] | Florida [Member]
Loans and Leases Receivable, Gross, Carrying Amount [Abstract]
Total loans, net		574	0
Commercial [Member] | Commercial Real Estate [Member]
Loans and Leases Receivable, Gross, Carrying Amount [Abstract]
Total loans, net		167,441	179,938
Commercial [Member] | Commercial Real Estate [Member] | New York and Other States [Member]
Loans and Leases Receivable, Gross, Carrying Amount [Abstract]
Total loans, net	[1],[2]	148,775	162,186
Commercial [Member] | Commercial Real Estate [Member] | Florida [Member]
Loans and Leases Receivable, Gross, Carrying Amount [Abstract]
Total loans, net		18,666	17,752
Commercial [Member] | Commercial Real Estate [Member] | Pass [Member]
Loans and Leases Receivable, Gross, Carrying Amount [Abstract]
Total loans, net		163,833	175,032
Commercial [Member] | Commercial Real Estate [Member] | Pass [Member] | New York and Other States [Member]
Loans and Leases Receivable, Gross, Carrying Amount [Abstract]
Total loans, net	[2]	145,741	157,280
Commercial [Member] | Commercial Real Estate [Member] | Pass [Member] | Florida [Member]
Loans and Leases Receivable, Gross, Carrying Amount [Abstract]
Total loans, net		18,092	17,752
Commercial [Member] | Commercial Real Estate [Member] | Classified [Member]
Loans and Leases Receivable, Gross, Carrying Amount [Abstract]
Total loans, net		3,608	4,906
Commercial [Member] | Commercial Real Estate [Member] | Classified [Member] | New York and Other States [Member]
Loans and Leases Receivable, Gross, Carrying Amount [Abstract]
Total loans, net	[2]	3,034	4,906
Commercial [Member] | Commercial Real Estate [Member] | Classified [Member] | Florida [Member]
Loans and Leases Receivable, Gross, Carrying Amount [Abstract]
Total loans, net		574	0
Commercial [Member] | Other [Member]
Loans and Leases Receivable, Gross, Carrying Amount [Abstract]
Total loans, net		45,051	19,561
Commercial [Member] | Other [Member] | New York and Other States [Member]
Loans and Leases Receivable, Gross, Carrying Amount [Abstract]
Total loans, net	[1],[2]	44,932	19,326
Commercial [Member] | Other [Member] | Florida [Member]
Loans and Leases Receivable, Gross, Carrying Amount [Abstract]
Total loans, net		119	235
Commercial [Member] | Other [Member] | Pass [Member]
Loans and Leases Receivable, Gross, Carrying Amount [Abstract]
Total loans, net		44,641	18,619
Commercial [Member] | Other [Member] | Pass [Member] | New York and Other States [Member]
Loans and Leases Receivable, Gross, Carrying Amount [Abstract]
Total loans, net	[2]	44,522	18,384
Commercial [Member] | Other [Member] | Pass [Member] | Florida [Member]
Loans and Leases Receivable, Gross, Carrying Amount [Abstract]
Total loans, net		119	235
Commercial [Member] | Other [Member] | Classified [Member]
Loans and Leases Receivable, Gross, Carrying Amount [Abstract]
Total loans, net		410	942
Commercial [Member] | Other [Member] | Classified [Member] | New York and Other States [Member]
Loans and Leases Receivable, Gross, Carrying Amount [Abstract]
Total loans, net	[2]	410	942
Commercial [Member] | Other [Member] | Classified [Member] | Florida [Member]
Loans and Leases Receivable, Gross, Carrying Amount [Abstract]
Total loans, net		0	$ 0
Commercial [Member] | PPP Loans [Member]
Loans and Leases Receivable, Gross, Carrying Amount [Abstract]
Total loans, net		$ 29,000

[1] Includes New York, New Jersey, Vermont and Massachusetts.

[2] Includes New York, New Jersey and Massachusetts.