Loans and Allowance for Loan Losses (Details) - USD ($) $ in Thousands		Mar. 31, 2021		Dec. 31, 2020	Mar. 31, 2020	Dec. 31, 2019
Loans and Leases Receivable [Abstract]
Total loans, net		$ 4,269,172		$ 4,244,470
Less: Allowance for loan losses		49,991		49,595	$ 46,155	$ 44,317
Net loans		4,219,181		4,194,875
Other estate owned		420		541
New York and Other States [Member]
Loans and Leases Receivable [Abstract]
Total loans, net	[1]	3,062,023		3,061,352
Florida [Member]
Loans and Leases Receivable [Abstract]
Total loans, net		1,207,149		1,183,118
Real Estate Construction Loans [Member]
Loans and Leases Receivable [Abstract]
Total loans, net		28,500		24,700
Residential Borrowers [Member]
Loans and Leases Receivable [Abstract]
Total loans, net		14,400		10,500
Commercial Borrowers Residential Purpose [Member]
Loans and Leases Receivable [Abstract]
Total loans, net		14,100		14,200
Residential Real Estate [Member]
Loans and Leases Receivable [Abstract]
Non accrual residential mortgage loans in the process of foreclosure		12,300		11,600
Commercial [Member]
Loans and Leases Receivable [Abstract]
Total loans, net		217,021		212,492
Less: Allowance for loan losses		4,052		4,140	3,960	3,999
Commercial [Member] | New York and Other States [Member]
Loans and Leases Receivable [Abstract]
Total loans, net		198,667	[2]	193,707
Commercial [Member] | Florida [Member]
Loans and Leases Receivable [Abstract]
Total loans, net		18,354		18,785
Commercial [Member] | Commercial Real Estate [Member]
Loans and Leases Receivable [Abstract]
Total loans, net		164,983		167,441
Commercial [Member] | Commercial Real Estate [Member] | New York and Other States [Member]
Loans and Leases Receivable [Abstract]
Total loans, net	[1]	146,994	[2]	148,775
Commercial [Member] | Commercial Real Estate [Member] | Florida [Member]
Loans and Leases Receivable [Abstract]
Total loans, net		17,989		18,666
Commercial [Member] | Other [Member]
Loans and Leases Receivable [Abstract]
Total loans, net		52,038		45,051
Commercial [Member] | Other [Member] | New York and Other States [Member]
Loans and Leases Receivable [Abstract]
Total loans, net	[1]	51,673	[2]	44,932
Commercial [Member] | Other [Member] | Florida [Member]
Loans and Leases Receivable [Abstract]
Total loans, net		365		119
Commercial [Member] | PPP Loans [Member]
Loans and Leases Receivable [Abstract]
Total loans, net		36,600		28,900
Real Estate Mortgage - 1 to 4 Family [Member]
Loans and Leases Receivable [Abstract]
Total loans, net		4,043,481		4,022,361
Less: Allowance for loan losses		45,507		44,950	41,643	39,748
Real Estate Mortgage - 1 to 4 Family [Member] | First Mortgages [Member]
Loans and Leases Receivable [Abstract]
Total loans, net		3,738,047		3,705,696
Real Estate Mortgage - 1 to 4 Family [Member] | First Mortgages [Member] | New York and Other States [Member]
Loans and Leases Receivable [Abstract]
Total loans, net	[1]	2,615,578		2,606,781
Real Estate Mortgage - 1 to 4 Family [Member] | First Mortgages [Member] | Florida [Member]
Loans and Leases Receivable [Abstract]
Total loans, net		1,122,469		1,098,915
Real Estate Mortgage - 1 to 4 Family [Member] | Home Equity Loans [Member]
Loans and Leases Receivable [Abstract]
Total loans, net		69,790		74,471
Real Estate Mortgage - 1 to 4 Family [Member] | Home Equity Loans [Member] | New York and Other States [Member]
Loans and Leases Receivable [Abstract]
Total loans, net	[1]	55,469		59,400
Real Estate Mortgage - 1 to 4 Family [Member] | Home Equity Loans [Member] | Florida [Member]
Loans and Leases Receivable [Abstract]
Total loans, net		14,321		15,071
Real Estate Mortgage - 1 to 4 Family [Member] | Home Equity Lines of Credit [Member]
Loans and Leases Receivable [Abstract]
Total loans, net		235,644		242,194
Real Estate Mortgage - 1 to 4 Family [Member] | Home Equity Lines of Credit [Member] | New York and Other States [Member]
Loans and Leases Receivable [Abstract]
Total loans, net	[1]	185,237		193,654
Real Estate Mortgage - 1 to 4 Family [Member] | Home Equity Lines of Credit [Member] | Florida [Member]
Loans and Leases Receivable [Abstract]
Total loans, net		50,407		48,540
Installment [Member]
Loans and Leases Receivable [Abstract]
Total loans, net		8,670		9,617
Less: Allowance for loan losses		432		505	$ 552	$ 570
Installment [Member] | New York and Other States [Member]
Loans and Leases Receivable [Abstract]
Total loans, net	[1]	7,072		7,810
Installment [Member] | Florida [Member]
Loans and Leases Receivable [Abstract]
Total loans, net		$ 1,598		$ 1,807

[1] Includes New York, New Jersey, Vermont and Massachusetts.

[2] Includes New York, New Jersey and Massachusetts.