Loans and Allowance for Loan Losses, Allowance for Loan Losses (Details) - USD ($) $ in Thousands	3 Months Ended
	Mar. 31, 2016	Mar. 31, 2015
Allowance for loan losses by portfolio segment [Roll Forward]
Balance at beginning of period	$ 44,762	$ 46,327
Loans charged off [Abstract]
Total loan charge offs	1,334	1,316
Recoveries of loans previously charged off [Abstract]
Total recoveries	170	133
Net loans charged off	1,164	1,183
Provision (credit) for loan losses	800	800
Balance at end of period	44,398	45,944
Commercial [Member]
Allowance for loan losses by portfolio segment [Roll Forward]
Balance at beginning of period	4,491	4,071
Loans charged off [Abstract]
Total loan charge offs	264	50
Recoveries of loans previously charged off [Abstract]
Total recoveries	40	17
Net loans charged off	224	33
Provision (credit) for loan losses	652	(14)
Balance at end of period	4,919	4,024
Real Estate Mortgage - 1 to 4 Family [Member]
Allowance for loan losses by portfolio segment [Roll Forward]
Balance at beginning of period	39,753	42,088
Loans charged off [Abstract]
Total loan charge offs	973	1,223
Recoveries of loans previously charged off [Abstract]
Total recoveries	119	110
Net loans charged off	854	1,113
Provision (credit) for loan losses	118	554
Balance at end of period	39,017	41,529
Installment [Member]
Allowance for loan losses by portfolio segment [Roll Forward]
Balance at beginning of period	518	168
Loans charged off [Abstract]
Total loan charge offs	97	43
Recoveries of loans previously charged off [Abstract]
Total recoveries	11	6
Net loans charged off	86	37
Provision (credit) for loan losses	30	260
Balance at end of period	462	391
New York and Other States [Member]
Loans charged off [Abstract]
Total loan charge offs	1,234	1,207
Recoveries of loans previously charged off [Abstract]
Total recoveries	169	132
New York and Other States [Member] | Commercial [Member]
Loans charged off [Abstract]
Total loan charge offs	264	50
Recoveries of loans previously charged off [Abstract]
Total recoveries	40	16
New York and Other States [Member] | Real Estate Mortgage - 1 to 4 Family [Member]
Loans charged off [Abstract]
Total loan charge offs	889	1,114
Recoveries of loans previously charged off [Abstract]
Total recoveries	118	110
New York and Other States [Member] | Installment [Member]
Loans charged off [Abstract]
Total loan charge offs	81	43
Recoveries of loans previously charged off [Abstract]
Total recoveries	11	6
Florida [Member]
Loans charged off [Abstract]
Total loan charge offs	100	109
Recoveries of loans previously charged off [Abstract]
Total recoveries	1	1
Florida [Member] | Commercial [Member]
Loans charged off [Abstract]
Total loan charge offs	0	0
Recoveries of loans previously charged off [Abstract]
Total recoveries	0	1
Florida [Member] | Real Estate Mortgage - 1 to 4 Family [Member]
Loans charged off [Abstract]
Total loan charge offs	84	109
Recoveries of loans previously charged off [Abstract]
Total recoveries	1	0
Florida [Member] | Installment [Member]
Loans charged off [Abstract]
Total loan charge offs	16	0
Recoveries of loans previously charged off [Abstract]
Total recoveries	$ 0	$ 0