Loans and Allowance for Loan Losses, Allowance for Loan Losses (Details) - USD ($) $ in Thousands	3 Months Ended		9 Months Ended
	Sep. 30, 2015	Sep. 30, 2014	Sep. 30, 2015	Sep. 30, 2014
Allowance for loan losses by portfolio segment [Roll Forward]
Balance at beginning of period	$ 45,571	$ 46,935	$ 46,327	$ 47,714
Loans charged off [Abstract]
Total loan charge offs	1,392	1,656	4,026	5,794
Recoveries of loans previously charged off [Abstract]
Total recoveries	170	133	448	492
Net loans charged off	1,222	1,523	3,578	5,302
Provision for loan losses	800	1,100	2,400	4,100
Balance at end of period	45,149	46,512	45,149	46,512
Commercial [Member]
Allowance for loan losses by portfolio segment [Roll Forward]
Balance at beginning of period	4,022	4,073	4,071	4,019
Loans charged off [Abstract]
Total loan charge offs	3	124	103	1,010
Recoveries of loans previously charged off [Abstract]
Total recoveries	3	1	21	22
Net loans charged off	0	123	82	988
Provision for loan losses	34	95	67	1,014
Balance at end of period	4,056	4,045	4,056	4,045
Real Estate Mortgage - 1 to 4 Family [Member]
Allowance for loan losses by portfolio segment [Roll Forward]
Balance at beginning of period	41,087	42,752	42,088	43,597
Loans charged off [Abstract]
Total loan charge offs	1,335	1,465	3,793	4,624
Recoveries of loans previously charged off [Abstract]
Total recoveries	143	118	388	443
Net loans charged off	1,192	1,347	3,405	4,181
Provision for loan losses	752	935	1,964	2,924
Balance at end of period	40,647	42,340	40,647	42,340
Installment [Member]
Allowance for loan losses by portfolio segment [Roll Forward]
Balance at beginning of period	462	110	168	98
Loans charged off [Abstract]
Total loan charge offs	54	67	130	160
Recoveries of loans previously charged off [Abstract]
Total recoveries	24	14	39	27
Net loans charged off	30	53	91	133
Provision for loan losses	14	70	369	162
Balance at end of period	446	127	446	127
New York and Other States [Member]
Loans charged off [Abstract]
Total loan charge offs	1,353	1,378	3,709	4,349
Recoveries of loans previously charged off [Abstract]
Total recoveries	165	92	439	393
New York and Other States [Member] | Commercial [Member]
Loans charged off [Abstract]
Total loan charge offs	3	124	103	397
Recoveries of loans previously charged off [Abstract]
Total recoveries	0	0	16	18
New York and Other States [Member] | Real Estate Mortgage - 1 to 4 Family [Member]
Loans charged off [Abstract]
Total loan charge offs	1,300	1,187	3,480	3,804
Recoveries of loans previously charged off [Abstract]
Total recoveries	141	82	384	352
New York and Other States [Member] | Installment [Member]
Loans charged off [Abstract]
Total loan charge offs	50	67	126	148
Recoveries of loans previously charged off [Abstract]
Total recoveries	24	10	39	23
Florida [Member]
Loans charged off [Abstract]
Total loan charge offs	39	278	317	1,445
Recoveries of loans previously charged off [Abstract]
Total recoveries	5	41	9	99
Florida [Member] | Commercial [Member]
Loans charged off [Abstract]
Total loan charge offs	0	0	0	613
Recoveries of loans previously charged off [Abstract]
Total recoveries	3	1	5	4
Florida [Member] | Real Estate Mortgage - 1 to 4 Family [Member]
Loans charged off [Abstract]
Total loan charge offs	35	278	313	820
Recoveries of loans previously charged off [Abstract]
Total recoveries	2	36	4	91
Florida [Member] | Installment [Member]
Loans charged off [Abstract]
Total loan charge offs	4	0	4	12
Recoveries of loans previously charged off [Abstract]
Total recoveries	$ 0	$ 4	$ 0	$ 4