Loans and Allowance for Loan Losses, Allowance for Loan Losses (Details) (USD $) In Thousands, unless otherwise specified	3 Months Ended		9 Months Ended
	Sep. 30, 2014	Sep. 30, 2013	Sep. 30, 2014	Sep. 30, 2013
Allowance for loan losses by portfolio segment [Roll Forward]
Balance at beginning of period	$ 46,935	$ 47,589	$ 47,714	$ 47,927
Loans charged off [Abstract]
Total loan charge offs	1,656	2,490	5,793	7,136
Recoveries of loans previously charged off [Abstract]
Total recoveries	133	1,123	491	1,431
Net loans charged off	1,523	1,367	5,302	5,705
Provision for loan losses	1,100	1,500	4,100	5,500
Balance at end of period	46,512	47,722	46,512	47,722
Commercial [Member]
Allowance for loan losses by portfolio segment [Roll Forward]
Balance at beginning of period	4,073	3,719	4,019	3,771
Loans charged off [Abstract]
Total loan charge offs	124	585	1,010	984
Recoveries of loans previously charged off [Abstract]
Total recoveries	1	502	22	506
Net loans charged off	123	83	988	478
Provision for loan losses	95	228	1,014	571
Balance at end of period	4,045	3,864	4,045	3,864
Real Estate Mortgage - 1 to 4 Family [Member]
Allowance for loan losses by portfolio segment [Roll Forward]
Balance at beginning of period	42,752	43,766	43,597	44,069
Loans charged off [Abstract]
Total loan charge offs	1,465	1,872	4,624	6,084
Recoveries of loans previously charged off [Abstract]
Total recoveries	118	616	443	913
Net loans charged off	1,347	1,256	4,181	5,171
Provision for loan losses	935	1,240	2,924	4,852
Balance at end of period	42,340	43,750	42,340	43,750
Installment [Member]
Allowance for loan losses by portfolio segment [Roll Forward]
Balance at beginning of period	110	104	98	87
Loans charged off [Abstract]
Total loan charge offs	67	33	160	68
Recoveries of loans previously charged off [Abstract]
Total recoveries	14	5	27	12
Net loans charged off	53	28	133	56
Provision for loan losses	70	32	162	77
Balance at end of period	127	108	127	108
New York and Other States [Member]
Loans charged off [Abstract]
Total loan charge offs	1,378	2,253	4,349	6,232
Recoveries of loans previously charged off [Abstract]
Total recoveries	92	428	393	634
New York and Other States [Member] | Commercial [Member]
Loans charged off [Abstract]
Total loan charge offs	124	585	397	884
Recoveries of loans previously charged off [Abstract]
Total recoveries	0	0	18	2
New York and Other States [Member] | Real Estate Mortgage - 1 to 4 Family [Member]
Loans charged off [Abstract]
Total loan charge offs	1,187	1,638	3,804	5,283
Recoveries of loans previously charged off [Abstract]
Total recoveries	82	423	352	620
New York and Other States [Member] | Installment [Member]
Loans charged off [Abstract]
Total loan charge offs	67	30	148	65
Recoveries of loans previously charged off [Abstract]
Total recoveries	10	5	23	12
Florida [Member]
Loans charged off [Abstract]
Total loan charge offs	278	237	1,445	904
Recoveries of loans previously charged off [Abstract]
Total recoveries	41	695	99	797
Florida [Member] | Commercial [Member]
Loans charged off [Abstract]
Total loan charge offs	0	0	613	100
Recoveries of loans previously charged off [Abstract]
Total recoveries	1	502	4	504
Florida [Member] | Real Estate Mortgage - 1 to 4 Family [Member]
Loans charged off [Abstract]
Total loan charge offs	278	234	820	801
Recoveries of loans previously charged off [Abstract]
Total recoveries	36	193	91	293
Florida [Member] | Installment [Member]
Loans charged off [Abstract]
Total loan charge offs	0	3	12	3
Recoveries of loans previously charged off [Abstract]
Total recoveries	$ 4	$ 0	$ 4	$ 0