Loans and Allowance for Loan Losses, Non-accrual Loans, By Loan Class (Details) (USD $) In Thousands, unless otherwise specified	Jun. 30, 2014		Dec. 31, 2013
Non accrual loans by loan class [Abstract]
Total non-accrual loans	$ 40,748		$ 43,227
Restructured real estate mortgages - 1 to 4 family	159		166
Total nonperforming loans	40,907		43,393
Commercial Real Estate [Member]
Non accrual loans by loan class [Abstract]
Total non-accrual loans	5,527		6,620
Other [Member]
Non accrual loans by loan class [Abstract]
Total non-accrual loans	122		332
First Mortgages [Member]
Non accrual loans by loan class [Abstract]
Total non-accrual loans	30,026		31,494
Home Equity Loans [Member]
Non accrual loans by loan class [Abstract]
Total non-accrual loans	517		691
Home Equity Lines of Credit [Member]
Non accrual loans by loan class [Abstract]
Total non-accrual loans	4,468		3,997
Installment [Member]
Non accrual loans by loan class [Abstract]
Total non-accrual loans	88		93
New York and Other States [Member]
Non accrual loans by loan class [Abstract]
Total non-accrual loans	36,652	[1]	38,090	[1]
Restructured real estate mortgages - 1 to 4 family	159	[1]	166	[1]
Total nonperforming loans	36,811	[1]	38,256	[1]
New York and Other States [Member] | Commercial Real Estate [Member]
Non accrual loans by loan class [Abstract]
Total non-accrual loans	5,010	[1]	6,620	[1]
New York and Other States [Member] | Other [Member]
Non accrual loans by loan class [Abstract]
Total non-accrual loans	122	[1]	332	[1]
New York and Other States [Member] | First Mortgages [Member]
Non accrual loans by loan class [Abstract]
Total non-accrual loans	26,886	[1]	26,713	[1]
New York and Other States [Member] | Home Equity Loans [Member]
Non accrual loans by loan class [Abstract]
Total non-accrual loans	517	[1]	691	[1]
New York and Other States [Member] | Home Equity Lines of Credit [Member]
Non accrual loans by loan class [Abstract]
Total non-accrual loans	4,030	[1]	3,641	[1]
New York and Other States [Member] | Installment [Member]
Non accrual loans by loan class [Abstract]
Total non-accrual loans	87	[1]	93	[1]
Florida [Member]
Non accrual loans by loan class [Abstract]
Total non-accrual loans	4,096		5,137
Restructured real estate mortgages - 1 to 4 family	0		0
Total nonperforming loans	4,096		5,137
Florida [Member] | Commercial Real Estate [Member]
Non accrual loans by loan class [Abstract]
Total non-accrual loans	517		0
Florida [Member] | Other [Member]
Non accrual loans by loan class [Abstract]
Total non-accrual loans	0		0
Florida [Member] | First Mortgages [Member]
Non accrual loans by loan class [Abstract]
Total non-accrual loans	3,140		4,781
Florida [Member] | Home Equity Loans [Member]
Non accrual loans by loan class [Abstract]
Total non-accrual loans	0		0
Florida [Member] | Home Equity Lines of Credit [Member]
Non accrual loans by loan class [Abstract]
Total non-accrual loans	438		356
Florida [Member] | Installment [Member]
Non accrual loans by loan class [Abstract]
Total non-accrual loans	$ 1		$ 0

[1] Includes New York, New Jersey, Vermont and Massachusetts.