Loans and Allowance for Loan Losses, Allowance for Loan Losses (Details) (USD $) In Thousands, unless otherwise specified	3 Months Ended				6 Months Ended
	Jun. 30, 2014		Jun. 30, 2013		Jun. 30, 2014	Jun. 30, 2013
Allowance for loan losses by portfolio segment [Roll Forward]
Balance at beginning of period	$ 47,035		$ 47,658		$ 47,714	$ 47,927
Loans charged off [Abstract]
Total loan charge offs	1,821		2,235		4,138	4,646
Recoveries of loans previously charged off [Abstract]
Total recoveries	221		166		359	308
Net loans charged off	1,600		2,069		3,779	4,338
Provision (credit) for loan losses	1,500		2,000		3,000	4,000
Balance at end of period	46,935		47,589		46,935	47,589
Commercial [Member]
Allowance for loan losses by portfolio segment [Roll Forward]
Balance at beginning of period	3,840		3,964		4,019	3,771
Loans charged off [Abstract]
Total loan charge offs	13		49		886	399
Recoveries of loans previously charged off [Abstract]
Total recoveries	2		1		21	4
Net loans charged off	11		48		865	395
Provision (credit) for loan losses	244		(197)		919	343
Balance at end of period	4,073		3,719		4,073	3,719
Real Estate Mortgage - 1 to 4 Family [Member]
Allowance for loan losses by portfolio segment [Roll Forward]
Balance at beginning of period	43,091		43,595		43,597	44,069
Loans charged off [Abstract]
Total loan charge offs	1,766		2,170		3,159	4,212
Recoveries of loans previously charged off [Abstract]
Total recoveries	211		162		325	297
Net loans charged off	1,555		2,008		2,834	3,915
Provision (credit) for loan losses	1,216		2,179		1,989	3,612
Balance at end of period	42,752		43,766		42,752	43,766
Installment [Member]
Allowance for loan losses by portfolio segment [Roll Forward]
Balance at beginning of period	104		99		98	87
Loans charged off [Abstract]
Total loan charge offs	42		16		93	35
Recoveries of loans previously charged off [Abstract]
Total recoveries	8		3		13	7
Net loans charged off	34		13		80	28
Provision (credit) for loan losses	40		18		92	45
Balance at end of period	110		104		110	104
New York and Other States [Member]
Loans charged off [Abstract]
Total loan charge offs	1,736	[1]	2,073	[1]	2,971	3,979
Recoveries of loans previously charged off [Abstract]
Total recoveries	203	[1]	126	[1]	301	206
New York and Other States [Member] | Commercial [Member]
Loans charged off [Abstract]
Total loan charge offs	13	[1]	49	[1]	273	299
Recoveries of loans previously charged off [Abstract]
Total recoveries	0	[1]	0	[1]	18	2
New York and Other States [Member] | Real Estate Mortgage - 1 to 4 Family [Member]
Loans charged off [Abstract]
Total loan charge offs	1,691	[1]	2,008	[1]	2,617	3,645
Recoveries of loans previously charged off [Abstract]
Total recoveries	195	[1]	123	[1]	270	197
New York and Other States [Member] | Installment [Member]
Loans charged off [Abstract]
Total loan charge offs	32	[1]	16	[1]	81	35
Recoveries of loans previously charged off [Abstract]
Total recoveries	8	[1]	3	[1]	13	7
Florida [Member]
Loans charged off [Abstract]
Total loan charge offs	85		162		1,167	667
Recoveries of loans previously charged off [Abstract]
Total recoveries	18		40		58	102
Florida [Member] | Commercial [Member]
Loans charged off [Abstract]
Total loan charge offs	0		0		613	100
Recoveries of loans previously charged off [Abstract]
Total recoveries	2		1		3	2
Florida [Member] | Real Estate Mortgage - 1 to 4 Family [Member]
Loans charged off [Abstract]
Total loan charge offs	75		162		542	567
Recoveries of loans previously charged off [Abstract]
Total recoveries	16		39		55	100
Florida [Member] | Installment [Member]
Loans charged off [Abstract]
Total loan charge offs	10		0		12	0
Recoveries of loans previously charged off [Abstract]
Total recoveries	$ 0		$ 0		$ 0	$ 0

[1] Includes New York, New Jersey, Vermont and Massachusetts.