Loans and Allowance for Loan Losses, Allowance for Loan Losses (Details) (USD $) In Thousands, unless otherwise specified	3 Months Ended
	Mar. 31, 2014		Mar. 31, 2013
Allowance for loan losses by portfolio segment [Roll Forward]
Balance at beginning of period	$ 47,714		$ 47,927
Loans charged off [Abstract]
Total loan chargeoffs	2,316		2,411
Recoveries of loans previously charged off [Abstract]
Total recoveries	137		142
Net loans charged off	2,179		2,269
Provision for loan losses	1,500		2,000
Balance at end of period	47,035		47,658
Commercial [Member]
Allowance for loan losses by portfolio segment [Roll Forward]
Balance at beginning of period	4,019		3,771
Loans charged off [Abstract]
Total loan chargeoffs	873		350
Recoveries of loans previously charged off [Abstract]
Total recoveries	19		3
Net loans charged off	854		347
Provision for loan losses	675		540
Balance at end of period	3,840		3,964
Real Estate Mortgage - 1 to 4 Family [Member]
Allowance for loan losses by portfolio segment [Roll Forward]
Balance at beginning of period	43,597		44,069
Loans charged off [Abstract]
Total loan chargeoffs	1,392		2,042
Recoveries of loans previously charged off [Abstract]
Total recoveries	113		135
Net loans charged off	1,279		1,907
Provision for loan losses	773		1,433
Balance at end of period	43,091		43,595
Installment [Member]
Allowance for loan losses by portfolio segment [Roll Forward]
Balance at beginning of period	98		87
Loans charged off [Abstract]
Total loan chargeoffs	51		19
Recoveries of loans previously charged off [Abstract]
Total recoveries	5		4
Net loans charged off	46		15
Provision for loan losses	52		27
Balance at end of period	104		99
New York and Other States [Member]
Loans charged off [Abstract]
Total loan chargeoffs	1,234	[1]	1,906	[1]
Recoveries of loans previously charged off [Abstract]
Total recoveries	97	[1]	80	[1]
New York and Other States [Member] | Commercial [Member]
Loans charged off [Abstract]
Total loan chargeoffs	260	[1]	250	[1]
Recoveries of loans previously charged off [Abstract]
Total recoveries	18	[1]	2	[1]
New York and Other States [Member] | Real Estate Mortgage - 1 to 4 Family [Member]
Loans charged off [Abstract]
Total loan chargeoffs	925	[1]	1,637	[1]
Recoveries of loans previously charged off [Abstract]
Total recoveries	74	[1]	74	[1]
New York and Other States [Member] | Installment [Member]
Loans charged off [Abstract]
Total loan chargeoffs	49	[1]	19	[1]
Recoveries of loans previously charged off [Abstract]
Total recoveries	5	[1]	4	[1]
Florida [Member]
Loans charged off [Abstract]
Total loan chargeoffs	1,082		505
Recoveries of loans previously charged off [Abstract]
Total recoveries	40		62
Florida [Member] | Commercial [Member]
Loans charged off [Abstract]
Total loan chargeoffs	613		100
Recoveries of loans previously charged off [Abstract]
Total recoveries	1		1
Florida [Member] | Real Estate Mortgage - 1 to 4 Family [Member]
Loans charged off [Abstract]
Total loan chargeoffs	467		405
Recoveries of loans previously charged off [Abstract]
Total recoveries	39		61
Florida [Member] | Installment [Member]
Loans charged off [Abstract]
Total loan chargeoffs	2		0
Recoveries of loans previously charged off [Abstract]
Total recoveries	$ 0		$ 0

[1] Includes New York, New Jersey, Vermont and Massachusetts.