Loans and Allowance for Loan Losses, Allowance for Loan Losses (Details) (USD $) In Thousands, unless otherwise specified	3 Months Ended		9 Months Ended
	Sep. 30, 2013	Sep. 30, 2012	Sep. 30, 2013	Sep. 30, 2012
Allowance for loan losses by portfolio segment [Roll Forward]
Balance at beginning of period	$ 47,589	$ 48,018	$ 47,927	$ 48,717
Loans charged off [Abstract]
Total loan chargeoffs	2,490	3,732	7,136	10,819
Recoveries of loans previously charged off [Abstract]
Total recoveries	1,123	178	1,431	466
Net loans charged off	1,367	3,554	5,705	10,353
Provision for loan losses	1,500	2,900	5,500	9,000
Balance at end of period	47,722	47,364	47,722	47,364
Commercial [Member]
Allowance for loan losses by portfolio segment [Roll Forward]
Balance at beginning of period	3,719	3,792	3,771	4,140
Loans charged off [Abstract]
Total loan chargeoffs	585	770	984	2,265
Recoveries of loans previously charged off [Abstract]
Total recoveries	502	25	506	38
Net loans charged off	83	745	478	2,227
Provision for loan losses	228	813	571	1,947
Balance at end of period	3,864	3,860	3,864	3,860
Real Estate Mortgage - 1 to 4 Family [Member]
Allowance for loan losses by portfolio segment [Roll Forward]
Balance at beginning of period	43,766	44,147	44,069	44,479
Loans charged off [Abstract]
Total loan chargeoffs	1,872	2,944	6,084	8,506
Recoveries of loans previously charged off [Abstract]
Total recoveries	616	143	913	387
Net loans charged off	1,256	2,801	5,171	8,119
Provision for loan losses	1,240	2,080	4,852	7,066
Balance at end of period	43,750	43,426	43,750	43,426
Installment [Member]
Allowance for loan losses by portfolio segment [Roll Forward]
Balance at beginning of period	104	79	87	98
Loans charged off [Abstract]
Total loan chargeoffs	33	18	68	48
Recoveries of loans previously charged off [Abstract]
Total recoveries	5	10	12	41
Net loans charged off	28	8	56	7
Provision for loan losses	32	7	77	(13)
Balance at end of period	108	78	108	78
New York and Other States [Member]
Loans charged off [Abstract]
Total loan chargeoffs	2,253	2,248	6,232	6,530
Recoveries of loans previously charged off [Abstract]
Total recoveries	428	75	634	344
New York and Other States [Member] | Commercial [Member]
Loans charged off [Abstract]
Total loan chargeoffs	585	34	884	1,073
Recoveries of loans previously charged off [Abstract]
Total recoveries	0	25	2	30
New York and Other States [Member] | Real Estate Mortgage - 1 to 4 Family [Member]
Loans charged off [Abstract]
Total loan chargeoffs	1,638	2,197	5,283	5,410
Recoveries of loans previously charged off [Abstract]
Total recoveries	423	40	620	273
New York and Other States [Member] | Installment [Member]
Loans charged off [Abstract]
Total loan chargeoffs	30	17	65	47
Recoveries of loans previously charged off [Abstract]
Total recoveries	5	10	12	41
Florida [Member]
Loans charged off [Abstract]
Total loan chargeoffs	237	1,484	904	4,289
Recoveries of loans previously charged off [Abstract]
Total recoveries	695	103	797	122
Florida [Member] | Commercial [Member]
Loans charged off [Abstract]
Total loan chargeoffs	0	736	100	1,192
Recoveries of loans previously charged off [Abstract]
Total recoveries	502	0	504	8
Florida [Member] | Real Estate Mortgage - 1 to 4 Family [Member]
Loans charged off [Abstract]
Total loan chargeoffs	234	747	801	3,096
Recoveries of loans previously charged off [Abstract]
Total recoveries	193	103	293	114
Florida [Member] | Installment [Member]
Loans charged off [Abstract]
Total loan chargeoffs	3	1	3	1
Recoveries of loans previously charged off [Abstract]
Total recoveries	$ 0	$ 0	$ 0	$ 0