Loans and Allowance for Loan Losses, Impaired Receivables (Details) (USD $) In Thousands, unless otherwise specified	12 Months Ended
	Dec. 31, 2013	Dec. 31, 2012	Dec. 31, 2011
Impaired loans by loans class [Abstract]
Recorded Investment	$ 29,340	$ 26,073	$ 13,711
Unpaid Principal Balance	32,595	30,566	19,632
Related Allowance	0	0	0
Average Recorded Investment	26,683	19,358	13,985
Commercial [Member] | Commercial Real Estate [Member]
Impaired loans by loans class [Abstract]
Recorded Investment	7,750	9,209	9,968
Unpaid Principal Balance	9,169	11,059	14,726
Related Allowance	0	0	0
Average Recorded Investment	7,414	11,486	11,972
Commercial [Member] | Other [Member]
Impaired loans by loans class [Abstract]
Recorded Investment	332	124	13
Unpaid Principal Balance	332	124	32
Related Allowance	0	0	0
Average Recorded Investment	165	107	56
Real Estate Mortgage - 1 to 4 Family [Member] | First Mortgages [Member]
Impaired loans by loans class [Abstract]
Recorded Investment	17,887	14,436	3,579
Unpaid Principal Balance	19,275	16,808	4,600
Related Allowance	0	0	0
Average Recorded Investment	16,317	7,023	1,888
Real Estate Mortgage - 1 to 4 Family [Member] | Home Equity Loans [Member]
Impaired loans by loans class [Abstract]
Recorded Investment	561	623	151
Unpaid Principal Balance	614	664	199
Related Allowance	0	0	0
Average Recorded Investment	636	260	69
Real Estate Mortgage - 1 to 4 Family [Member] | Home Equity Lines of Credit [Member]
Impaired loans by loans class [Abstract]
Recorded Investment	2,810	1,681	0
Unpaid Principal Balance	3,205	1,911	75
Related Allowance	0	0	0
Average Recorded Investment	2,151	482	0
New York and Other States [Member]
Impaired loans by loans class [Abstract]
Recorded Investment	26,298	21,855	8,006
Unpaid Principal Balance	29,163	23,835	9,289
Related Allowance	0	0	0
Average Recorded Investment	23,571	12,515	6,987
New York and Other States [Member] | Commercial [Member] | Commercial Real Estate [Member]
Impaired loans by loans class [Abstract]
Recorded Investment	6,620	6,511	4,968
Unpaid Principal Balance	8,039	7,169	5,684
Related Allowance	0	0	0
Average Recorded Investment	6,013	5,615	5,198
New York and Other States [Member] | Commercial [Member] | Other [Member]
Impaired loans by loans class [Abstract]
Recorded Investment	332	124	13
Unpaid Principal Balance	332	124	32
Related Allowance	0	0	0
Average Recorded Investment	165	107	56
New York and Other States [Member] | Real Estate Mortgage - 1 to 4 Family [Member] | First Mortgages [Member]
Impaired loans by loans class [Abstract]
Recorded Investment	16,257	12,964	2,874
Unpaid Principal Balance	17,353	14,143	3,299
Related Allowance	0	0	0
Average Recorded Investment	14,706	6,075	1,664
New York and Other States [Member] | Real Estate Mortgage - 1 to 4 Family [Member] | Home Equity Loans [Member]
Impaired loans by loans class [Abstract]
Recorded Investment	561	623	151
Unpaid Principal Balance	614	664	199
Related Allowance	0	0	0
Average Recorded Investment	636	260	69
New York and Other States [Member] | Real Estate Mortgage - 1 to 4 Family [Member] | Home Equity Lines of Credit [Member]
Impaired loans by loans class [Abstract]
Recorded Investment	2,528	1,633	0
Unpaid Principal Balance	2,825	1,735	75
Related Allowance	0	0	0
Average Recorded Investment	2,051	458	0
Florida [Member]
Impaired loans by loans class [Abstract]
Recorded Investment	3,042	4,218	5,705
Unpaid Principal Balance	3,432	6,731	10,343
Related Allowance	0	0	0
Average Recorded Investment	3,112	6,843	6,998
Florida [Member] | Commercial [Member] | Commercial Real Estate [Member]
Impaired loans by loans class [Abstract]
Recorded Investment	1,130	2,698	5,000
Unpaid Principal Balance	1,130	3,890	9,042
Related Allowance	0	0	0
Average Recorded Investment	1,401	5,871	6,774
Florida [Member] | Commercial [Member] | Other [Member]
Impaired loans by loans class [Abstract]
Recorded Investment	0	0	0
Unpaid Principal Balance	0	0	0
Related Allowance	0	0	0
Average Recorded Investment	0	0	0
Florida [Member] | Real Estate Mortgage - 1 to 4 Family [Member] | First Mortgages [Member]
Impaired loans by loans class [Abstract]
Recorded Investment	1,630	1,472	705
Unpaid Principal Balance	1,922	2,665	1,301
Related Allowance	0	0	0
Average Recorded Investment	1,611	948	224
Florida [Member] | Real Estate Mortgage - 1 to 4 Family [Member] | Home Equity Loans [Member]
Impaired loans by loans class [Abstract]
Recorded Investment	0
Unpaid Principal Balance	0
Related Allowance	0
Average Recorded Investment	0
Florida [Member] | Real Estate Mortgage - 1 to 4 Family [Member] | Home Equity Lines of Credit [Member]
Impaired loans by loans class [Abstract]
Recorded Investment	282	48
Unpaid Principal Balance	380	176
Related Allowance	0	0
Average Recorded Investment	$ 100	$ 24