Loans and Allowance for Loan Losses, Allowance for Loan Losses (Details) - USD ($) $ in Thousands	3 Months Ended		6 Months Ended
	Jun. 30, 2016	Jun. 30, 2015	Jun. 30, 2016	Jun. 30, 2015
Allowance for loan losses by portfolio segment [Roll Forward]
Balance at beginning of period	$ 44,398	$ 45,944	$ 44,762	$ 46,327
Loans charged off [Abstract]
Total loan charge offs	1,268	1,318	2,602	2,634
Recoveries of loans previously charged off [Abstract]
Total recoveries	134	145	304	278
Net loans charged off	1,134	1,173	2,298	2,356
Provision (credit) for loan losses	800	800	1,600	1,600
Balance at end of period	44,064	45,571	44,064	45,571
Commercial [Member]
Allowance for loan losses by portfolio segment [Roll Forward]
Balance at beginning of period	4,919	4,024	4,491	4,071
Loans charged off [Abstract]
Total loan charge offs	68	50	332	100
Recoveries of loans previously charged off [Abstract]
Total recoveries	1	1	41	18
Net loans charged off	67	49	291	82
Provision (credit) for loan losses	194	47	846	33
Balance at end of period	5,046	4,022	5,046	4,022
Real Estate Mortgage - 1 to 4 Family [Member]
Allowance for loan losses by portfolio segment [Roll Forward]
Balance at beginning of period	39,017	41,529	39,753	42,088
Loans charged off [Abstract]
Total loan charge offs	1,107	1,235	2,080	2,458
Recoveries of loans previously charged off [Abstract]
Total recoveries	118	135	237	245
Net loans charged off	989	1,100	1,843	2,213
Provision (credit) for loan losses	561	658	679	1,212
Balance at end of period	38,589	41,087	38,589	41,087
Installment [Member]
Allowance for loan losses by portfolio segment [Roll Forward]
Balance at beginning of period	462	391	518	168
Loans charged off [Abstract]
Total loan charge offs	93	33	190	76
Recoveries of loans previously charged off [Abstract]
Total recoveries	15	9	26	15
Net loans charged off	78	24	164	61
Provision (credit) for loan losses	45	95	75	355
Balance at end of period	429	462	429	462
New York and Other States [Member]
Loans charged off [Abstract]
Total loan charge offs	1,250	1,149	2,484	2,356
Recoveries of loans previously charged off [Abstract]
Total recoveries	133	142	302	274
New York and Other States [Member] | Commercial [Member]
Loans charged off [Abstract]
Total loan charge offs	68	50	332	100
Recoveries of loans previously charged off [Abstract]
Total recoveries	1	0	41	16
New York and Other States [Member] | Real Estate Mortgage - 1 to 4 Family [Member]
Loans charged off [Abstract]
Total loan charge offs	1,090	1,066	1,979	2,180
Recoveries of loans previously charged off [Abstract]
Total recoveries	117	133	235	243
New York and Other States [Member] | Installment [Member]
Loans charged off [Abstract]
Total loan charge offs	92	33	173	76
Recoveries of loans previously charged off [Abstract]
Total recoveries	15	9	26	15
Florida [Member]
Loans charged off [Abstract]
Total loan charge offs	18	169	118	278
Recoveries of loans previously charged off [Abstract]
Total recoveries	1	3	2	4
Florida [Member] | Commercial [Member]
Loans charged off [Abstract]
Total loan charge offs	0	0	0	0
Recoveries of loans previously charged off [Abstract]
Total recoveries	0	1	0	2
Florida [Member] | Real Estate Mortgage - 1 to 4 Family [Member]
Loans charged off [Abstract]
Total loan charge offs	17	169	101	278
Recoveries of loans previously charged off [Abstract]
Total recoveries	1	2	2	2
Florida [Member] | Installment [Member]
Loans charged off [Abstract]
Total loan charge offs	1	0	17	0
Recoveries of loans previously charged off [Abstract]
Total recoveries	$ 0	$ 0	$ 0	$ 0