Loans and Allowance for Loan Losses, Based on Impairment Review Method (Details) (USD $) In Thousands, unless otherwise specified	Sep. 30, 2014		Jun. 30, 2014	Dec. 31, 2013		Sep. 30, 2013	Jun. 30, 2013	Dec. 31, 2012
Ending allowance balance attributable to loans [Abstract]
Individually evaluated for impairment	$ 0			$ 0
Collectively evaluated for impairment	46,512			47,714
Total ending allowance balance	46,512		46,935	47,714		47,722	47,589	47,927
Loans [Abstract]
Individually evaluated for impairment	27,670			29,340
Collectively evaluated for impairment	3,055,359			2,879,469
Total loans	3,083,029			2,908,809
Commercial [Member]
Ending allowance balance attributable to loans [Abstract]
Individually evaluated for impairment	0			0
Collectively evaluated for impairment	4,045			4,019
Total ending allowance balance	4,045		4,073	4,019		3,864	3,719	3,771
Loans [Abstract]
Individually evaluated for impairment	5,043			8,082
Collectively evaluated for impairment	214,782			215,399
Total loans	219,825			223,481
1 to 4 Family Residential Real Estate [Member]
Ending allowance balance attributable to loans [Abstract]
Individually evaluated for impairment	0			0
Collectively evaluated for impairment	42,340			43,597
Total ending allowance balance	42,340		42,752	43,597		43,750	43,766	44,069
Loans [Abstract]
Individually evaluated for impairment	22,627			21,258
Collectively evaluated for impairment	2,834,020			2,658,175
Total loans	2,856,647			2,679,433
Installment [Member]
Ending allowance balance attributable to loans [Abstract]
Individually evaluated for impairment	0			0
Collectively evaluated for impairment	127			98
Total ending allowance balance	127		110	98		108	104	87
Loans [Abstract]
Individually evaluated for impairment	0			0
Collectively evaluated for impairment	6,557			5,895
Total loans	6,557			5,895
New York and Other States [Member]
Loans [Abstract]
Total loans	2,566,489	[1]		2,468,322	[1]
Florida [Member]
Loans [Abstract]
Total loans	$ 516,540			$ 440,487

[1] Includes New York, New Jersey, Vermont and Massachusetts.