Operating Group Information - Sales by operating group (Details) - USD ($) $ in Thousands	3 Months Ended		6 Months Ended
	Aug. 03, 2019	Aug. 04, 2018	Aug. 03, 2019	Aug. 04, 2018
Operating groups
Net Sales	$ 302,000	$ 302,641	$ 583,973	$ 575,269
Operating Segments | Tommy Bahama
Operating groups
Net Sales	188,870	192,728	353,600	359,860
Operating Segments | Lilly Pulitzer
Operating groups
Net Sales	75,555	71,623	148,150	140,250
Operating Segments | Lanier Apparel
Operating groups
Net Sales	20,905	23,860	47,494	43,769
Operating Segments | Southern Tide
Operating groups
Net Sales	12,468	11,777	26,602	25,249
Corporate and Other
Operating groups
Net Sales	4,202	2,653	8,127	6,141
Retail
Operating groups
Net Sales	$ 133,250	$ 134,581	$ 241,256	$ 242,316
Net Sales (as a percent)	44.00%	45.00%	41.00%	42.00%
Retail | Operating Segments | Tommy Bahama
Operating groups
Net Sales (as a percent)	50.00%	50.00%	49.00%	49.00%
Retail | Operating Segments | Lilly Pulitzer
Operating groups
Net Sales (as a percent)	51.00%	52.00%	46.00%	48.00%
E-commerce
Operating groups
Net Sales	$ 70,437	$ 63,363	$ 118,426	$ 107,885
Net Sales (as a percent)	23.00%	21.00%	20.00%	19.00%
E-commerce | Operating Segments | Tommy Bahama
Operating groups
Net Sales (as a percent)	23.00%	21.00%	19.00%	18.00%
E-commerce | Operating Segments | Lilly Pulitzer
Operating groups
Net Sales (as a percent)	29.00%	27.00%	27.00%	26.00%
E-commerce | Operating Segments | Lanier Apparel
Operating groups
Net Sales (as a percent)	2.00%		1.00%
E-commerce | Operating Segments | Southern Tide
Operating groups
Net Sales (as a percent)	22.00%	20.00%	17.00%	16.00%
E-commerce | Corporate and Other
Operating groups
Net Sales (as a percent)	62.00%	51.00%	62.00%	56.00%
Restaurant
Operating groups
Net Sales	$ 20,531	$ 21,467	$ 44,132	$ 46,760
Net Sales (as a percent)	7.00%	7.00%	8.00%	8.00%
Restaurant | Operating Segments | Tommy Bahama
Operating groups
Net Sales (as a percent)	11.00%	11.00%	13.00%	13.00%
Wholesale
Operating groups
Net Sales	$ 77,273	$ 82,402	$ 179,199	$ 176,778
Net Sales (as a percent)	26.00%	27.00%	31.00%	31.00%
Wholesale | Operating Segments | Tommy Bahama
Operating groups
Net Sales (as a percent)	16.00%	18.00%	19.00%	20.00%
Wholesale | Operating Segments | Lilly Pulitzer
Operating groups
Net Sales (as a percent)	20.00%	21.00%	27.00%	26.00%
Wholesale | Operating Segments | Lanier Apparel
Operating groups
Net Sales (as a percent)	98.00%	100.00%	99.00%	100.00%
Wholesale | Operating Segments | Southern Tide
Operating groups
Net Sales (as a percent)	78.00%	80.00%	83.00%	84.00%
Wholesale | Corporate and Other
Operating groups
Net Sales (as a percent)	29.00%	22.00%	29.00%	23.00%
Other
Operating groups
Net Sales	$ 509	$ 828	$ 960	$ 1,530
Other | Corporate and Other
Operating groups
Net Sales (as a percent)	9.00%	27.00%	9.00%	21.00%