Operating Group Information - Sales by operating group (Details) - USD ($) $ in Thousands	3 Months Ended		6 Months Ended
	Aug. 01, 2020	Aug. 03, 2019	Aug. 01, 2020	Aug. 03, 2019
Operating groups
Net sales	$ 191,988	$ 302,000	$ 352,331	$ 583,973
Operating Segments | Tommy Bahama
Operating groups
Net sales	95,254	188,870	182,238	353,600
Operating Segments | Lilly Pulitzer
Operating groups
Net sales	73,860	75,555	123,009	148,150
Operating Segments | Lanier Apparel
Operating groups
Net sales	8,450	20,466	19,175	46,620
Operating Segments | Southern Tide
Operating groups
Net sales	8,812	12,468	17,113	26,602
Corporate and Other
Operating groups
Net sales	5,612	4,641	10,796	9,001
Retail
Operating groups
Net sales	$ 41,783	$ 133,250	$ 87,286	$ 241,256
Net sales (as a percent)	22.00%	44.00%	25.00%	41.00%
Retail | Operating Segments | Tommy Bahama
Operating groups
Net sales (as a percent)	33.00%	50.00%	35.00%	49.00%
Retail | Operating Segments | Lilly Pulitzer
Operating groups
Net sales (as a percent)	14.00%	51.00%	18.00%	46.00%
Retail | Operating Segments | Southern Tide
Operating groups
Net sales (as a percent)	5.00%		3.00%
E-commerce
Operating groups
Net sales	$ 106,789	$ 70,437	$ 160,483	$ 118,426
Net sales (as a percent)	56.00%	23.00%	45.00%	20.00%
E-commerce | Operating Segments | Tommy Bahama
Operating groups
Net sales (as a percent)	48.00%	23.00%	39.00%	19.00%
E-commerce | Operating Segments | Lilly Pulitzer
Operating groups
Net sales (as a percent)	71.00%	29.00%	62.00%	27.00%
E-commerce | Operating Segments | Lanier Apparel
Operating groups
Net sales (as a percent)	1.00%
E-commerce | Operating Segments | Southern Tide
Operating groups
Net sales (as a percent)	43.00%	22.00%	32.00%	17.00%
E-commerce | Corporate and Other
Operating groups
Net sales (as a percent)	78.00%	61.00%	70.00%	62.00%
Restaurant
Operating groups
Net sales	$ 8,528	$ 20,531	$ 20,291	$ 44,132
Net sales (as a percent)	4.00%	7.00%	6.00%	8.00%
Restaurant | Operating Segments | Tommy Bahama
Operating groups
Net sales (as a percent)	9.00%	11.00%	11.00%	13.00%
Wholesale
Operating groups
Net sales	$ 34,489	$ 77,273	$ 83,547	$ 179,199
Net sales (as a percent)	18.00%	26.00%	24.00%	31.00%
Wholesale | Operating Segments | Tommy Bahama
Operating groups
Net sales (as a percent)	10.00%	16.00%	15.00%	19.00%
Wholesale | Operating Segments | Lilly Pulitzer
Operating groups
Net sales (as a percent)	15.00%	20.00%	20.00%	27.00%
Wholesale | Operating Segments | Lanier Apparel
Operating groups
Net sales (as a percent)	99.00%	100.00%	100.00%	100.00%
Wholesale | Operating Segments | Southern Tide
Operating groups
Net sales (as a percent)	52.00%	78.00%	65.00%	83.00%
Wholesale | Corporate and Other
Operating groups
Net sales (as a percent)	16.00%	31.00%	24.00%	29.00%
Other
Operating groups
Net sales	$ 399	$ 509	$ 724	$ 960
Other | Corporate and Other
Operating groups
Net sales (as a percent)	6.00%	8.00%	6.00%	9.00%