Operating Group Information - Sales by operating group (Details) - USD ($) $ in Thousands	3 Months Ended
	May 01, 2021	May 02, 2020
Operating groups
Net sales	$ 265,762	$ 160,343
Operating Segments | Tommy Bahama
Operating groups
Net sales	156,698	86,984
Operating Segments | Lilly Pulitzer
Operating groups
Net sales	73,576	49,149
Operating Segments | Southern Tide
Operating groups
Net sales	15,466	8,301
Operating Segments | Lanier Apparel
Operating groups
Net sales	12,019	10,725
Corporate and Other
Operating groups
Net sales	8,003	5,184
Retail
Operating groups
Net sales	$ 91,280	$ 45,503
Net sales (as a percent)	34.00%	28.00%
Retail | Operating Segments | Tommy Bahama
Operating groups
Net sales (as a percent)	42.00%	39.00%
Retail | Operating Segments | Lilly Pulitzer
Operating groups
Net sales (as a percent)	35.00%	24.00%
Retail | Operating Segments | Southern Tide
Operating groups
Net sales (as a percent)	5.00%	1.00%
E-commerce
Operating groups
Net sales	$ 74,238	$ 53,694
Net sales (as a percent)	28.00%	34.00%
E-commerce | Operating Segments | Tommy Bahama
Operating groups
Net sales (as a percent)	23.00%	28.00%
E-commerce | Operating Segments | Lilly Pulitzer
Operating groups
Net sales (as a percent)	42.00%	50.00%
E-commerce | Operating Segments | Southern Tide
Operating groups
Net sales (as a percent)	19.00%	21.00%
E-commerce | Corporate and Other
Operating groups
Net sales (as a percent)	57.00%	63.00%
Restaurant
Operating groups
Net sales	$ 25,208	$ 11,763
Net sales (as a percent)	10.00%	7.00%
Restaurant | Operating Segments | Tommy Bahama
Operating groups
Net sales (as a percent)	16.00%	13.00%
Wholesale
Operating groups
Net sales	$ 74,453	$ 49,058
Net sales (as a percent)	28.00%	31.00%
Wholesale | Operating Segments | Tommy Bahama
Operating groups
Net sales (as a percent)	19.00%	20.00%
Wholesale | Operating Segments | Lilly Pulitzer
Operating groups
Net sales (as a percent)	23.00%	26.00%
Wholesale | Operating Segments | Southern Tide
Operating groups
Net sales (as a percent)	76.00%	78.00%
Wholesale | Operating Segments | Lanier Apparel
Operating groups
Net sales (as a percent)	100.00%	100.00%
Wholesale | Corporate and Other
Operating groups
Net sales (as a percent)	37.00%	32.00%
Other
Operating groups
Net sales	$ 583	$ 325
Other | Corporate and Other
Operating groups
Net sales (as a percent)	6.00%	5.00%