Operating Group Information - Financial information (Details) - USD ($) $ in Thousands	12 Months Ended
	Jan. 29, 2022	Jan. 30, 2021	Feb. 01, 2020
Operating groups
Net sales	$ 1,142,079	$ 748,833	$ 1,122,790
Depreciation and amortization	41,598	59,914	40,287
Operating income (loss)	165,503	(123,849)	93,675
Interest expense, net	944	2,028	1,245
Earnings (loss) before income taxes	164,559	(125,877)	92,430
Impairment of goodwill and intangible assets		60,452
Corporate and Other
Operating groups
Net sales	39,871	24,478	19,829
Depreciation and amortization	1,203	1,336	1,358
Operating income (loss)	(24,687)	(6,786)	(18,834)
LIFO accounting charge (credit)	16,000	9,000	1,000
Tommy Bahama | Operating Segments
Operating groups
Net sales	724,305	419,817	676,652
Depreciation and amortization	27,830	46,698	27,852
Operating income (loss)	111,733	(53,310)	53,207
Lilly Pulitzer | Operating Segments
Operating groups
Net sales	298,995	231,078	284,700
Depreciation and amortization	11,678	9,965	10,106
Operating income (loss)	63,601	27,702	51,795
Southern Tide
Operating groups
Impairment of goodwill and intangible assets	0	60,000	0
Southern Tide | Operating Segments
Operating groups
Net sales	54,050	34,664	46,409
Depreciation and amortization	780	676	549
Operating income (loss)	9,968	(64,801)	5,554
Lanier Apparel | Operating Segments
Operating groups
Net sales	24,858	38,796	95,200
Depreciation and amortization	107	1,239	422
Operating income (loss)	$ 4,888	$ (26,654)	$ 1,953