Operating Group Information - Financial information (Details) - USD ($) $ in Thousands	3 Months Ended
	May 01, 2021	May 02, 2020	Jan. 30, 2021
Operating groups
Net sales	$ 265,762	$ 160,343
Depreciation and amortization	9,683	10,693
Operating income (loss)	34,893	(85,489)
Interest expense, net	252	658
Earnings (loss) before income taxes	34,641	(86,147)
Assets	900,982	1,097,470	$ 865,634
Impairment of goodwill and intangible assets		60,452
Corporate and Other
Operating groups
Net sales	8,003	5,184
Depreciation and amortization	321	331
Operating income (loss)	(9,820)	(270)
Assets	100,800	167,634	76,705
Tommy Bahama | Operating Segments
Operating groups
Net sales	156,698	86,984
Depreciation and amortization	7,040	7,800
Operating income (loss)	20,660	(23,362)
Assets	569,391	650,057	569,854
Lilly Pulitzer | Operating Segments
Operating groups
Net sales	73,576	49,149
Depreciation and amortization	2,099	2,316
Operating income (loss)	19,945	4,146
Assets	188,886	202,983	176,467
Southern Tide | Operating Segments
Operating groups
Net sales	15,466	8,301
Depreciation and amortization	187	144
Operating income (loss)	3,253	(63,366)
Assets	32,285	37,513	31,641
Lanier Apparel | Operating Segments
Operating groups
Net sales	12,019	10,725
Depreciation and amortization	36	102
Operating income (loss)	855	(2,637)
Assets	$ 9,620	$ 39,283	$ 10,967