Intangible Assets and Goodwill - Goodwill (Details) - USD ($) $ in Thousands	3 Months Ended	9 Months Ended	12 Months Ended
	May 02, 2020	Jan. 30, 2021	Jan. 29, 2022	Jan. 30, 2021	Feb. 01, 2020
Changes in the carrying amount of goodwill
Balance at the beginning of the period	$ 66,578		$ 23,910	$ 66,578	$ 66,621
Impairment	(43,000)	$ 0	0	(42,745)	0
Other, including foreign currency			(41)	77	(43)
Balance at the end of the period		23,910	23,869	23,910	66,578
Operating Segments | Tommy Bahama
Changes in the carrying amount of goodwill
Balance at the beginning of the period	711		788	711	754
Other, including foreign currency			(41)	77	(43)
Balance at the end of the period		788	747	788	711
Operating Segments | Lilly Pulitzer
Changes in the carrying amount of goodwill
Balance at the beginning of the period	19,522		19,522	19,522	19,522
Balance at the end of the period		19,522	19,522	19,522	19,522
Operating Segments | Southern Tide
Changes in the carrying amount of goodwill
Balance at the beginning of the period	42,745			42,745	42,745
Impairment				(42,745)
Balance at the end of the period					42,745
Corporate and Other
Changes in the carrying amount of goodwill
Balance at the beginning of the period	$ 3,600		3,600	3,600	3,600
Balance at the end of the period		$ 3,600	$ 3,600	$ 3,600	$ 3,600