Intangible Assets and Goodwill - Goodwill (Details) - USD ($) $ in Thousands	6 Months Ended	12 Months Ended
	Aug. 01, 2020	Feb. 01, 2020
Changes in the carrying amount of goodwill
Balance at the beginning of the period	$ 66,578	$ 66,621
Impairment	(42,745)
Other, including foreign currency	33	(43)
Balance at the end of the period	23,866	66,578
Operating Segments | Tommy Bahama
Changes in the carrying amount of goodwill
Balance at the beginning of the period	711	754
Other, including foreign currency	33	(43)
Balance at the end of the period	744	711
Operating Segments | Lilly Pulitzer
Changes in the carrying amount of goodwill
Balance at the beginning of the period	19,522	19,522
Balance at the end of the period	19,522	19,522
Operating Segments | Southern Tide
Changes in the carrying amount of goodwill
Balance at the beginning of the period	42,745	42,745
Impairment	(42,745)
Balance at the end of the period		42,745
Corporate and Other
Changes in the carrying amount of goodwill
Balance at the beginning of the period	3,600	3,600
Balance at the end of the period	$ 3,600	$ 3,600