Intangible Assets and Goodwill (Details) - USD ($) $ in Thousands	12 Months Ended
	Jan. 28, 2017	Jan. 30, 2016	Jan. 31, 2015	Jan. 28, 2017	Jan. 30, 2016
Intangible assets with finite lives, which primarily consist of customer relationships:
Intangible assets with finite lives				$ 46,030	$ 38,897
Accumulated amortization				(35,785)	(33,359)
Total intangible assets with finite lives, net				10,245	5,538
Intangible assets with indefinite lives:
Trademarks				165,000	138,200
Total intangible assets, net	$ 143,738	$ 143,738		175,245	143,738
Changes in carrying amount of intangible assets
Balance at the beginning of the period	143,738
Amortization	(2,150)	(1,951)	$ (2,481)
Balance at the end of the period	175,245	143,738
Expected amortization expense
2016				2,200
2017				1,500
2018				600
2019				600
2020				600
Continuing Operations
Intangible assets with indefinite lives:
Total intangible assets, net	143,738	146,134	149,168	175,245	143,738
Changes in carrying amount of intangible assets
Balance at the beginning of the period	143,738	146,134	149,168
Amortization	(2,150)	(1,926)	(2,282)
Other, including foreign currency changes	280	(470)	(752)
Acquisition	33,377
Balance at the end of the period	175,245	143,738	146,134
Tommy Bahama | Continuing Operations
Intangible assets with indefinite lives:
Total intangible assets, net	114,944	117,102	119,858	113,625	114,944
Changes in carrying amount of intangible assets
Balance at the beginning of the period	114,944	117,102	119,858
Amortization	(1,599)	(1,688)	(2,004)
Other, including foreign currency changes	280	(470)	(752)
Balance at the end of the period	113,625	114,944	117,102
Lilly Pulitzer | Continuing Operations
Intangible assets with indefinite lives:
Total intangible assets, net	28,794	29,032	29,310	28,595	$ 28,794
Changes in carrying amount of intangible assets
Balance at the beginning of the period	28,794	29,032	29,310
Amortization	(199)	(238)	(278)
Other, including foreign currency changes	0	0	0
Balance at the end of the period	28,595	28,794	29,032
Lanier Apparel | Continuing Operations
Intangible assets with indefinite lives:
Total intangible assets, net	3,048			3,048
Changes in carrying amount of intangible assets
Amortization	(89)
Other, including foreign currency changes	0	0	0
Acquisition	3,137
Balance at the end of the period	3,048
Southern Tide | Continuing Operations
Intangible assets with indefinite lives:
Total intangible assets, net	29,977			$ 29,977
Changes in carrying amount of intangible assets
Amortization	(263)
Other, including foreign currency changes	0	$ 0	$ 0
Acquisition	30,240
Balance at the end of the period	$ 29,977