Operating Groups (Details) - USD ($) $ in Thousands	3 Months Ended								12 Months Ended
	Jan. 30, 2016	Oct. 31, 2015	Aug. 01, 2015	May. 02, 2015	Jan. 31, 2015	Nov. 01, 2014	Aug. 02, 2014	May. 03, 2014	Jan. 30, 2016	Jan. 31, 2015	Feb. 01, 2014
Operating groups
Net sales	$ 259,583	$ 198,624	$ 250,689	$ 260,394	$ 249,031	$ 201,178	$ 227,550	$ 242,566	$ 969,290	$ 920,325	$ 849,879
Depreciation and Amortization of Intangible Assets									35,760	34,564	30,748
Operating Income (Loss)	28,451	$ (1,166)	$ 34,746	$ 35,483	26,139	$ 4,388	$ 29,559	$ 32,733	97,514	92,819	96,312
Interest expense, net									2,458	3,236	3,940
Earnings from continuing operations before income taxes									95,056	89,583	92,372
Purchases of Property and Equipment									73,082	50,355	43,372
Total Assets	582,690				622,407				582,690	622,407
Continuing Operations
Operating groups
Purchases of Property and Equipment									72,422	46,065	42,235
Assets related to discontinued operations
Operating groups
Total Assets	0				79,870				0	79,870
Tommy Bahama
Operating groups
Net sales									658,467	627,498	584,941
Depreciation and Amortization of Intangible Assets									28,103	27,412	24,806
Operating Income (Loss)									65,993	71,132	72,207
Tommy Bahama | Operating Segments
Operating groups
Total Assets	458,234				420,083				458,234	420,083
Tommy Bahama | Operating Segments | Continuing Operations
Operating groups
Purchases of Property and Equipment									54,490	35,782	30,810
Lilly Pulitzer
Operating groups
Net sales									204,626	167,736	137,943
Depreciation and Amortization of Intangible Assets									5,644	4,616	3,215
Operating Income (Loss)									42,525	32,190	25,951
Lilly Pulitzer | Operating Segments
Operating groups
Total Assets	115,419				104,352				115,419	104,352
Lilly Pulitzer | Operating Segments | Continuing Operations
Operating groups
Purchases of Property and Equipment									17,197	7,335	10,343
Lanier Apparel
Operating groups
Net sales									105,106	126,430	127,421
Depreciation and Amortization of Intangible Assets									456	350	347
Operating Income (Loss)									7,700	10,043	11,900
Lanier Apparel | Operating Segments
Operating groups
Total Assets	35,451				41,455				35,451	41,455
Lanier Apparel | Operating Segments | Continuing Operations
Operating groups
Purchases of Property and Equipment									206	1,740	30
Corporate and Other
Operating groups
Net sales									1,091	(1,339)	(426)
Depreciation and Amortization of Intangible Assets									1,557	2,186	2,380
Operating Income (Loss)									(18,704)	(20,546)	(13,746)
Corporate and Other | Corporate, Non-Segment
Operating groups
Total Assets	$ (26,414)				$ (23,353)				(26,414)	(23,353)
Corporate and Other | Corporate, Non-Segment | Continuing Operations
Operating groups
Purchases of Property and Equipment									$ 529	$ 1,208	$ 1,052