Operating Groups (Details) - USD ($) $ in Thousands	3 Months Ended								12 Months Ended
	Jan. 28, 2017	Oct. 29, 2016	Jul. 30, 2016	Apr. 30, 2016	Jan. 30, 2016	Oct. 31, 2015	Aug. 01, 2015	May 02, 2015	Jan. 28, 2017	Jan. 30, 2016	Jan. 31, 2015
Operating groups
Net sales	$ 261,049	$ 222,308	$ 282,996	$ 256,235	$ 259,583	$ 198,624	$ 250,689	$ 260,394	$ 1,022,588	$ 969,290	$ 920,325
Depreciation and Amortization of Intangible Assets									42,083	35,760	34,564
Operating Income (Loss)	19,516	$ (327)	$ 38,689	$ 32,006	28,451	$ (1,166)	$ 34,746	$ 35,483	89,884	97,514	92,819
Interest expense, net									3,421	2,458	3,236
Earnings from continuing operations before income taxes									86,463	95,056	89,583
Purchases of Property and Equipment									49,415	73,082	50,355
Total Assets	685,159				582,690				685,159	582,690
Continuing Operations
Operating groups
Purchases of Property and Equipment									49,415	72,422	46,065
Tommy Bahama
Operating groups
Depreciation and Amortization of Intangible Assets									31,796	28,103	27,412
Operating Income (Loss)									44,101	65,993	71,132
Tommy Bahama | Operating Segments
Operating groups
Net sales									658,911	658,467	627,498
Total Assets	451,990				458,234				451,990	458,234
Tommy Bahama | Operating Segments | Continuing Operations
Operating groups
Purchases of Property and Equipment									34,191	54,490	35,782
Lilly Pulitzer
Operating groups
Depreciation and Amortization of Intangible Assets									7,968	5,644	4,616
Operating Income (Loss)									51,995	42,525	32,190
Lilly Pulitzer | Operating Segments
Operating groups
Net sales									233,294	204,626	167,736
Total Assets	126,506				115,419				126,506	115,419
Lilly Pulitzer | Operating Segments | Continuing Operations
Operating groups
Purchases of Property and Equipment									14,142	17,197	7,335
Lanier Apparel
Operating groups
Depreciation and Amortization of Intangible Assets									478	456	350
Operating Income (Loss)									6,955	7,700	10,043
Lanier Apparel | Operating Segments
Operating groups
Net sales									100,753	105,106	126,430
Total Assets	30,269				35,451				30,269	35,451
Lanier Apparel | Operating Segments | Continuing Operations
Operating groups
Purchases of Property and Equipment									295	206	1,740
Southern Tide
Operating groups
Depreciation and Amortization of Intangible Assets									390	0	0
Southern Tide | Operating Segments
Operating groups
Net sales									27,432	0	0
Depreciation and Amortization of Intangible Assets									(282)	0	0
Total Assets	96,208				0				96,208	0
Southern Tide | Operating Segments | Continuing Operations
Operating groups
Purchases of Property and Equipment									27	0	0
Corporate and Other
Operating groups
Net sales									2,198	1,091	(1,339)
Depreciation and Amortization of Intangible Assets									1,451	1,557	2,186
Operating Income (Loss)									(12,885)	(18,704)	(20,546)
Corporate and Other | Corporate, Non-Segment
Operating groups
Total Assets	$ (19,814)				$ (26,414)				(19,814)	(26,414)
Corporate and Other | Corporate, Non-Segment | Continuing Operations
Operating groups
Purchases of Property and Equipment									$ 760	$ 529	$ 1,208