Operating Groups (Details) - USD ($)	3 Months Ended								12 Months Ended
	Feb. 03, 2018	Oct. 28, 2017	Jul. 29, 2017	Apr. 29, 2017	Jan. 28, 2017	Oct. 29, 2016	Jul. 30, 2016	Apr. 30, 2016	Feb. 03, 2018	Jan. 28, 2017	Jan. 30, 2016
Operating groups
Net Sales	$ 293,179,000	$ 235,960,000	$ 284,709,000	$ 272,363,000	$ 261,049,000	$ 222,308,000	$ 282,996,000	$ 256,235,000	$ 1,086,211,000	$ 1,022,588,000	$ 969,290,000
Depreciation and Amortization of Intangible Assets									42,402,000	42,083,000	35,760,000
Operating Income (Loss)	18,515,000	$ 1,124,000	$ 36,402,000	$ 29,959,000	19,516,000	$ (327,000)	$ 38,689,000	$ 32,006,000	86,000,000	89,884,000	97,514,000
Interest expense, net									3,109,000	3,421,000	2,458,000
Earnings from continuing operations before income taxes									82,891,000	86,463,000	95,056,000
Purchases of Property and Equipment									38,748,000	49,415,000	73,082,000
Total Assets	699,941,000				685,159,000				699,941,000	685,159,000
LIFO accounting charge	4,100,000				3,600,000				7,800,000	(5,900,000)
LIFO reserve	61,500,000				58,000,000				61,500,000	58,000,000
Corporate and Other
Operating groups
Net Sales									3,467,000	2,198,000	1,091,000
Depreciation and Amortization of Intangible Assets									1,359,000	1,451,000	1,557,000
Operating Income (Loss)									(26,660,000)	(12,885,000)	(18,704,000)
Total Assets	(8,419,000)				(19,814,000)				(8,419,000)	(19,814,000)
LIFO accounting charge									7,800,000	5,900,000	300,000
LIFO reserve	61,500				58,000				61,500	58,000
Tommy Bahama | Operating Segments
Operating groups
Net Sales									686,021,000	658,911,000	658,467,000
Depreciation and Amortization of Intangible Assets									30,998,000	31,796,000	28,103,000
Operating Income (Loss)									55,002,000	44,101,000	65,993,000
Total Assets	439,871,000				451,990,000				439,871,000	451,990,000
Lilly Pulitzer | Operating Segments
Operating groups
Net Sales									248,931,000	233,294,000	204,626,000
Depreciation and Amortization of Intangible Assets									9,021,000	7,968,000	5,644,000
Operating Income (Loss)									46,608,000	51,995,000	42,525,000
Total Assets	142,882,000				126,506,000				142,882,000	126,506,000
Lanier Apparel | Operating Segments
Operating groups
Net Sales									106,852,000	100,753,000	105,106,000
Depreciation and Amortization of Intangible Assets									583,000	478,000	456,000
Operating Income (Loss)									6,546,000	6,955,000	7,700,000
Total Assets	31,575,000				30,269,000				31,575,000	30,269,000
Southern Tide | Operating Segments
Operating groups
Net Sales									40,940,000	27,432,000	0
Depreciation and Amortization of Intangible Assets									441,000	390,000	0
Operating Income (Loss)									4,504,000	(282,000)	0
Total Assets	$ 94,032,000				$ 96,208,000				94,032,000	96,208,000
Continuing Operations
Operating groups
Purchases of Property and Equipment									38,748,000	49,415,000	72,422,000
Continuing Operations | Corporate and Other
Operating groups
Purchases of Property and Equipment									1,193,000	760,000	529,000
Continuing Operations | Tommy Bahama | Operating Segments
Operating groups
Purchases of Property and Equipment									24,962,000	34,191,000	54,490,000
Continuing Operations | Lilly Pulitzer | Operating Segments
Operating groups
Purchases of Property and Equipment									11,150,000	14,142,000	17,197,000
Continuing Operations | Lanier Apparel | Operating Segments
Operating groups
Purchases of Property and Equipment									305,000	295,000	206,000
Continuing Operations | Southern Tide | Operating Segments
Operating groups
Purchases of Property and Equipment									$ 1,138,000	$ 27,000	$ 0