Reportable Segment Information (Details) $ in Thousands	3 Months Ended		9 Months Ended
	Sep. 26, 2020 USD ($)	Sep. 28, 2019 USD ($)	Sep. 26, 2020 USD ($) segment	Sep. 28, 2019 USD ($)	Dec. 28, 2019 USD ($)
Segment Reporting Information [Line Items]
Number of reportable segments | segment			2
Net sales	$ 507,063	$ 625,386	$ 1,393,224	$ 1,630,868
Operating income	38,762	60,656	29,876	87,581
Interest expense, net	1,517	2,205	5,271	6,795
Income before income taxes	37,245	58,451	24,605	80,786
Depreciation and amortization	19,312	19,386	57,917	57,838
Capital expenditures (including capitalized software)	11,225	12,584	32,001	50,191
Identifiable assets	1,374,725		1,374,725		$ 1,452,512
Workplace furnishings
Segment Reporting Information [Line Items]
Net sales	353,361	484,755	999,827	1,247,778
Residential building products
Segment Reporting Information [Line Items]
Net sales	153,702	140,631	393,397	383,090
Operating segments | Workplace furnishings
Segment Reporting Information [Line Items]
Net sales	353,361	484,755	999,827	1,247,778
Operating income	16,826	51,162	(8,619)	68,180
Depreciation and amortization	11,065	11,232	33,177	33,540
Capital expenditures (including capitalized software)	6,946	6,524	18,340	29,190
Identifiable assets	714,896		714,896		874,913
Operating segments | Residential building products
Segment Reporting Information [Line Items]
Net sales	153,702	140,631	393,397	383,090
Operating income	30,197	23,772	65,232	54,743
Depreciation and amortization	2,351	2,291	6,976	6,521
Capital expenditures (including capitalized software)	2,695	3,204	5,874	10,779
Identifiable assets	393,941		393,941		364,653
General corporate
Segment Reporting Information [Line Items]
Operating income	(8,261)	(14,278)	(26,737)	(35,342)
Depreciation and amortization	5,896	5,863	17,764	17,777
Capital expenditures (including capitalized software)	1,584	$ 2,856	7,787	$ 10,222
Identifiable assets	$ 265,888		$ 265,888		$ 212,946