Segment Reporting - Segment Results as Presented in Consolidated Financial Statements (Details) - USD ($) $ in Thousands	12 Months Ended
	Dec. 31, 2019	Dec. 31, 2018	Dec. 31, 2017
Segment Reporting Information [Line Items]
Revenues	$ 1,268,000	$ 1,416,424	$ 1,110,284
Voyage expenses	(129,910)	(151,808)	(99,444)
Vessel operating expenses	(426,951)	(437,671)	(353,564)
Time-charter hire expenses	(44,427)	(52,616)	(80,315)
General and administrative	(76,245)	(65,427)	(62,249)
Restructuring charge	0	(1,520)	(2,664)
Realized gain on foreign currency forward contracts	(5,054)	(1,228)	900
Adjusted EBITDA From Equity-Accounted Vessels	97,849	92,637	33,360
Adjusted EBITDA Attributable to Non-Controlling Interests	(11,364)	(16,270)	(23,914)
Adjusted EBITDA	671,898	782,521	522,394
Equity income	32,794	39,458	14,442
Investment in joint ventures	234,627	212,202	169,875
Expenditures for vessels and equipment, including advances on newbuilding contracts and conversion costs(3)	201,439	233,736	533,260
Expenditures for dry docking	13,546	23,602	17,183
Operating Segments | FPSO Segment
Segment Reporting Information [Line Items]
Revenues	492,658	533,186	458,388
Voyage expenses	0	0	0
Vessel operating expenses	(227,873)	(214,623)	(149,153)
Time-charter hire expenses	0	0	0
General and administrative	(40,846)	(34,052)	(33,046)
Restructuring charge		(1,520)	(450)
Realized gain on foreign currency forward contracts	0	0	0
Adjusted EBITDA From Equity-Accounted Vessels	97,849	92,637	33,360
Adjusted EBITDA Attributable to Non-Controlling Interests	0	0	0
Adjusted EBITDA	321,788	375,628	309,099
Equity income	32,794	39,458	14,442
Investment in joint ventures	234,627	212,202	169,875
Expenditures for vessels and equipment, including advances on newbuilding contracts and conversion costs(3)	8,284	54,371	193,817
Expenditures for dry docking	0	0	0
Operating Segments | Shuttle Tanker Segment
Segment Reporting Information [Line Items]
Revenues	549,587	636,413	536,852
Voyage expenses	(86,519)	(109,796)	(80,964)
Vessel operating expenses	(126,433)	(149,226)	(129,517)
Time-charter hire expenses	(40,108)	(36,421)	(62,899)
General and administrative	(20,788)	(21,763)	(17,425)
Restructuring charge		0	(210)
Realized gain on foreign currency forward contracts	0	0	0
Adjusted EBITDA From Equity-Accounted Vessels	0	0	0
Adjusted EBITDA Attributable to Non-Controlling Interests	(10,864)	(15,593)	(23,035)
Adjusted EBITDA	264,875	303,614	222,802
Equity income	0	0	0
Investment in joint ventures	0	0	0
Expenditures for vessels and equipment, including advances on newbuilding contracts and conversion costs(3)	185,156	147,540	216,157
Expenditures for dry docking	11,902	22,135	16,323
Operating Segments | FSO Segment
Segment Reporting Information [Line Items]
Revenues	140,117	136,557	66,901
Voyage expenses	(800)	(769)	(1,172)
Vessel operating expenses	(42,597)	(42,913)	(25,241)
Time-charter hire expenses	0	0	0
General and administrative	(4,006)	(2,174)	(1,864)
Restructuring charge		0	0
Realized gain on foreign currency forward contracts	0	0	0
Adjusted EBITDA From Equity-Accounted Vessels	0	0	0
Adjusted EBITDA Attributable to Non-Controlling Interests	(500)	(677)	(879)
Adjusted EBITDA	92,214	90,024	37,745
Equity income	0	0	0
Investment in joint ventures	0	0	0
Expenditures for vessels and equipment, including advances on newbuilding contracts and conversion costs(3)	6,967	6,987	88,039
Expenditures for dry docking	0	0	199
Operating Segments | UMS Segment
Segment Reporting Information [Line Items]
Revenues	2,940	36,536	4,236
Voyage expenses	(76)	(47)	(1,152)
Vessel operating expenses	(1,216)	(3,679)	(33,656)
Time-charter hire expenses	0	0	0
General and administrative	(6,100)	(3,547)	(5,068)
Restructuring charge		0	(2,004)
Realized gain on foreign currency forward contracts	0	0	0
Adjusted EBITDA From Equity-Accounted Vessels	0	0	0
Adjusted EBITDA Attributable to Non-Controlling Interests	0	0	0
Adjusted EBITDA	(4,452)	29,263	(37,644)
Equity income	0	0	0
Investment in joint ventures	0	0	0
Expenditures for vessels and equipment, including advances on newbuilding contracts and conversion costs(3)	922	0	3,931
Expenditures for dry docking	0	0	0
Operating Segments | Towage Segment
Segment Reporting Information [Line Items]
Revenues	74,726	53,327	38,771
Voyage expenses	(37,530)	(28,925)	(17,727)
Vessel operating expenses	(28,832)	(27,346)	(21,074)
Time-charter hire expenses	0	0	(925)
General and administrative	(4,401)	(3,531)	(4,486)
Restructuring charge		0	0
Realized gain on foreign currency forward contracts	0	0	0
Adjusted EBITDA From Equity-Accounted Vessels	0	0	0
Adjusted EBITDA Attributable to Non-Controlling Interests	0	0	0
Adjusted EBITDA	3,963	(6,475)	(5,441)
Equity income	0	0	0
Investment in joint ventures	0	0	0
Expenditures for vessels and equipment, including advances on newbuilding contracts and conversion costs(3)	110	24,838	31,316
Expenditures for dry docking	1,644	1,467	661
Operating Segments | Conventional Tanker Segment
Segment Reporting Information [Line Items]
Revenues	7,972	21,325	14,022
Voyage expenses	(4,985)	(12,453)	(359)
Vessel operating expenses	0	0	10
Time-charter hire expenses	(4,319)	(16,195)	(16,491)
General and administrative	(104)	(360)	(360)
Restructuring charge		0	0
Realized gain on foreign currency forward contracts	0	0	0
Adjusted EBITDA From Equity-Accounted Vessels	0	0	0
Adjusted EBITDA Attributable to Non-Controlling Interests	0	0	0
Adjusted EBITDA	(1,436)	(7,683)	(3,178)
Equity income	0	0	0
Investment in joint ventures	0	0	0
Expenditures for vessels and equipment, including advances on newbuilding contracts and conversion costs(3)	0	0	0
Expenditures for dry docking	0	0	0
Eliminations
Segment Reporting Information [Line Items]
Revenues	0	(920)	(8,886)
Voyage expenses	0	182	1,930
Vessel operating expenses	0	116	5,067
Time-charter hire expenses	0	0	0
General and administrative	0	0	0
Restructuring charge		0	0
Realized gain on foreign currency forward contracts	(5,054)	(1,228)	900
Adjusted EBITDA From Equity-Accounted Vessels	0	0	0
Adjusted EBITDA Attributable to Non-Controlling Interests	0	0	0
Adjusted EBITDA	(5,054)	(1,850)	(989)
Equity income	0	0	0
Investment in joint ventures	0	0	0
Expenditures for vessels and equipment, including advances on newbuilding contracts and conversion costs(3)	0	0	0
Expenditures for dry docking	$ 0	$ 0	$ 0