Investment Fair Value Changes, Net - Components of Investment Activities (Details) - Investment Fair Value Changes, Net - USD ($) $ in Thousands	3 Months Ended		9 Months Ended
	Sep. 30, 2019	Sep. 30, 2018	Sep. 30, 2019	Sep. 30, 2018
Investment Holdings [Line Items]
Changes in fair value of assets	$ 11,444	$ 10,332	$ 34,741	$ 12,830
Residential loans held-for-investment, at Redwood
Investment Holdings [Line Items]
Changes in fair value of assets	7,667	(17,063)	71,323	(71,058)
Single-family rental loans held-for-investment
Investment Holdings [Line Items]
Changes in fair value of assets	22	0	22	0
Residential bridge loans
Investment Holdings [Line Items]
Changes in fair value of assets	(742)	53	(1,363)	53
Trading securities
Investment Holdings [Line Items]
Changes in fair value of assets	15,275	6,314	55,577	2,429
Servicer advance investments
Investment Holdings [Line Items]
Changes in fair value of assets	1,585	0	3,025	0
Excess MSRs
Investment Holdings [Line Items]
Changes in fair value of assets	(1,635)	0	(2,137)	0
Home appreciation options
Investment Holdings [Line Items]
Changes in fair value of assets	29	0	29	0
REO
Investment Holdings [Line Items]
Changes in fair value of assets	(331)	0	(470)	0
Net investments in Legacy Sequoia entities
Investment Holdings [Line Items]
Changes in fair value of assets	(407)	(248)	(904)	(976)
Net investments in Sequoia Choice entities
Investment Holdings [Line Items]
Changes in fair value of assets	2,722	(943)	8,866	43
Net investments in Freddie Mac SLST entities
Investment Holdings [Line Items]
Changes in fair value of assets	17,300	0	31,702	0
Net investments in Freddie Mac K-Series entities
Investment Holdings [Line Items]
Changes in fair value of assets	7,445	511	13,810	511
Risk-sharing investments
Investment Holdings [Line Items]
Changes in fair value of assets	(53)	(126)	(191)	(474)
Risk management derivatives, net
Investment Holdings [Line Items]
Changes in fair value of assets	(37,433)	21,867	(144,548)	82,391
Impairments on AFS securities
Investment Holdings [Line Items]
Changes in fair value of assets	$ 0	$ (33)	$ 0	$ (89)