Multifamily Loans (Details) $ in Millions			3 Months Ended	9 Months Ended
	Sep. 30, 2019 USD ($) loan	Dec. 31, 2018 USD ($) loan	Sep. 30, 2019 USD ($) loan	Sep. 30, 2019 USD ($) loan
Multifamily loans, held-for-investment, at fair value
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Number of loans | loan	250	162
Loan principal	$ 3,540.0	$ 2,130.0	$ 3,540.0	$ 3,540.0
Loans, at fair value	$ 3,790.0	$ 2,140.0	$ 3,790.0	$ 3,790.0
Weighted average original loan-to-value (LTV)	69.00%		69.00%	69.00%
Weighted average coupon rate	4.19%		4.19%	4.19%
Contract maturities				6 years
Number of loans past due | loan	0	0	0	0
Number of loans in foreclosure | loan	0	0	0	0
Valuation adjustment gain (loss)			$ 47.0	$ 178.0
Multifamily loans, held-for-investment, at fair value | Minimum
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Weighted average maturity (in years)				7 years
Multifamily loans, held-for-investment, at fair value | Maximum
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Weighted average maturity (in years)				10 years
Held-for-investment at fair value | Redwood
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Number of loans | loan	3,118	3,296
Loan principal	$ 2,200.0	$ 2,390.0	$ 2,200.0	$ 2,200.0
Loans, at fair value		$ 2,380.0
Weighted average original loan-to-value (LTV)	66.00%		66.00%	66.00%
Number of loans in foreclosure | loan		0
Held-for-investment at fair value | Redwood | Financing Receivables, Equal to Greater than 90 Days Past Due
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Loan principal	$ 0.5	$ 1.0	$ 0.5	$ 0.5
Amount of loans in foreclosure	$ 0.6		$ 0.6	$ 0.6
Number of loans past due | loan	1	2	1	1