Secured Financing Agreements (Details) $ in Thousands	1 Months Ended			6 Months Ended
	Jun. 30, 2017 USD ($)	Mar. 31, 2017 USD ($) item	Feb. 28, 2017 USD ($) item	Jun. 30, 2017 USD ($)	May 31, 2017 USD ($)	Dec. 31, 2016 USD ($)
Secured Financing Agreements
Principal Amount	$ 2,073,229			$ 2,073,229		$ 2,053,229
Unamortized deferred financing costs	(6,169)			(6,169)		(5,822)
Carrying Value	4,750,923			4,750,923		4,154,126
Lender 1 Repo 1 Facility
Secured Financing Agreements
Maximum borrowing capacity	1,800,000			1,800,000
Maximum facility size subject to certain conditions	2,000,000			2,000,000
Lender 4 Repo 2 Facility
Secured Financing Agreements
Maximum borrowing capacity	600,000			600,000
Maximum facility size subject to certain conditions	1,000,000			1,000,000
Lender 6 Repo 2 Facility
Secured Financing Agreements
Pledged Asset Carrying Value	181,171			181,171
Maximum Facility Size	126,627			126,627
Principal Amount	126,627			126,627		121,509
Lender 9 Repo 1 Facility
Secured Financing Agreements
Pledged Asset Carrying Value	379,111			379,111
Maximum Facility Size	283,575			283,575
Principal Amount	283,575			$ 283,575		283,575
Lender 9 Repo 1 Facility | LIBOR
Secured Financing Agreements
Pricing margin (as a percent)				1.65%
Lender 10 Repo 1 Facility
Secured Financing Agreements
Pledged Asset Carrying Value	169,610			$ 169,610
Maximum Facility Size	140,000	$ 125,000		140,000	$ 140,000
Principal Amount	136,800			136,800
Maturity period		3 years
Number of extension options | item		2
Extended term / option		1 year
Lender 10 Repo 1 Facility | LIBOR
Secured Financing Agreements
Interest rate (as a percent)		2.00%
Lender 10 Repo 1 Facility | Maximum | LIBOR
Secured Financing Agreements
Interest rate (as a percent)		2.75%
Lender 11 Repo 1 Facility
Secured Financing Agreements
Maximum Facility Size	$ 200,000			$ 200,000
Maturity period	2 years
Extended term / option	1 year
Lender 11 Repo 1 Facility | LIBOR
Secured Financing Agreements
Pricing margin (as a percent)				2.75%
Lender 7 Secured Financing
Secured Financing Agreements
Maximum borrowing capacity	$ 450,000			$ 450,000
Maximum facility size subject to certain conditions	650,000			$ 650,000
MBS Repo 1 Facility
Secured Financing Agreements
Rolling maturity period				11 months
MBS Repo 3 Facility
Secured Financing Agreements
Rolling maturity period				12 months
Investing and Servicing Segment Property Mortgages
Secured Financing Agreements
Maximum Facility Size			$ 24,000
Maturity period			5 years
Investing and Servicing Segment Property Mortgages | LIBOR
Secured Financing Agreements
Pricing margin (as a percent)			2.00%
Medical Office Portfolio Mortgages
Secured Financing Agreements
Floor interest rate (as a percent)				0.25%
Medical Office Portfolio Mortgages | Additional Mortgage Facilities Acquired
Secured Financing Agreements
Maximum Facility Size			$ 7,300
Maturity period			5 years
Number of extension options | item			2
Extended term / option			12 months
Medical Office Portfolio Mortgages | Additional Mortgage Facilities Acquired | LIBOR
Secured Financing Agreements
Pricing margin (as a percent)			2.50%
Secured financing agreements
Secured Financing Agreements
Pledged Asset Carrying Value	7,958,342			$ 7,958,342
Maximum Facility Size	8,574,589			8,574,589
Principal Amount	4,788,996			4,788,996		4,197,218
Unamortized premium (discount), net	2,599			2,599		2,640
Unamortized deferred financing costs	(40,672)			(40,672)		(45,732)
Secured financing agreements | Lender 1 Repo 1 Facility
Secured Financing Agreements
Pledged Asset Carrying Value	1,637,815			1,637,815
Maximum Facility Size	2,000,000			2,000,000
Principal Amount	1,196,618			$ 1,196,618		944,712
Secured financing agreements | Lender 1 Repo 1 Facility | Minimum | LIBOR
Secured Financing Agreements
Pricing margin (as a percent)				1.75%
Secured financing agreements | Lender 1 Repo 1 Facility | Maximum | LIBOR
Secured Financing Agreements
Pricing margin (as a percent)				5.75%
Secured financing agreements | Lender 2 Repo 1 Facility
Secured Financing Agreements
Pledged Asset Carrying Value	363,744			$ 363,744
Maximum Facility Size	500,000			500,000
Principal Amount	110,646			$ 110,646		132,941
Secured financing agreements | Lender 2 Repo 1 Facility | Minimum | LIBOR
Secured Financing Agreements
Pricing margin (as a percent)				1.75%
Secured financing agreements | Lender 2 Repo 1 Facility | Maximum | LIBOR
Secured Financing Agreements
Pricing margin (as a percent)				2.75%
Secured financing agreements | Lender 3 Repo I Facility
Secured Financing Agreements
Pledged Asset Carrying Value	110,193			$ 110,193
Maximum Facility Size	77,645			77,645
Principal Amount	77,645			$ 77,645		78,288
Secured financing agreements | Lender 3 Repo I Facility | Minimum | LIBOR
Secured Financing Agreements
Pricing margin (as a percent)				2.75%
Secured financing agreements | Lender 3 Repo I Facility | Maximum | LIBOR
Secured Financing Agreements
Pricing margin (as a percent)				3.10%
Secured financing agreements | Lender 4 Repo 2 Facility
Secured Financing Agreements
Pledged Asset Carrying Value	624,738			$ 624,738
Maximum Facility Size	1,000,000			1,000,000
Principal Amount	230,740			$ 230,740		166,394
Secured financing agreements | Lender 4 Repo 2 Facility | Minimum | LIBOR
Secured Financing Agreements
Pricing margin (as a percent)				2.00%
Secured financing agreements | Lender 4 Repo 2 Facility | Maximum | LIBOR
Secured Financing Agreements
Pricing margin (as a percent)				2.50%
Secured financing agreements | Lender 6 Repo 1 Facility
Secured Financing Agreements
Pledged Asset Carrying Value	268,037			$ 268,037
Maximum Facility Size	500,000			500,000
Principal Amount	210,254			$ 210,254		182,586
Secured financing agreements | Lender 6 Repo 1 Facility | Minimum | LIBOR
Secured Financing Agreements
Pricing margin (as a percent)				2.50%
Secured financing agreements | Lender 6 Repo 1 Facility | Maximum | LIBOR
Secured Financing Agreements
Pricing margin (as a percent)				2.75%
Secured financing agreements | Lender 6 Repo 2 Facility | GBP LIBOR
Secured Financing Agreements
Pricing margin (as a percent)				2.75%
Secured financing agreements | Lender 10 Repo 1 Facility | Minimum | LIBOR
Secured Financing Agreements
Pricing margin (as a percent)				2.00%
Secured financing agreements | Lender 10 Repo 1 Facility | Maximum | LIBOR
Secured Financing Agreements
Pricing margin (as a percent)				2.75%
Secured financing agreements | Lender 7 Secured Financing
Secured Financing Agreements
Pledged Asset Carrying Value	85,324			$ 85,324
Maximum Facility Size	650,000			$ 650,000
Secured financing agreements | Lender 7 Secured Financing | LIBOR
Secured Financing Agreements
Pricing margin (as a percent)				2.75%
Secured financing agreements | Lender 8 Secured Financing
Secured Financing Agreements
Pledged Asset Carrying Value	49,183			$ 49,183
Maximum Facility Size	75,000			75,000
Principal Amount	32,124			$ 32,124		43,555
Secured financing agreements | Lender 8 Secured Financing | LIBOR
Secured Financing Agreements
Pricing margin (as a percent)				4.00%
Secured financing agreements | Conduit Repo 2 Facility
Secured Financing Agreements
Pledged Asset Carrying Value	138,789			$ 138,789
Maximum Facility Size	150,000			150,000
Principal Amount	109,070			$ 109,070		14,944
Secured financing agreements | Conduit Repo 2 Facility | LIBOR
Secured Financing Agreements
Pricing margin (as a percent)				2.25%
Secured financing agreements | Conduit Repo 3 Facility
Secured Financing Agreements
Maximum Facility Size	150,000			$ 150,000
Secured financing agreements | Conduit Repo 3 Facility | LIBOR
Secured Financing Agreements
Pricing margin (as a percent)				2.10%
Secured financing agreements | Conduit Repo 4 Facility
Secured Financing Agreements
Maximum Facility Size	100,000			$ 100,000
Secured financing agreements | Conduit Repo 4 Facility | LIBOR
Secured Financing Agreements
Pricing margin (as a percent)				2.25%
Secured financing agreements | MBS Repo 1 Facility
Secured Financing Agreements
Pledged Asset Carrying Value	31,250			$ 31,250
Maximum Facility Size	20,838			20,838
Principal Amount	20,838			$ 20,838		21,052
Secured financing agreements | MBS Repo 1 Facility | LIBOR
Secured Financing Agreements
Pricing margin (as a percent)				1.90%
Secured financing agreements | MBS Repo 2 Facility
Secured Financing Agreements
Pledged Asset Carrying Value	344,829			$ 344,829
Maximum Facility Size	250,178			250,178
Principal Amount	250,178			$ 250,178		239,434
Secured financing agreements | MBS Repo 2 Facility | Minimum | LIBOR/EURIBOR
Secured Financing Agreements
Pricing margin (as a percent)				2.00%
Secured financing agreements | MBS Repo 2 Facility | Maximum | LIBOR/EURIBOR
Secured Financing Agreements
Pricing margin (as a percent)				2.95%
Secured financing agreements | MBS Repo 3 Facility
Secured Financing Agreements
Pledged Asset Carrying Value	386,771			$ 386,771
Maximum Facility Size	256,832			256,832
Principal Amount	256,832			$ 256,832		285,209
Secured financing agreements | MBS Repo 3 Facility | Minimum | LIBOR
Secured Financing Agreements
Pricing margin (as a percent)				1.32%
Secured financing agreements | MBS Repo 3 Facility | Maximum | LIBOR
Secured Financing Agreements
Pricing margin (as a percent)				1.95%
Secured financing agreements | MBS Repo 4 Facility
Secured Financing Agreements
Pledged Asset Carrying Value	184,678			$ 184,678
Maximum Facility Size	225,000			225,000
Principal Amount	32,000			$ 32,000		5,633
Current maturity period, relative to when the buyer delivers notice to the seller				270 days
Secured financing agreements | MBS Repo 4 Facility | Minimum | LIBOR
Secured Financing Agreements
Pricing margin (as a percent)				1.20%
Secured financing agreements | MBS Repo 4 Facility | Maximum | LIBOR
Secured Financing Agreements
Pricing margin (as a percent)				1.90%
Secured financing agreements | Investing and Servicing Segment Property Mortgages
Secured Financing Agreements
Pledged Asset Carrying Value	235,310			$ 235,310
Maximum Facility Size	192,596			192,596
Principal Amount	172,953			172,953		164,611
Secured financing agreements | Ireland Portfolio Mortgage
Secured Financing Agreements
Pledged Asset Carrying Value	477,387			477,387
Maximum Facility Size	333,225			333,225
Principal Amount	333,225			$ 333,225		309,246
Secured financing agreements | Ireland Portfolio Mortgage | EURIBOR
Secured Financing Agreements
Pricing margin (as a percent)				1.69%
Secured financing agreements | Woodstar Portfolio Mortgages
Secured Financing Agreements
Pledged Asset Carrying Value	371,409			$ 371,409
Maximum Facility Size	276,748			276,748
Principal Amount	$ 276,748			$ 276,748		276,748
Secured financing agreements | Woodstar Portfolio Mortgages | Minimum
Secured Financing Agreements
Interest rate (as a percent)	3.72%			3.72%
Secured financing agreements | Woodstar Portfolio Mortgages | Maximum
Secured Financing Agreements
Interest rate (as a percent)	3.97%			3.97%
Secured financing agreements | Woodstar Portfolio Government Financing
Secured Financing Agreements
Pledged Asset Carrying Value	$ 310,303			$ 310,303
Maximum Facility Size	134,510			134,510
Principal Amount	$ 134,510			$ 134,510		135,584
Secured financing agreements | Woodstar Portfolio Government Financing | Minimum
Secured Financing Agreements
Interest rate (as a percent)	1.00%			1.00%
Secured financing agreements | Woodstar Portfolio Government Financing | Maximum
Secured Financing Agreements
Interest rate (as a percent)	5.00%			5.00%
Secured financing agreements | Medical Office Portfolio Mortgages
Secured Financing Agreements
Pledged Asset Carrying Value	$ 749,973			$ 749,973
Maximum Facility Size	531,815			531,815
Principal Amount	497,613			$ 497,613		491,197
Secured financing agreements | Medical Office Portfolio Mortgages | LIBOR
Secured Financing Agreements
Pricing margin (as a percent)				2.50%
Secured financing agreements | Term Loan A
Secured Financing Agreements
Pledged Asset Carrying Value	858,717			$ 858,717
Maximum Facility Size	300,000			300,000
Principal Amount	300,000			$ 300,000		$ 300,000
Secured financing agreements | Term Loan A | LIBOR
Secured Financing Agreements
Pricing margin (as a percent)				2.25%
Secured financing agreements | Revolving Secured Financing
Secured Financing Agreements
Maximum Facility Size	$ 100,000			$ 100,000
Secured financing agreements | Revolving Secured Financing | LIBOR
Secured Financing Agreements
Pricing margin (as a percent)				2.25%