Secured Financing Agreements (Details) $ in Thousands	1 Months Ended	3 Months Ended
	Feb. 28, 2017 USD ($) item	Mar. 31, 2017 USD ($)	Dec. 31, 2016 USD ($)
Secured Financing Agreements
Principal Amount		$ 2,073,229	$ 2,053,229
Unamortized deferred financing costs		(6,641)	(5,822)
Carrying Value		4,414,917	4,154,126
Lender 1 Repo 1 Facility
Secured Financing Agreements
Maximum borrowing capacity		1,800,000
Maximum facility size subject to certain conditions		2,000,000
Lender 4 Repo 2 Facility
Secured Financing Agreements
Maximum borrowing capacity		600,000
Maximum facility size subject to certain conditions		1,000,000
Lender 6 Repo 2 Facility
Secured Financing Agreements
Pledged Asset Carrying Value		176,686
Maximum Facility Size		123,568
Principal Amount		123,568	121,509
Lender 9 Repo 1 Facility
Secured Financing Agreements
Pledged Asset Carrying Value		378,607
Maximum Facility Size		283,575
Principal Amount		$ 283,575	283,575
Lender 9 Repo 1 Facility | LIBOR
Secured Financing Agreements
Pricing margin (as a percent)		1.65%
Lender 10 Repo 1 Facility
Secured Financing Agreements
Maximum Facility Size		$ 125,000
Lender 7 Secured Financing
Secured Financing Agreements
Maximum borrowing capacity		450,000
Maximum facility size subject to certain conditions		$ 650,000
MBS Repo 1 Facility
Secured Financing Agreements
Rolling maturity period		11 months
MBS Repo 3 Facility
Secured Financing Agreements
Rolling maturity period		12 months
Investing and Servicing Segment Property Mortgages
Secured Financing Agreements
Maximum Facility Size	$ 24,000
Maturity period	5 years
Investing and Servicing Segment Property Mortgages | LIBOR
Secured Financing Agreements
Pricing margin (as a percent)	2.00%
Medical Office Portfolio Mortgages | Additional Mortgage Facilities Acquired
Secured Financing Agreements
Maximum Facility Size	$ 7,300
Maturity period	5 years
Number of extension options | item	2
Extended term / option	12 months
Medical Office Portfolio Mortgages | Additional Mortgage Facilities Acquired | LIBOR
Secured Financing Agreements
Pricing margin (as a percent)	2.50%
Secured financing agreements
Secured Financing Agreements
Pledged Asset Carrying Value		$ 7,824,451
Maximum Facility Size		8,332,405
Principal Amount		4,456,347	4,197,218
Unamortized premium (discount), net		2,619	2,640
Unamortized deferred financing costs		(44,049)	(45,732)
Secured financing agreements | Lender 1 Repo 1 Facility
Secured Financing Agreements
Pledged Asset Carrying Value		1,775,380
Maximum Facility Size		2,000,000
Principal Amount		$ 1,213,260	944,712
Secured financing agreements | Lender 1 Repo 1 Facility | Minimum | LIBOR
Secured Financing Agreements
Pricing margin (as a percent)		1.75%
Secured financing agreements | Lender 1 Repo 1 Facility | Maximum | LIBOR
Secured Financing Agreements
Pricing margin (as a percent)		5.75%
Secured financing agreements | Lender 2 Repo 1 Facility
Secured Financing Agreements
Pledged Asset Carrying Value		$ 366,942
Maximum Facility Size		500,000
Principal Amount		$ 94,349	132,941
Secured financing agreements | Lender 2 Repo 1 Facility | Minimum | LIBOR
Secured Financing Agreements
Pricing margin (as a percent)		1.75%
Secured financing agreements | Lender 2 Repo 1 Facility | Maximum | LIBOR
Secured Financing Agreements
Pricing margin (as a percent)		2.75%
Secured financing agreements | Lender 3 Repo I Facility
Secured Financing Agreements
Pledged Asset Carrying Value		$ 110,076
Maximum Facility Size		78,017
Principal Amount		$ 78,017	78,288
Secured financing agreements | Lender 3 Repo I Facility | Minimum | LIBOR
Secured Financing Agreements
Pricing margin (as a percent)		2.50%
Secured financing agreements | Lender 3 Repo I Facility | Maximum | LIBOR
Secured Financing Agreements
Pricing margin (as a percent)		2.85%
Secured financing agreements | Lender 4 Repo 2 Facility
Secured Financing Agreements
Pledged Asset Carrying Value		$ 514,725
Maximum Facility Size		1,000,000
Principal Amount		$ 135,161	166,394
Secured financing agreements | Lender 4 Repo 2 Facility | Minimum | LIBOR
Secured Financing Agreements
Pricing margin (as a percent)		2.00%
Secured financing agreements | Lender 4 Repo 2 Facility | Maximum | LIBOR
Secured Financing Agreements
Pricing margin (as a percent)		2.50%
Secured financing agreements | Lender 6 Repo 1 Facility
Secured Financing Agreements
Pledged Asset Carrying Value		$ 339,964
Maximum Facility Size		500,000
Principal Amount		$ 218,728	182,586
Secured financing agreements | Lender 6 Repo 1 Facility | Minimum | LIBOR
Secured Financing Agreements
Pricing margin (as a percent)		2.50%
Secured financing agreements | Lender 6 Repo 1 Facility | Maximum | LIBOR
Secured Financing Agreements
Pricing margin (as a percent)		2.75%
Secured financing agreements | Lender 6 Repo 2 Facility | GBP LIBOR
Secured Financing Agreements
Pricing margin (as a percent)		2.75%
Secured financing agreements | Lender 10 Repo 1 Facility | Minimum | LIBOR
Secured Financing Agreements
Pricing margin (as a percent)		2.00%
Secured financing agreements | Lender 10 Repo 1 Facility | Maximum | LIBOR
Secured Financing Agreements
Pricing margin (as a percent)		2.75%
Secured financing agreements | Lender 7 Secured Financing
Secured Financing Agreements
Pledged Asset Carrying Value		$ 85,127
Maximum Facility Size		$ 650,000
Secured financing agreements | Lender 7 Secured Financing | LIBOR
Secured Financing Agreements
Pricing margin (as a percent)		2.75%
Secured financing agreements | Lender 8 Secured Financing
Secured Financing Agreements
Pledged Asset Carrying Value		$ 66,578
Maximum Facility Size		75,000
Principal Amount		$ 43,647	43,555
Secured financing agreements | Lender 8 Secured Financing | LIBOR
Secured Financing Agreements
Pricing margin (as a percent)		4.00%
Secured financing agreements | Conduit Repo 2 Facility
Secured Financing Agreements
Pledged Asset Carrying Value		$ 43,368
Maximum Facility Size		150,000
Principal Amount		$ 33,050	14,944
Secured financing agreements | Conduit Repo 2 Facility | LIBOR
Secured Financing Agreements
Pricing margin (as a percent)		2.25%
Secured financing agreements | Conduit Repo 3 Facility
Secured Financing Agreements
Pledged Asset Carrying Value		$ 8,683
Maximum Facility Size		150,000
Principal Amount		$ 6,525
Secured financing agreements | Conduit Repo 3 Facility | LIBOR
Secured Financing Agreements
Pricing margin (as a percent)		2.10%
Secured financing agreements | Conduit Repo 4 Facility
Secured Financing Agreements
Maximum Facility Size		$ 100,000
Secured financing agreements | Conduit Repo 4 Facility | LIBOR
Secured Financing Agreements
Pricing margin (as a percent)		2.25%
Secured financing agreements | MBS Repo 1 Facility
Secured Financing Agreements
Pledged Asset Carrying Value		$ 31,250
Maximum Facility Size		20,838
Principal Amount		$ 20,838	21,052
Secured financing agreements | MBS Repo 1 Facility | LIBOR
Secured Financing Agreements
Pricing margin (as a percent)		1.90%
Secured financing agreements | MBS Repo 2 Facility
Secured Financing Agreements
Pledged Asset Carrying Value		$ 331,800
Maximum Facility Size		240,892
Principal Amount		$ 240,892	239,434
Secured financing agreements | MBS Repo 2 Facility | Minimum | LIBOR/EURIBOR
Secured Financing Agreements
Pricing margin (as a percent)		2.00%
Secured financing agreements | MBS Repo 2 Facility | Maximum | LIBOR/EURIBOR
Secured Financing Agreements
Pricing margin (as a percent)		2.95%
Secured financing agreements | MBS Repo 3 Facility
Secured Financing Agreements
Pledged Asset Carrying Value		$ 389,540
Maximum Facility Size		260,933
Principal Amount		$ 260,933	285,209
Secured financing agreements | MBS Repo 3 Facility | Minimum | LIBOR
Secured Financing Agreements
Pricing margin (as a percent)		1.32%
Secured financing agreements | MBS Repo 3 Facility | Maximum | LIBOR
Secured Financing Agreements
Pricing margin (as a percent)		2.00%
Secured financing agreements | MBS Repo 4 Facility
Secured Financing Agreements
Pledged Asset Carrying Value		$ 185,435
Maximum Facility Size		225,000
Principal Amount		$ 8,146	5,633
Current maturity period, relative to when the buyer delivers notice to the seller		270 days
Secured financing agreements | MBS Repo 4 Facility | Minimum | LIBOR
Secured Financing Agreements
Pricing margin (as a percent)		1.20%
Secured financing agreements | MBS Repo 4 Facility | Maximum | LIBOR
Secured Financing Agreements
Pricing margin (as a percent)		1.90%
Secured financing agreements | Investing and Servicing Segment Property Mortgages
Secured Financing Agreements
Pledged Asset Carrying Value		$ 238,193
Maximum Facility Size		192,703
Principal Amount		172,981	164,611
Secured financing agreements | Ireland Portfolio Mortgage
Secured Financing Agreements
Pledged Asset Carrying Value		452,360
Maximum Facility Size		313,266
Principal Amount		$ 313,266	309,246
Secured financing agreements | Ireland Portfolio Mortgage | EURIBOR
Secured Financing Agreements
Pricing margin (as a percent)		1.69%
Secured financing agreements | Woodstar Portfolio Mortgages
Secured Financing Agreements
Pledged Asset Carrying Value		$ 373,957
Maximum Facility Size		276,748
Principal Amount		$ 276,748	276,748
Secured financing agreements | Woodstar Portfolio Mortgages | Minimum
Secured Financing Agreements
Interest rate (as a percent)		3.72%
Secured financing agreements | Woodstar Portfolio Mortgages | Maximum
Secured Financing Agreements
Interest rate (as a percent)		3.97%
Secured financing agreements | Woodstar Portfolio Government Financing
Secured Financing Agreements
Pledged Asset Carrying Value		$ 312,316
Maximum Facility Size		135,050
Principal Amount		$ 135,050	135,584
Secured financing agreements | Woodstar Portfolio Government Financing | Minimum
Secured Financing Agreements
Interest rate (as a percent)		1.00%
Secured financing agreements | Woodstar Portfolio Government Financing | Maximum
Secured Financing Agreements
Interest rate (as a percent)		5.00%
Secured financing agreements | Medical Office Portfolio Mortgages
Secured Financing Agreements
Pledged Asset Carrying Value		$ 758,684
Maximum Facility Size		531,815
Principal Amount		$ 497,613	491,197
Secured financing agreements | Medical Office Portfolio Mortgages | LIBOR
Secured Financing Agreements
Pricing margin (as a percent)		2.50%
Secured financing agreements | Term Loan A
Secured Financing Agreements
Pledged Asset Carrying Value		$ 884,780
Maximum Facility Size		300,000
Principal Amount		$ 300,000	$ 300,000
Secured financing agreements | Term Loan A | LIBOR
Secured Financing Agreements
Pricing margin (as a percent)		2.25%
Secured financing agreements | Revolving Secured Financing
Secured Financing Agreements
Maximum Facility Size		$ 100,000
Secured financing agreements | Revolving Secured Financing | LIBOR
Secured Financing Agreements
Pricing margin (as a percent)		2.25%