Mortgage Loans - Credit Quality Indicators for Amortized Cost of Mortgage Loans by Year (Details) - USD ($) $ in Thousands	Mar. 31, 2020	Dec. 31, 2019
Financing Receivable, Credit Quality Indicator [Line Items]
2020	$ 103,449
2019	476,951
2018	807,775
2017	980,783
2016	1,121,359
Prior	1,694,493
Total	5,184,810
Allowance for loan losses	(59,445)	$ (19,160)
Total, net of allowance	5,125,365
Apartment
Financing Receivable, Credit Quality Indicator [Line Items]
2020	0
2019	82,319
2018	39,362
2017	189,540
2016	69,053
Prior	52,107
Total	432,381
Allowance for loan losses	(2,555)
Hotel
Financing Receivable, Credit Quality Indicator [Line Items]
2020	0
2019	54,043
2018	203,956
2017	219,528
2016	148,142
Prior	274,135
Total	899,804
Allowance for loan losses	(26,563)
Industrial
Financing Receivable, Credit Quality Indicator [Line Items]
2020	53,193
2019	164,383
2018	123,825
2017	51,638
2016	125,680
Prior	137,387
Total	656,106
Allowance for loan losses	(3,849)
Office
Financing Receivable, Credit Quality Indicator [Line Items]
2020	0
2019	53,326
2018	189,338
2017	349,173
2016	315,911
Prior	663,489
Total	1,571,237
Allowance for loan losses	(5,891)
Retail
Financing Receivable, Credit Quality Indicator [Line Items]
2020	22,584
2019	41,237
2018	108,546
2017	81,359
2016	224,469
Prior	379,668
Total	857,863
Allowance for loan losses	(5,650)
Other
Financing Receivable, Credit Quality Indicator [Line Items]
2020	27,672
2019	81,643
2018	142,748
2017	89,545
2016	238,104
Prior	187,707
Total	767,419
Allowance for loan losses	$ (14,937)