Commercial Mortgage Loans - Internal Credit Qualities (Details) - Commercial Portfolio Segment - Commercial Mortgage Receivable, Held-For-Investment $ in Thousands	Mar. 31, 2020 USD ($)
Financing Receivable, Credit Quality Indicator [Line Items]
2020	$ 251,595
2019	988,116
2018	1,136,945
2017	228,731
2016	0
2015	44,355
Prior	12,939
Total	2,662,681
Multifamily
Financing Receivable, Credit Quality Indicator [Line Items]
2020	177,191
2019	456,036
2018	683,338
2017	72,687
2016	0
2015	0
Prior	3,489
Total	1,392,741
Multifamily | 1-2 internal grade
Financing Receivable, Credit Quality Indicator [Line Items]
2020	177,191
2019	456,036
2018	617,533
2017	34,875
2016	0
2015	0
Prior	3,489
Total	1,289,124
Multifamily | 3-4 internal grade
Financing Receivable, Credit Quality Indicator [Line Items]
2020	0
2019	0
2018	65,805
2017	37,812
2016	0
2015	0
Prior	0
Total	103,617
Retail
Financing Receivable, Credit Quality Indicator [Line Items]
2020	0
2019	58,119
2018	60,226
2017	0
2016	0
2015	0
Prior	9,450
Total	127,795
Retail | 1-2 internal grade
Financing Receivable, Credit Quality Indicator [Line Items]
2020	0
2019	54,618
2018	16,319
2017	0
2016	0
2015	0
Prior	9,450
Total	80,387
Retail | 3-4 internal grade
Financing Receivable, Credit Quality Indicator [Line Items]
2020	0
2019	3,501
2018	43,907
2017	0
2016	0
2015	0
Prior	0
Total	47,408
Office
Financing Receivable, Credit Quality Indicator [Line Items]
2020	49,163
2019	173,988
2018	143,195
2017	52,151
2016	0
2015	10,700
Prior	0
Total	429,197
Office | 1-2 internal grade
Financing Receivable, Credit Quality Indicator [Line Items]
2020	49,163
2019	173,988
2018	97,936
2017	41,645
2016	0
2015	10,700
Prior	0
Total	373,432
Office | 3-4 internal grade
Financing Receivable, Credit Quality Indicator [Line Items]
2020	0
2019	0
2018	45,259
2017	10,506
2016	0
2015	0
Prior	0
Total	55,765
Industrial
Financing Receivable, Credit Quality Indicator [Line Items]
2020	25,241
2019	84,576
2018	0
2017	0
2016	0
2015	33,655
Prior	0
Total	143,472
Industrial | 1-2 internal grade
Financing Receivable, Credit Quality Indicator [Line Items]
2020	25,241
2019	84,576
2018	0
2017	0
2016	0
2015	33,655
Prior	0
Total	143,472
Industrial | 3-4 internal grade
Financing Receivable, Credit Quality Indicator [Line Items]
2020	0
2019	0
2018	0
2017	0
2016	0
2015	0
Prior	0
Total	0
Mixed Use
Financing Receivable, Credit Quality Indicator [Line Items]
2020	0
2019	0
2018	52,097
2017	12,953
2016	0
2015	0
Prior	0
Total	65,050
Mixed Use | 1-2 internal grade
Financing Receivable, Credit Quality Indicator [Line Items]
2020	0
2019	0
2018	52,097
2017	12,953
2016	0
2015	0
Prior	0
Total	65,050
Mixed Use | 3-4 internal grade
Financing Receivable, Credit Quality Indicator [Line Items]
2020	0
2019	0
2018	0
2017	0
2016	0
2015	0
Prior	0
Total	0
Industrial
Financing Receivable, Credit Quality Indicator [Line Items]
2020	0
2019	170,683
2018	135,152
2017	90,940
2016	0
2015	0
Prior	0
Total	396,775
Industrial | 1-2 internal grade
Financing Receivable, Credit Quality Indicator [Line Items]
2020	0
2019	8,735
2018	20,962
2017	0
2016	0
2015	0
Prior	0
Total	29,697
Industrial | 3-4 internal grade
Financing Receivable, Credit Quality Indicator [Line Items]
2020	0
2019	161,948
2018	114,190
2017	90,940
2016	0
2015	0
Prior	0
Total	367,078
Self Storage
Financing Receivable, Credit Quality Indicator [Line Items]
2020	0
2019	0
2018	62,937
2017	0
2016	0
2015	0
Prior	0
Total	62,937
Self Storage | 1-2 internal grade
Financing Receivable, Credit Quality Indicator [Line Items]
2020	0
2019	0
2018	62,937
2017	0
2016	0
2015	0
Prior	0
Total	62,937
Self Storage | 3-4 internal grade
Financing Receivable, Credit Quality Indicator [Line Items]
2020	0
2019	0
2018	0
2017	0
2016	0
2015	0
Prior	0
Total	0
Manufactured Housing
Financing Receivable, Credit Quality Indicator [Line Items]
2020	0
2019	44,714
2018	0
2017	0
2016	0
2015	0
Prior	0
Total	44,714
Manufactured Housing | 1-2 internal grade
Financing Receivable, Credit Quality Indicator [Line Items]
2020	0
2019	44,714
2018	0
2017	0
2016	0
2015	0
Prior	0
Total	44,714
Manufactured Housing | 3-4 internal grade
Financing Receivable, Credit Quality Indicator [Line Items]
2020	0
2019	0
2018	0
2017	0
2016	0
2015	0
Prior	0
Total	$ 0