Commercial Mortgage Loans - Allowance For Credit Losses (Details) - USD ($) $ in Thousands	3 Months Ended
	Mar. 31, 2020	Mar. 31, 2019
Financing Receivable, Allowance for Credit Loss [Roll Forward]
Increase/(decrease) for credit losses	$ 14,597	$ 2,495
Commercial Portfolio Segment
Financing Receivable, Allowance for Credit Loss [Roll Forward]
Beginning balance	921
Increase/(decrease) for credit losses	13,997
Write-offs	(427)
Ending balance	21,702
Commercial Portfolio Segment | Unfunded loan commitment
Financing Receivable, Allowance for Credit Loss [Roll Forward]
Beginning balance	0
Increase/(decrease) for credit losses	600
Write-offs	0
Ending balance	1,150
Transition Adjustment | Commercial Portfolio Segment
Financing Receivable, Allowance for Credit Loss [Roll Forward]
Beginning balance	7,211
Transition Adjustment | Commercial Portfolio Segment | Unfunded loan commitment
Financing Receivable, Allowance for Credit Loss [Roll Forward]
Beginning balance	550
Multifamily | Commercial Portfolio Segment
Financing Receivable, Allowance for Credit Loss [Roll Forward]
Beginning balance	322
Increase/(decrease) for credit losses	8,647
Write-offs	0
Ending balance	12,189
Multifamily | Commercial Portfolio Segment | Unfunded loan commitment
Financing Receivable, Allowance for Credit Loss [Roll Forward]
Beginning balance	0
Increase/(decrease) for credit losses	56
Write-offs	0
Ending balance	295
Multifamily | Transition Adjustment | Commercial Portfolio Segment
Financing Receivable, Allowance for Credit Loss [Roll Forward]
Beginning balance	3,220
Multifamily | Transition Adjustment | Commercial Portfolio Segment | Unfunded loan commitment
Financing Receivable, Allowance for Credit Loss [Roll Forward]
Beginning balance	239
Retail | Commercial Portfolio Segment
Financing Receivable, Allowance for Credit Loss [Roll Forward]
Beginning balance	202
Increase/(decrease) for credit losses	565
Write-offs	0
Ending balance	1,153
Retail | Commercial Portfolio Segment | Unfunded loan commitment
Financing Receivable, Allowance for Credit Loss [Roll Forward]
Beginning balance	0
Increase/(decrease) for credit losses	(40)
Write-offs	0
Ending balance	0
Retail | Transition Adjustment | Commercial Portfolio Segment
Financing Receivable, Allowance for Credit Loss [Roll Forward]
Beginning balance	386
Retail | Transition Adjustment | Commercial Portfolio Segment | Unfunded loan commitment
Financing Receivable, Allowance for Credit Loss [Roll Forward]
Beginning balance	40
Office | Commercial Portfolio Segment
Financing Receivable, Allowance for Credit Loss [Roll Forward]
Beginning balance	249
Increase/(decrease) for credit losses	2,358
Write-offs	0
Ending balance	4,573
Office | Commercial Portfolio Segment | Unfunded loan commitment
Financing Receivable, Allowance for Credit Loss [Roll Forward]
Beginning balance	0
Increase/(decrease) for credit losses	242
Write-offs	0
Ending balance	392
Office | Transition Adjustment | Commercial Portfolio Segment
Financing Receivable, Allowance for Credit Loss [Roll Forward]
Beginning balance	1,966
Office | Transition Adjustment | Commercial Portfolio Segment | Unfunded loan commitment
Financing Receivable, Allowance for Credit Loss [Roll Forward]
Beginning balance	150
Industrial | Commercial Portfolio Segment
Financing Receivable, Allowance for Credit Loss [Roll Forward]
Beginning balance	23
Increase/(decrease) for credit losses	1,875
Write-offs	0
Ending balance	2,332
Industrial | Commercial Portfolio Segment | Unfunded loan commitment
Financing Receivable, Allowance for Credit Loss [Roll Forward]
Beginning balance	0
Increase/(decrease) for credit losses	257
Write-offs	0
Ending balance	287
Industrial | Transition Adjustment | Commercial Portfolio Segment
Financing Receivable, Allowance for Credit Loss [Roll Forward]
Beginning balance	434
Industrial | Transition Adjustment | Commercial Portfolio Segment | Unfunded loan commitment
Financing Receivable, Allowance for Credit Loss [Roll Forward]
Beginning balance	30
Mixed Use | Commercial Portfolio Segment
Financing Receivable, Allowance for Credit Loss [Roll Forward]
Beginning balance	4
Increase/(decrease) for credit losses	28
Write-offs	0
Ending balance	41
Mixed Use | Commercial Portfolio Segment | Unfunded loan commitment
Financing Receivable, Allowance for Credit Loss [Roll Forward]
Beginning balance	0
Increase/(decrease) for credit losses	(1)
Write-offs	0
Ending balance	0
Mixed Use | Transition Adjustment | Commercial Portfolio Segment
Financing Receivable, Allowance for Credit Loss [Roll Forward]
Beginning balance	9
Mixed Use | Transition Adjustment | Commercial Portfolio Segment | Unfunded loan commitment
Financing Receivable, Allowance for Credit Loss [Roll Forward]
Beginning balance	1
Industrial | Commercial Portfolio Segment
Financing Receivable, Allowance for Credit Loss [Roll Forward]
Beginning balance	103
Increase/(decrease) for credit losses	655
Write-offs	(427)
Ending balance	1,070
Industrial | Commercial Portfolio Segment | Unfunded loan commitment
Financing Receivable, Allowance for Credit Loss [Roll Forward]
Beginning balance	0
Increase/(decrease) for credit losses	117
Write-offs	0
Ending balance	174
Industrial | Transition Adjustment | Commercial Portfolio Segment
Financing Receivable, Allowance for Credit Loss [Roll Forward]
Beginning balance	739
Industrial | Transition Adjustment | Commercial Portfolio Segment | Unfunded loan commitment
Financing Receivable, Allowance for Credit Loss [Roll Forward]
Beginning balance	57
Self Storage | Commercial Portfolio Segment
Financing Receivable, Allowance for Credit Loss [Roll Forward]
Beginning balance	0
Increase/(decrease) for credit losses	(123)
Write-offs	0
Ending balance	276
Self Storage | Commercial Portfolio Segment | Unfunded loan commitment
Financing Receivable, Allowance for Credit Loss [Roll Forward]
Beginning balance	0
Increase/(decrease) for credit losses	(28)
Write-offs	0
Ending balance	0
Self Storage | Transition Adjustment | Commercial Portfolio Segment
Financing Receivable, Allowance for Credit Loss [Roll Forward]
Beginning balance	399
Self Storage | Transition Adjustment | Commercial Portfolio Segment | Unfunded loan commitment
Financing Receivable, Allowance for Credit Loss [Roll Forward]
Beginning balance	28
Manufactured Housing | Commercial Portfolio Segment
Financing Receivable, Allowance for Credit Loss [Roll Forward]
Beginning balance	18
Increase/(decrease) for credit losses	(8)
Write-offs	0
Ending balance	68
Manufactured Housing | Commercial Portfolio Segment | Unfunded loan commitment
Financing Receivable, Allowance for Credit Loss [Roll Forward]
Beginning balance	0
Increase/(decrease) for credit losses	(3)
Write-offs	0
Ending balance	2
Manufactured Housing | Transition Adjustment | Commercial Portfolio Segment
Financing Receivable, Allowance for Credit Loss [Roll Forward]
Beginning balance	58
Manufactured Housing | Transition Adjustment | Commercial Portfolio Segment | Unfunded loan commitment
Financing Receivable, Allowance for Credit Loss [Roll Forward]
Beginning balance	$ 5