Commercial Mortgage Loans - Commercial Mortgage Loan Portfolio, Excluding Fair Value (Details) - USD ($) $ in Thousands	3 Months Ended	12 Months Ended
	Mar. 31, 2020	Dec. 31, 2019
Commercial Mortgage Receivable, Held-For-Investment
Accounts, Notes, Loans and Financing Receivable [Line Items]
Par Value	$ 2,670,133	$ 2,771,299
Percentage	100.00%	100.00%
Commercial Mortgage Receivable, Held-For-Investment | Multifamily
Accounts, Notes, Loans and Financing Receivable [Line Items]
Par Value	$ 1,395,956	$ 1,491,971
Percentage	52.30%	53.90%
Commercial Mortgage Receivable, Held-For-Investment | Office
Accounts, Notes, Loans and Financing Receivable [Line Items]
Par Value	$ 430,870	$ 414,772
Percentage	16.10%	15.00%
Commercial Mortgage Receivable, Held-For-Investment | Hospitality
Accounts, Notes, Loans and Financing Receivable [Line Items]
Par Value	$ 398,186	$ 446,562
Percentage	14.90%	16.10%
Commercial Mortgage Receivable, Held-For-Investment | Industrial
Accounts, Notes, Loans and Financing Receivable [Line Items]
Par Value	$ 144,093	$ 118,743
Percentage	5.40%	4.30%
Commercial Mortgage Receivable, Held-For-Investment | Retail
Accounts, Notes, Loans and Financing Receivable [Line Items]
Par Value	$ 111,620	$ 111,620
Percentage	4.20%	4.00%
Commercial Mortgage Receivable, Held-For-Investment | Mixed Use
Accounts, Notes, Loans and Financing Receivable [Line Items]
Par Value	$ 65,040	$ 58,808
Percentage	2.40%	2.10%
Commercial Mortgage Receivable, Held-For-Investment | Self Storage
Accounts, Notes, Loans and Financing Receivable [Line Items]
Par Value	$ 63,057	$ 67,767
Percentage	2.40%	2.40%
Commercial Mortgage Receivable, Held-For-Investment | Land
Accounts, Notes, Loans and Financing Receivable [Line Items]
Par Value	$ 16,400	$ 16,400
Percentage	0.60%	0.60%
Commercial Mortgage Receivable, Held-For-Investment | Manufactured Housing
Accounts, Notes, Loans and Financing Receivable [Line Items]
Par Value	$ 44,911	$ 44,656
Percentage	1.70%	1.60%
Commercial Mortgage Receivable, Held-For-Sale
Accounts, Notes, Loans and Financing Receivable [Line Items]
Par Value	$ 93,668	$ 112,488
Percentage	100.00%	100.00%
Commercial Mortgage Receivable, Held-For-Sale | Multifamily
Accounts, Notes, Loans and Financing Receivable [Line Items]
Par Value	$ 41,000	$ 78,250
Percentage	43.80%	69.60%
Commercial Mortgage Receivable, Held-For-Sale | Office
Accounts, Notes, Loans and Financing Receivable [Line Items]
Par Value	$ 6,000	$ 0
Percentage	6.40%	0.00%
Commercial Mortgage Receivable, Held-For-Sale | Hospitality
Accounts, Notes, Loans and Financing Receivable [Line Items]
Par Value	$ 0	$ 8,000
Percentage	0.00%	7.10%
Commercial Mortgage Receivable, Held-For-Sale | Industrial
Accounts, Notes, Loans and Financing Receivable [Line Items]
Par Value	$ 23,625	$ 23,625
Percentage	25.20%	21.00%
Commercial Mortgage Receivable, Held-For-Sale | Retail
Accounts, Notes, Loans and Financing Receivable [Line Items]
Par Value	$ 21,193	$ 2,613
Percentage	22.60%	2.30%
Commercial Mortgage Receivable, Held-For-Sale | Manufactured Housing
Accounts, Notes, Loans and Financing Receivable [Line Items]
Par Value	$ 1,850	$ 0
Percentage	2.00%	0.00%