Investment Securities - HTM (Details) - USD ($) $ in Thousands	3 Months Ended
	Mar. 31, 2020	Mar. 31, 2020	Jan. 01, 2020	Dec. 31, 2019
HTM Securities
Amortized Cost Basis		$ 570,888		$ 570,638
Credit Loss Allowance	$ (5,037)	(5,037)
Net Carrying Amount		565,851		570,638
Gross Unrealized Holdings Gains		411		2,094
Gross Unrealized Holdings Losses		(25,994)		(4,005)
Fair Value		540,268		568,727
Activity in credit loss allowance for HTM debt securities
Credit loss provision, net	907
Credit loss allowance at ending	5,037
HTM preferred equity interests
Less than one year		283,716
One to three years		238,011
Three to five years		5,295
Thereafter		38,829
Total		565,851		570,638
CMBS
HTM Securities
Amortized Cost Basis		375,873		383,473
Net Carrying Amount		375,873		383,473
Gross Unrealized Holdings Gains		104		946
Gross Unrealized Holdings Losses		(17,975)		(3,001)
Fair Value		358,002		381,418
HTM preferred equity interests
Less than one year		278,404
One to three years		97,469
Total		375,873		383,473
Preferred interests
HTM Securities
Amortized Cost Basis		147,855		142,012
Credit Loss Allowance	(2,018)	(2,018)
Net Carrying Amount		145,837		142,012
Gross Unrealized Holdings Gains				1,148
Gross Unrealized Holdings Losses		(7,977)		(353)
Fair Value		137,860		142,807
Activity in credit loss allowance for HTM debt securities
Credit loss provision, net	904
Credit loss allowance at ending	2,018
HTM preferred equity interests
One to three years		140,542
Three to five years		5,295
Total		145,837		142,012
Infrastructure bonds
HTM Securities
Amortized Cost Basis		47,160		45,153
Credit Loss Allowance	(3,019)	(3,019)
Net Carrying Amount		44,141		45,153
Gross Unrealized Holdings Gains		307
Gross Unrealized Holdings Losses		(42)		(651)
Fair Value		44,406		44,502
Activity in credit loss allowance for HTM debt securities
Credit loss provision, net	3
Credit loss allowance at ending	$ 3,019
HTM preferred equity interests
Less than one year		5,312
Thereafter		38,829
Total		$ 44,141		$ 45,153
ASU 2016-13
Activity in credit loss allowance for HTM debt securities
Accumulated deficit charge			$ 1,293
Gross-up of PCD bond amortized cost basis			2,837
ASU 2016-13 | Preferred interests
Activity in credit loss allowance for HTM debt securities
Accumulated deficit charge			1,114
ASU 2016-13 | Infrastructure bonds
Activity in credit loss allowance for HTM debt securities
Accumulated deficit charge			179
Gross-up of PCD bond amortized cost basis			$ 2,837