Other Investments - Mortgage Servicing Rights (Details) - USD ($) $ in Thousands	12 Months Ended
	Dec. 31, 2019	Dec. 31, 2018	Dec. 31, 2017
Servicing Asset at Fair Value, Amount [Roll Forward]
Additions	$ 868	$ 328	$ 7,387
Mortgage servicing rights
Servicing Asset at Fair Value, Amount [Roll Forward]
Balance at beginning of period	60,281	63,598	118,526
Additions	868	328	8,026
Sales	0	(1,077)	(52,788)
Changes in fair value due to: Changes in assumptions	(10,659)	4,434	(1,088)
Changes in fair value due to: Other changes	(8,266)	(7,002)	(9,078)
Balance at End of Period	42,224	60,281	63,598
Servicing income	15,038	15,372	21,120
Cost of sub-servicer	(1,465)	(1,444)	(2,828)
Net servicing fee income	13,573	13,928	18,292
Market valuation changes of MSRs	(18,856)	(2,508)	(10,166)
Market valuation changes of associated derivatives	8,596	(4,734)	(568)
MSR reversal of provision for repurchases	208	390	302
MSR Income (Loss), Net	$ 3,521	$ 7,076	$ 7,860