Income Taxes - Schedule of Deferred Tax Assets and Liabilities (Details) - USD ($) $ in Thousands	Dec. 31, 2019	Dec. 31, 2018
Deferred tax assets
Net operating loss carryforwards - federal and foreign	$ 64,817	$ 31,587
Net operating loss carryforwards and credits - state and local	70,254	0
Reserve for servicing exposure	7,711	10,331
Accrued other liabilities	6,377	8,966
Foreign deferred assets	3,620	7,142
Partnership losses	7,029	6,681
Stock-based compensation expense	5,297	5,610
Interest expense disallowance	12,423	4,773
Intangible asset amortization	4,946	4,579
Accrued incentive compensation	5,063	4,527
Accrued legal settlements	6,028	4,350
Bad debt and allowance for loan losses	2,530	3,498
Tax residuals and deferred income on tax residuals	2,885	2,905
Foreign tax credit	94	357
Lease liabilities	5,459	580
Deferred income	8,493	0
Other	8,708	8,252
Deferred tax assets, gross	221,734	104,138
Deferred tax liabilities
Mortgage servicing rights amortization	16,358	27,860
Foreign undistributed earnings	1,615	2,059
Other	1,151	804
Deferred tax liabilities, gross	19,124	30,723
Deferred tax assets (liability), gross	202,610	73,415
Valuation allowance	(200,441)	(68,126)
Deferred tax assets, net	$ 2,169	$ 5,289