Segment Reporting - Statement of Operations Items (Details) $ in Thousands	3 Months Ended			9 Months Ended
	Sep. 30, 2018 USD ($)	Jun. 30, 2018 USD ($)	Sep. 30, 2017 USD ($)	Sep. 30, 2018 USD ($) item	Sep. 30, 2017 USD ($)
Segment Reporting
Number of reportable segments | item				3
Gain on sale of loans, net	$ 13,673		$ 42,476	$ 53,896	$ 116,602
Real estate services fees, net	711		1,355	3,134	4,492
Servicing fees, net	10,124		8,492	29,445	23,575
Gain (loss) on mortgage servicing rights, net	(5,192)		(10,513)	2,682	(18,159)
Other revenue	71		266	278	541
Accretion of contingent consideration			(396)		(1,948)
Change in fair value of contingent consideration			4,798		11,052
Loss on extinguishment of debt					(1,265)
Intangible asset impairment	(4,897)			(18,347)
Goodwill impairment	(29,925)	$ (74,700)		(104,587)
Other operating expense	(28,309)		(41,962)	(100,581)	(124,942)
Other income (expense)	(1,689)		(95)	(562)	7,011
(Loss) earnings before income taxes	(45,433)		4,421	(134,642)	16,959
Income tax expense	12		2,104	4,328	3,575
Net (loss) earnings	(45,445)		2,317	(138,970)	13,384
Corporate and Other.
Segment Reporting
Other revenue	(124)		185	(103)	313
Other operating expense	(6,339)		(4,435)	(16,673)	(12,176)
Other income (expense)	(459)		(480)	(1,343)	(1,758)
(Loss) earnings before income taxes	(6,922)		(4,730)	(18,119)	(13,621)
Mortgage Lending | Operating segments
Segment Reporting
Gain on sale of loans, net	13,673		42,476	53,896	116,602
Servicing fees, net	10,124		8,492	29,445	23,575
Gain (loss) on mortgage servicing rights, net	(5,192)		(10,513)	2,682	(18,159)
Other revenue					19
Accretion of contingent consideration			(396)		(1,948)
Change in fair value of contingent consideration			4,798		11,052
Intangible asset impairment	(4,897)			(18,347)
Goodwill impairment	(29,925)			(104,587)
Other operating expense	(21,452)		(36,663)	(81,985)	(109,979)
Other income (expense)	133		1,107	771	2,096
(Loss) earnings before income taxes	(37,536)		9,301	(118,125)	23,258
Real Estate Services | Operating segments
Segment Reporting
Real estate services fees, net	711		1,355	3,134	4,492
Other operating expense	(372)		(797)	(1,601)	(2,534)
(Loss) earnings before income taxes	339		558	1,533	1,958
Long-term Portfolio | Operating segments
Segment Reporting
Other revenue	195		81	381	209
Loss on extinguishment of debt					(1,265)
Other operating expense	(146)		(67)	(322)	(253)
Other income (expense)	(1,363)		(722)	10	6,673
(Loss) earnings before income taxes	$ (1,314)		$ (708)	$ 69	$ 5,364