Payables and Accrued Liabilities - Schedule of Exit Liability by Segment (Details) $ in Thousands	6 Months Ended
	Jun. 30, 2017 USD ($)
Exit Liability [Roll Forward]
Balance at January 1, 2017	$ 12,866
Charges	7,969
Cash payments or other settlements	(12,625)
Balance at June 30, 2017	8,210
Total cumulative charges incurred	45,288
Total expected costs to be incurred (2)	48,973
Servicing
Exit Liability [Roll Forward]
Balance at January 1, 2017	4,776
Charges	5,684
Cash payments or other settlements	(4,626)
Balance at June 30, 2017	5,834
Total cumulative charges incurred	23,767
Total expected costs to be incurred (2)	27,309
Originations
Exit Liability [Roll Forward]
Balance at January 1, 2017	1,283
Charges	875
Cash payments or other settlements	(1,518)
Balance at June 30, 2017	640
Total cumulative charges incurred	6,601
Total expected costs to be incurred (2)	6,601
Reverse Mortgage
Exit Liability [Roll Forward]
Balance at January 1, 2017	2,497
Charges	1,292
Cash payments or other settlements	(2,233)
Balance at June 30, 2017	1,556
Total cumulative charges incurred	8,369
Total expected costs to be incurred (2)	8,451
Other
Exit Liability [Roll Forward]
Balance at January 1, 2017	4,310
Charges	118
Cash payments or other settlements	(4,248)
Balance at June 30, 2017	180
Total cumulative charges incurred	6,551
Total expected costs to be incurred (2)	6,612
2015 Actions
Exit Liability [Roll Forward]
Balance at January 1, 2017	988
Charges	(28)
Cash payments or other settlements	(491)
Balance at June 30, 2017	469
Total cumulative charges incurred	13,745
Total expected costs to be incurred (2)	13,745
2015 Actions | Servicing
Exit Liability [Roll Forward]
Balance at January 1, 2017	453
Charges	(57)
Cash payments or other settlements	(133)
Balance at June 30, 2017	263
Total cumulative charges incurred	6,424
Total expected costs to be incurred (2)	6,424
2015 Actions | Originations
Exit Liability [Roll Forward]
Balance at January 1, 2017	260
Charges	21
Cash payments or other settlements	(185)
Balance at June 30, 2017	96
Total cumulative charges incurred	4,611
Total expected costs to be incurred (2)	4,611
2015 Actions | Reverse Mortgage
Exit Liability [Roll Forward]
Balance at January 1, 2017	275
Charges	8
Cash payments or other settlements	(173)
Balance at June 30, 2017	110
Total cumulative charges incurred	1,859
Total expected costs to be incurred (2)	1,859
2015 Actions | Other
Exit Liability [Roll Forward]
Balance at January 1, 2017	0
Charges	0
Cash payments or other settlements	0
Balance at June 30, 2017	0
Total cumulative charges incurred	851
Total expected costs to be incurred (2)	851
2016 Actions
Exit Liability [Roll Forward]
Balance at January 1, 2017	11,878
Charges	168
Cash payments or other settlements	(10,898)
Balance at June 30, 2017	1,148
Total cumulative charges incurred	23,714
Total expected costs to be incurred (2)	23,738
2016 Actions | Servicing
Exit Liability [Roll Forward]
Balance at January 1, 2017	4,323
Charges	82
Cash payments or other settlements	(3,875)
Balance at June 30, 2017	530
Total cumulative charges incurred	11,684
Total expected costs to be incurred (2)	11,684
2016 Actions | Originations
Exit Liability [Roll Forward]
Balance at January 1, 2017	1,023
Charges	(126)
Cash payments or other settlements	(876)
Balance at June 30, 2017	21
Total cumulative charges incurred	1,010
Total expected costs to be incurred (2)	1,010
2016 Actions | Reverse Mortgage
Exit Liability [Roll Forward]
Balance at January 1, 2017	2,222
Charges	94
Cash payments or other settlements	(1,899)
Balance at June 30, 2017	417
Total cumulative charges incurred	5,320
Total expected costs to be incurred (2)	5,344
2016 Actions | Other
Exit Liability [Roll Forward]
Balance at January 1, 2017	4,310
Charges	118
Cash payments or other settlements	(4,248)
Balance at June 30, 2017	180
Total cumulative charges incurred	5,700
Total expected costs to be incurred (2)	5,700
2017 Actions
Exit Liability [Roll Forward]
Balance at January 1, 2017	0
Charges	7,829
Cash payments or other settlements	(1,236)
Balance at June 30, 2017	6,593
Total cumulative charges incurred	7,829
Total expected costs to be incurred (2)	11,490
2017 Actions | Servicing
Exit Liability [Roll Forward]
Balance at January 1, 2017	0
Charges	5,659
Cash payments or other settlements	(618)
Balance at June 30, 2017	5,041
Total cumulative charges incurred	5,659
Total expected costs to be incurred (2)	9,201
2017 Actions | Originations
Exit Liability [Roll Forward]
Balance at January 1, 2017	0
Charges	980
Cash payments or other settlements	(457)
Balance at June 30, 2017	523
Total cumulative charges incurred	980
Total expected costs to be incurred (2)	980
2017 Actions | Reverse Mortgage
Exit Liability [Roll Forward]
Balance at January 1, 2017	0
Charges	1,190
Cash payments or other settlements	(161)
Balance at June 30, 2017	1,029
Total cumulative charges incurred	1,190
Total expected costs to be incurred (2)	1,248
2017 Actions | Other
Exit Liability [Roll Forward]
Balance at January 1, 2017	0
Charges	0
Cash payments or other settlements	0
Balance at June 30, 2017	0
Total cumulative charges incurred	0
Total expected costs to be incurred (2)	$ 61