Mortgage Servicing Rights - Income from Mortgage Servicing Rights, Net (Details) - USD ($) $ in Thousands	12 Months Ended
	Dec. 31, 2016	Dec. 31, 2015	Dec. 31, 2014
Transfers and Servicing [Abstract]
Income	$ 41,152	$ 38,964	$ 19,362
Cost of sub-servicer	(6,281)	(5,079)	(1,834)
Net servicing income	34,871	33,885	17,528
Market valuation changes of MSRs	(36,372)	(24,392)	(21,081)
Market valuation changes of associated derivatives	15,584	(12,708)	0
MSR reversal of (provision for) repurchases	270	(707)	(708)
MSR Income (Loss), Net	$ 14,353	$ (3,922)	$ (4,261)