Basis of Presentation and Significant Accounting Policies - Narrative (Details) $ in Thousands		3 Months Ended		9 Months Ended		19 Months Ended	22 Months Ended
	Dec. 01, 2014 USD ($) owner	Sep. 30, 2016 USD ($)	Sep. 30, 2015 USD ($)	Sep. 30, 2016 USD ($) Segment	Sep. 30, 2015 USD ($)	Jun. 30, 2016 USD ($)	Sep. 30, 2016 USD ($)	Dec. 31, 2015 USD ($)
Schedule of Equity Method Investments [Line Items]
Number of segments | Segment				2
Error Corrections and Prior Period Adjustments Restatement [Line Items]
Goodwill		$ 15,017		$ 15,017			$ 15,017	$ 22,334
Reduction in net income		$ (3,283)	$ (5,532)	$ (11,295)	$ (14,107)
Residential Mortgage Holding Company
Schedule of Equity Method Investments [Line Items]
Ownership percentage		100.00%		100.00%			100.00%
NBG
Schedule of Equity Method Investments [Line Items]
Ownership percentage		50.10%		50.10%			50.10%
Residential Mortgage Holding Company
Error Corrections and Prior Period Adjustments Restatement [Line Items]
Number of former equity owners | owner	4
Voting interests acquired	76.50%
Number of former equity owners with continuing employment contracts | owner	3
Number of former equity owners that must continue employment under terms of acquisition agreement | owner	2
Accrued compensation expense related to acquisition payments		$ 3,300
Residential Mortgage Holding Company | Accounting for Payments Made to Former Equity Owners
Error Corrections and Prior Period Adjustments Restatement [Line Items]
Total purchase price	$ 22,200
Goodwill	7,500
Residential Mortgage Holding Company | Previously Reported | Accounting for Payments Made to Former Equity Owners
Error Corrections and Prior Period Adjustments Restatement [Line Items]
Total purchase price	29,500
Goodwill	$ 14,800
Residential Mortgage Holding Company | Restatement Adjustment | Accounting for Payments Made to Former Equity Owners
Error Corrections and Prior Period Adjustments Restatement [Line Items]
Reduction in net income		$ 213				$ 1,400	$ 1,600