Basis of Presentation and Significant Accounting Policies (Details) $ in Thousands	9 Months Ended
	Sep. 30, 2020 USD ($) segment	Jun. 30, 2020 USD ($)	Mar. 31, 2020 USD ($)	Jan. 01, 2020 USD ($)	Dec. 31, 2019 USD ($)	Sep. 30, 2019 USD ($)	Jun. 30, 2019 USD ($)	Mar. 31, 2019 USD ($)	Dec. 31, 2018 USD ($)
Schedule of Equity Method Investments [Line Items]
Number of segments | segment	2
Accounting Standards Update [Extensible List]	us-gaap:AccountingStandardsUpdate201609Member
Decrease in retained earnings due to cumulative effect of new accounting principle	$ (214,616)	$ (206,923)	$ (197,723)		$ (207,117)	$ (204,039)	$ (206,338)	$ (208,838)	$ (205,947)
Residential Mortgage Holding Company
Schedule of Equity Method Investments [Line Items]
Ownership percentage	100.00%
Retained Earnings
Schedule of Equity Method Investments [Line Items]
Decrease in retained earnings due to cumulative effect of new accounting principle	$ (165,606)	$ (155,998)	$ (148,286)		(149,615)	$ (147,217)	$ (141,878)	$ (139,677)	$ (137,452)
Cumulative effect of adoption of accounting principles related to equity compensation expense
Schedule of Equity Method Investments [Line Items]
Decrease in retained earnings due to cumulative effect of new accounting principle					0
Cumulative effect of adoption of accounting principles related to equity compensation expense |  Retained Earnings
Schedule of Equity Method Investments [Line Items]
Decrease in retained earnings due to cumulative effect of new accounting principle				$ 139	$ 139