Residential Loans - Geographic Concentration of Residential Loans Recorded on Consolidated Balance Sheet (Footnotes) (Details) - Other states (none greater than 5%)	Dec. 31, 2016	Dec. 31, 2015
Residential Loans Held For Sale
Concentration Risk [Line Items]
Concentration risk percentage per other state	5.00%	5.00%
Residential loans, held-for-investment
Concentration Risk [Line Items]
Concentration risk percentage per other state	5.00%	5.00%
Residential Loans Held For Investment at Fair Value
Concentration Risk [Line Items]
Concentration risk percentage per other state	5.00%	5.00%