Residential Mortgage Securities and MSR-Related Assets (MSR Related Assets - Narrative) (Details) - USD ($) $ in Thousands	3 Months Ended	6 Months Ended
	Jun. 30, 2019	Jun. 30, 2019	Jun. 30, 2018	Dec. 31, 2018	Mar. 31, 2018	Dec. 31, 2016
Term notes backed by MSR-related collateral
Debt Securities, Available-for-sale [Line Items]
Term notes backed by MSR related collateral	$ 1,106,026	$ 1,106,026		$ 538,499
Amortized costs	1,100,000	1,100,000		538,500
Gross unrealized gains	3,400	$ 3,400		$ 7
Weighted average yield		5.17%	5.32%
Weighted average to maturity		3 years 9 months 18 days	4 years 8 months 12 days
Corporate Loan
Debt Securities, Available-for-sale [Line Items]
Commitment to lend	100,000	$ 100,000				$ 130,000
Amount drawn	$ 63,800	$ 63,800			$ 124,200
Interest coupon rate	5.77%	5.77%
Debt term		1 year 2 months 12 days
Commitment period on undrawn amount	1 year 2 months 12 days
Minimum | Corporate Loan
Debt Securities, Available-for-sale [Line Items]
Commitment fee on undrawn amount	0.25%	0.25%
Maximum | Corporate Loan
Debt Securities, Available-for-sale [Line Items]
Commitment fee on undrawn amount	1.00%	1.00%