Commitments, Contingencies and Indemnifications (Details) - USD ($) $ in Thousands	Mar. 31, 2020	Dec. 31, 2019
Originated Residential Agency loans
Commitments, contingencies and indemnifications
Commitments to originate residential agency loans	$ 601,633	$ 190,806
Unfunded loan commitments
Commitments, contingencies and indemnifications
Loans, net	329,809	128,719
Loans, held for sale at fair value	$ 8,855	$ 6,982