Commitments, Contingencies and Indemnifications (Details) - USD ($) $ in Thousands	Jun. 30, 2020	Dec. 31, 2019
Originated Residential Agency loans
Commitments, contingencies and indemnifications
Commitments to originate residential agency loans	$ 581,478	$ 190,806
Unfunded loan commitments
Commitments, contingencies and indemnifications
Loans, net	316,983	128,719
Loans, held for sale at fair value	$ 7,784	$ 6,982