Commitments, Contingencies and Indemnifications (Details) - USD ($) $ in Thousands	Dec. 31, 2020	Dec. 31, 2019
Originated Residential Agency loans
Commitments, contingencies and indemnifications
Commitments to originate residential agency loans	$ 575,600	$ 190,806
Unfunded loan commitments
Commitments, contingencies and indemnifications
Loans, net	285,389	128,719
Loans, held for sale at fair value	$ 7,809	$ 6,982