Borrowings - Mortgage Loans Sold Under Agreement to Repurchase by Counterparty (Details) - Assets sold under agreements to repurchase $ in Thousands	Mar. 31, 2020 USD ($)
Credit Suisse First Boston Mortgage Capital LLC Tranche Two
Mortgage loans sold under agreement to repurchase
Amount at risk	$ 983,098
Credit Suisse First Boston Mortgage Capital LLC Tranche One
Mortgage loans sold under agreement to repurchase
Amount at risk	245,618
Bank of America, N.A.
Mortgage loans sold under agreement to repurchase
Amount at risk	61,809
JP Morgan
Mortgage loans sold under agreement to repurchase
Amount at risk	97,688
Citibank, N.A.
Mortgage loans sold under agreement to repurchase
Amount at risk	67,788
Morgan Stanley Bank
Mortgage loans sold under agreement to repurchase
Amount at risk	26,408
BNP Paribas
Mortgage loans sold under agreement to repurchase
Amount at risk	15,575
Royal Bank of Canada
Mortgage loans sold under agreement to repurchase
Amount at risk	$ 24,012