Borrowings - Mortgage Loans Sold Under Agreement to Repurchase by Counterparty (Details) - Assets sold under agreements to repurchase $ in Thousands	Dec. 31, 2019 USD ($)
Credit Suisse First Boston Mortgage Capital LLC Tranche Two
Mortgage loans sold under agreement to repurchase
Amount at risk	$ 1,709,197
Credit Suisse First Boston Mortgage Capital LLC Tranche One
Mortgage loans sold under agreement to repurchase
Amount at risk	72,865
Bank of America, N.A.
Mortgage loans sold under agreement to repurchase
Amount at risk	29,252
JP Morgan
Mortgage loans sold under agreement to repurchase
Amount at risk	61,561
Citibank, N.A.
Mortgage loans sold under agreement to repurchase
Amount at risk	48,017
Morgan Stanley Bank
Mortgage loans sold under agreement to repurchase
Amount at risk	42,181
BNP Paribas
Mortgage loans sold under agreement to repurchase
Amount at risk	10,233
Royal Bank of Canada
Mortgage loans sold under agreement to repurchase
Amount at risk	$ 13,811