Borrowings - Mortgage Loans Sold Under Agreement to Repurchase by Counterparty (Details) - Assets sold under agreements to repurchase $ in Thousands	Jun. 30, 2020 USD ($)
Credit Suisse First Boston Mortgage Capital LLC Tranche Two
Mortgage loans sold under agreement to repurchase
Amount at risk	$ 1,032,963
Credit Suisse First Boston Mortgage Capital LLC Tranche One
Mortgage loans sold under agreement to repurchase
Amount at risk	352,900
Bank of America, N.A.
Mortgage loans sold under agreement to repurchase
Amount at risk	120,991
JP Morgan
Mortgage loans sold under agreement to repurchase
Amount at risk	9,812
Citibank, N.A.
Mortgage loans sold under agreement to repurchase
Amount at risk	27,323
Morgan Stanley Bank
Mortgage loans sold under agreement to repurchase
Amount at risk	34,949
BNP Paribas
Mortgage loans sold under agreement to repurchase
Amount at risk	14,888
Royal Bank of Canada
Mortgage loans sold under agreement to repurchase
Amount at risk	$ 35,008