Borrowings - Mortgage Loans Sold Under Agreement to Repurchase by Counterparty (Details) - Assets sold under agreements to repurchase $ in Thousands	Dec. 31, 2018 USD ($)
Credit Suisse First Boston Mortgage Capital LLC Tranche Two
Mortgage loans sold under agreement to repurchase
Amount at risk	$ 1,416,794
Credit Suisse First Boston Mortgage Capital LLC Tranche One
Mortgage loans sold under agreement to repurchase
Amount at risk	33,906
Deutsche Bank AG
Mortgage loans sold under agreement to repurchase
Amount at risk	53,901
Bank of America, N.A.
Mortgage loans sold under agreement to repurchase
Amount at risk	15,863
Morgan Stanley Bank
Mortgage loans sold under agreement to repurchase
Amount at risk	5,825
JP Morgan
Mortgage loans sold under agreement to repurchase
Amount at risk	5,286
BNP Paribas
Mortgage loans sold under agreement to repurchase
Amount at risk	9,222
Royal Bank of Canada
Mortgage loans sold under agreement to repurchase
Amount at risk	2,129
Citibank, N.A.
Mortgage loans sold under agreement to repurchase
Amount at risk	$ 586