Secured Borrowings - Collateral Pledged (Details) - USD ($) $ in Thousands	Sep. 30, 2019	Dec. 31, 2018
Collateral pledged
Pledged Assets Carrying Value	$ 1,707,541	$ 1,112,522
Secured borrowings
Collateral pledged
Pledged Assets Carrying Value	1,707,541	1,112,522
Borrowings under credit facilities
Collateral pledged
Pledged Assets Carrying Value	703,953	279,012
Borrowings under repurchase agreements
Collateral pledged
Pledged Assets Carrying Value	1,003,588	833,510
Loans, net | Borrowings under credit facilities
Collateral pledged
Pledged Assets Carrying Value	627,490	215,533
Loans, net | Borrowings under repurchase agreements
Collateral pledged
Pledged Assets Carrying Value	710,801	685,274
Mortgage servicing rights | Borrowings under credit facilities
Collateral pledged
Pledged Assets Carrying Value	56,513	63,479
Real estate | Borrowings under credit facilities
Collateral pledged
Pledged Assets Carrying Value	19,950
MBS | Borrowings under repurchase agreements
Collateral pledged
Pledged Assets Carrying Value	92,189	112,552
Retained interest in assets of consolidated VIEs | Borrowings under repurchase agreements
Collateral pledged
Pledged Assets Carrying Value	$ 200,598	$ 35,684