Loans Held for Investment - Student Loan Portfolio Average Balances (Details) - USD ($) $ in Thousands	3 Months Ended
	Mar. 31, 2017	Mar. 31, 2016
Receivables [Abstract]
Average Balance, Private Education Loans	$ 15,449,555	$ 11,817,708
Average Balance FFELP Loans	1,003,128	1,103,253
Average Balance of Personal Loans	35,830	0
Average balance, Total portfolio	$ 16,488,513	$ 12,920,961
Weighted Average Interest Rate, Private Education Loans	8.26%	8.03%
Weighted Average Interest Rate FFELP loans	3.69%	3.42%
Weighted Average Interest Rate of Personal Loans	9.16%