Loans Held for Investment - Student Loan Portfolio Average Balances (Details) - USD ($) $ in Thousands	12 Months Ended
	Dec. 31, 2016	Dec. 31, 2015	Dec. 31, 2014
Receivables [Abstract]
Average balance of Private Education Loans	$ 12,747,756	$ 9,819,053	$ 7,563,356
Average Balance, FFELP Loans	1,063,325	1,179,723	1,353,497
Average Balance, Total portfolio	$ 13,811,081	$ 10,998,776	$ 8,916,853
Weighted Average Interest Rate, Private Education Loans	8.02%	7.93%	8.16%
Weighted Average Interest Rate, FFELP Loans	3.53%	3.26%	3.24%