Loans Held for Investment - Student Loan Portfolio Average Balances (Details) - USD ($) $ in Thousands	3 Months Ended		6 Months Ended
	Jun. 30, 2016	Jun. 30, 2015	Jun. 30, 2016	Jun. 30, 2015
Receivables [Abstract]
Average Balance, Private Education Loans	$ 12,217,890	$ 9,361,711	$ 12,017,799	$ 9,407,888
Average Balance FFELP Loans	1,076,419	1,194,309	1,089,836	1,214,384
Average balance, Total portfolio	$ 13,294,309	$ 10,556,020	$ 13,107,635	$ 10,622,272
Weighted Average Interest Rate, Private Education Loans	7.98%	7.96%	8.00%	8.01%
Weighted Average Interest Rate FFELP loans	3.48%	3.22%	3.45%	3.20%