Loans Held for Investment - Student Loan Portfolio Average Balances (Details) - USD ($) $ in Thousands	3 Months Ended		9 Months Ended
	Sep. 30, 2016	Sep. 30, 2015	Sep. 30, 2016	Sep. 30, 2015
Receivables [Abstract]
Average Balance, Private Education Loans	$ 12,881,890	$ 9,869,025	$ 12,307,932	$ 9,563,290
Average Balance FFELP Loans	1,049,803	1,161,288	1,076,394	1,196,491
Average balance, Total portfolio	$ 13,931,693	$ 11,030,313	$ 13,384,326	$ 10,759,781
Weighted Average Interest Rate, Private Education Loans	8.00%	7.87%	8.00%	7.96%
Weighted Average Interest Rate FFELP loans	3.52%	3.27%	3.48%	3.22%