Loans, Loans Held For Sale, Notes and Certificates and Loan Servicing Rights - Fair Value of Loans Invested in by the Company (Details) - Loans Invested in by Company - USD ($) $ in Thousands	3 Months Ended		9 Months Ended
	Sep. 30, 2017	Sep. 30, 2016	Sep. 30, 2017	Sep. 30, 2016
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Outstanding Principal Balance, Beginning	$ 57,221	$ 35,672	$ 27,860	$ 3,462
Valuation Adjustment, Beginning	(2,371)	(1,126)	(1,949)	(4)
Fair Value, Beginning	54,850	34,546	25,911	3,458
Outstanding Principal Balance, Purchases	586,708	3,579	777,117	175,600
Valuation Adjustment, Purchases	(473)	0	(481)	0
Fair Value, Purchases	586,235	3,579	776,636	175,600
Outstanding Principal Balance, Sales	(429,481)	(6)	(576,065)	(135,531)
Valuation Adjustment, Sales	1,741	0	2,634	0
Fair Value, Sales	(427,740)	(6)	(573,431)	(135,531)
Outstanding Principal Balance, Principal payments and retirements	(18,720)	(4,352)	(30,703)	(8,534)
Valuation Adjustment, Principal payments	0	0	0	0
Fair Value, Principal payments and retirements	(18,720)	(4,352)	(30,703)	(8,534)
Outstanding Principal Balance, Charge-offs	(1,538)	(701)	(4,019)	(805)
Valuation Adjustment, Charge-offs	1,538	701	4,019	805
Fair Value, Charge-offs	0	0	0	0
Outstanding Principal Balance, Recoveries	0	0	0	0
Valuation Adjustment, Recoveries	(56)	(562)	(256)	(580)
Fair Value, Recoveries	(56)	(562)	(256)	(580)
Outstanding Principal Balance, Change in fair value recorded in earnings	0	0	0	0
Valuation Adjustment, Change in fair value recorded in earnings	(7,775)	(476)	(11,363)	(1,684)
Fair Value, Change in fair value recorded in earnings	(7,775)	(476)	(11,363)	(1,684)
Outstanding Principal Balance, Ending	194,190	34,192	194,190	34,192
Valuation Adjustment, Ending	(7,396)	(1,463)	(7,396)	(1,463)
Fair Value, Ending	$ 186,794	$ 32,729	$ 186,794	$ 32,729