Loans Held For Investment, Loans Held For Sale, Notes, Certificates and Secured Borrowings and Loan Servicing Rights - Past Due Table (Details) $ in Thousands	Jun. 30, 2018 USD ($) Loan	Dec. 31, 2017 USD ($) Loan
Loans and Loans Held for Sale
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Outstanding principal balance, 90 days or more past due	$ 23,622	$ 36,588
Outstanding principal balance, non-accrual loans		3,289
Net fair value adjustments, 90 days or more past due	(19,368)	(30,071)
Net fair value adjustments, non-accrual loans		(2,675)
Fair value, past 90 days due	$ 4,254	6,517
Fair value, non-accrual loans		$ 614
Number of loans, 90 days past due | Loan	2,445	3,779
Number of loans, non-accrual loans | Loan		591
Loans Invested in by Company
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Outstanding principal balance, 90 days or more past due	$ 1,355	$ 1,015
Outstanding principal balance, non-accrual loans		122
Net fair value adjustments, 90 days or more past due	(1,151)	(861)
Net fair value adjustments, non-accrual loans		(107)
Fair value, past 90 days due	$ 204	154
Fair value, non-accrual loans		$ 15
Number of loans, 90 days past due | Loan	236	257
Number of loans, non-accrual loans | Loan		34