Loans Held For Investment, Loans Held For Sale, Notes, Certificates and Secured Borrowings and Loan Servicing Rights - Past Due Table (Details) $ in Thousands	Sep. 30, 2018 USD ($) Loan	Dec. 31, 2017 USD ($) Loan
Loans and Loans Held for Sale
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Outstanding principal balance, 90 days or more past due	$ 23,217	$ 36,588
Outstanding principal balance, non-accrual loans		3,289
Net fair value adjustments, 90 days or more past due	(19,015)	(30,071)
Net fair value adjustments, non-accrual loans		(2,675)
Fair value, past 90 days due	$ 4,202	6,517
Fair value, non-accrual loans		$ 614
Number of loans, 90 days past due | Loan	2,572	3,779
Number of loans, non-accrual loans | Loan		591
Loans Invested in by Company
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Outstanding principal balance, 90 days or more past due	$ 2,451	$ 1,015
Outstanding principal balance, non-accrual loans		122
Net fair value adjustments, 90 days or more past due	(2,039)	(861)
Net fair value adjustments, non-accrual loans		(107)
Fair value, past 90 days due	$ 412	154
Fair value, non-accrual loans		$ 15
Number of loans, 90 days past due | Loan	372	257
Number of loans, non-accrual loans | Loan		34