Rialto Segment (Schedule of Credit Facilities) (Details) - Rialto	3 Months Ended	6 Months Ended		12 Months Ended
	May 31, 2015 USD ($)	May 31, 2016 USD ($) extension	May 31, 2015 USD ($)	Dec. 31, 2010	Nov. 30, 2015 USD ($)
Line of Credit Facility [Line Items]
Gain upon acquisition of REO	$ 200,000	$ 2,100,000	$ 200,000
Warehouse Repurchase Facility
Line of Credit Facility [Line Items]
Maximum Aggregate Commitment		$ 1,100,000,000
364-day warehouse repurchase facility that matures August 2016
Line of Credit Facility [Line Items]
Term of note		364 days
364-day warehouse repurchase facility that matures August 2016 | Warehouse Repurchase Facility
Line of Credit Facility [Line Items]
Maximum Aggregate Commitment		$ 250,000,000
364-day warehouse repurchase facility that matures October 2016
Line of Credit Facility [Line Items]
Term of note		364 days
364-day warehouse repurchase facility that matures October 2016 | Warehouse Repurchase Facility
Line of Credit Facility [Line Items]
Maximum Aggregate Commitment		$ 400,000,000
Extension term		1 year
364-day warehouse repurchase facility that matures January 2017
Line of Credit Facility [Line Items]
Term of note		364 days
364-day warehouse repurchase facility that matures January 2017 | Warehouse Repurchase Facility
Line of Credit Facility [Line Items]
Maximum Aggregate Commitment		$ 250,000,000
Warehouse repurchase facility that matures December 2017 | Warehouse Repurchase Facility
Line of Credit Facility [Line Items]
Maximum Aggregate Commitment		100,000,000
Warehouse repurchase facility that matures August 2018 | Warehouse Repurchase Facility
Line of Credit Facility [Line Items]
Maximum Aggregate Commitment		100,000,000
Borrowings under the facilities		$ 53,800,000			$ 36,300,000
Number of extensions | extension		2
Extension term		1 year
Bank Portfolios
Line of Credit Facility [Line Items]
Term of note				5 years