Discontinued Operations (Details) $ in Thousands		3 Months Ended								12 Months Ended
	Jun. 02, 2016 USD ($)	Dec. 31, 2016 USD ($)	Sep. 30, 2016 USD ($)	Jun. 30, 2016 USD ($)	Mar. 31, 2016 USD ($)	Dec. 31, 2015 USD ($)	Sep. 30, 2015 USD ($)	Jun. 30, 2015 USD ($)	Mar. 31, 2015 USD ($)	Dec. 31, 2016 USD ($)	Dec. 31, 2015 USD ($)	Dec. 31, 2014 USD ($)	Jun. 20, 2016 USD ($)
Income Statement, Balance Sheet and Additional Disclosures by Disposal Groups, Including Discontinued Operations [Line Items]
Noncontrolling Interest, Ownership Percentage by Parent										83.68%
Discontinued Operation, Tax Effect of Discontinued Operation										$ 7,218	$ 3,407	$ 4,041
Other Income Tax Expense (Benefit), Continuing Operations		$ (5,072)	$ (751)	$ (2,627)	$ (3,812)	$ (712)	$ (1,437)	$ 400	$ 2,155	$ (12,262)	406	12,134
Noncontrolling Interest, Ownership Percentage by Noncontrolling Owners		16.32%								16.32%
Proceeds from Divestiture of Businesses										$ 45,448	0	0
Dividends paid to noncontrolling interest			5,700							5,740	$ 0	$ 0
Oppenheimer Multi Family Housing And Healthcare Finance [Member]
Income Statement, Balance Sheet and Additional Disclosures by Disposal Groups, Including Discontinued Operations [Line Items]
Number of Loans Insured by U.S. Department of Housing and Development	480
Disposal Group, Including Discontinued Operation, Consideration			3,800										$ 45,000
Disposal Group, Including Discontinued Operations, Consideration Withheld													$ 1,400
Discontinued Operation, Gain (Loss) from Disposal of Discontinued Operation, before Income Tax	$ 14,900
Dividends paid to noncontrolling interest										$ 5,700
Business Pipeline of Mortgage Loans [Member]
Income Statement, Balance Sheet and Additional Disclosures by Disposal Groups, Including Discontinued Operations [Line Items]
Proceeds from Divestiture of Businesses			$ 1,400	$ 1,500