Reportable Segments (Details) - USD ($) $ in Thousands		3 Months Ended				6 Months Ended
		Jun. 30, 2016	Jun. 30, 2015	Jun. 30, 2014	Jun. 30, 2013	Jun. 30, 2016	Jun. 30, 2015	Dec. 31, 2015
Segment Reporting Information [Line Items]
Interest Income (Expense), Net		$ 49,169	$ 45,085			$ 99,048	$ 91,157
Noninterest Income		36,795	42,660			74,817	84,005
Revenues		85,964	87,745			173,865	175,162
Provision/ (credit) for loan losses		(2,535)	0			(5,668)	(2,500)
Noninterest Expense		64,731	62,418			131,440	125,845
Income before income taxes		23,768	25,327			48,093	51,817
Income tax expense		7,626	8,000			15,064	16,572
Net income from continuing operations		16,142	17,327			33,029	35,245
Net income attributable to noncontrolling interests		989	1,263			1,900	2,492
Net income from discontinued operations		1,245	1,546			3,310	3,640
Net income attributable to the Company		16,398	17,610			34,439	36,393
Total assets		7,579,601	7,045,939			7,579,601	7,045,939	$ 7,542,508
Amortization of intangibles		1,586	1,655			3,172	3,257
Depreciation		1,698	1,652			3,374	3,356
Private Banking Segment [Member]
Segment Reporting Information [Line Items]
Interest Income (Expense), Net		49,713	46,017			100,133	93,007
Noninterest Income		2,227	3,437			5,605	6,221
Revenues		51,940	49,454			105,738	99,228
Provision/ (credit) for loan losses		(2,535)	0			(5,668)	(2,500)
Noninterest Expense		32,093	28,281			63,368	57,086
Income before income taxes		22,382	21,173			48,038	44,642
Income tax expense		7,038	6,848			15,412	14,616
Net income from continuing operations		15,344	14,325			32,626	30,026
Net income attributable to the Company		15,344	14,325			32,626	30,026
Total assets		7,417,465	6,860,288			7,417,465	6,860,288
Amortization of intangibles		0	45			0	91
Depreciation		1,125	1,159			2,271	2,364
Wealth Management and Trust Segment [Member]
Segment Reporting Information [Line Items]
Noninterest Income		11,300	14,647			22,356	28,604
Noninterest Expense	[1]	13,738	12,777			29,590	25,108
Income before income taxes		(2,438)	1,870			(7,234)	3,496
Income tax expense		(970)	806			(2,909)	1,500
Net income from continuing operations		(1,468)	1,064			(4,325)	1,996
Net income attributable to the Company		(1,468)	1,064			(4,325)	1,996
Total assets		86,397	78,480			86,397	78,480
Amortization of intangibles		745	624			1,490	1,195
Depreciation		269	194			500	381
Investment Managers Segment [Member]
Segment Reporting Information [Line Items]
Interest Income (Expense), Net		4	5			8	11
Noninterest Income		10,646	11,739			21,305	23,461
Revenues		10,650	11,744			21,313	23,472
Noninterest Expense		7,895	8,544			15,919	17,230
Income before income taxes		2,755	3,200			5,394	6,242
Income tax expense		898	1,040			1,777	2,042
Net income from continuing operations		1,857	2,160			3,617	4,200
Net income attributable to noncontrolling interests		469	620			946	1,257
Net income attributable to the Company		1,388	1,540			2,671	2,943
Total assets		93,975	100,029			93,975	100,029
Amortization of intangibles		651	739			1,301	1,478
Depreciation		74	70			147	140
Wealth Advisors Segment [Member]
Segment Reporting Information [Line Items]
Interest Income (Expense), Net		4	2			7	4
Noninterest Income		12,579	12,708			25,321	25,415
Revenues		12,583	12,710			25,328	25,419
Noninterest Expense		9,171	8,720			18,865	17,852
Income before income taxes		3,412	3,990			6,463	7,567
Income tax expense		1,266	1,483			2,414	2,804
Net income from continuing operations		2,146	2,507			4,049	4,763
Net income attributable to noncontrolling interests		520	643			954	1,231
Net income attributable to the Company		1,626	1,864			3,095	3,532
Total assets		76,370	76,847			76,370	76,847
Amortization of intangibles		190	247			381	493
Depreciation		220	216			435	427
Holding Company and Eliminations Segment [Member]
Segment Reporting Information [Line Items]
Interest Income (Expense), Net		(552)	(939)			(1,100)	(1,865)
Noninterest Income		43	129			230	304
Revenues		(509)	(810)			(870)	(1,561)
Noninterest Expense		1,834	4,096			3,698	8,569
Income before income taxes		(2,343)	(4,906)			(4,568)	(10,130)
Income tax expense		(606)	(2,177)			(1,630)	(4,390)
Net income from continuing operations		(1,737)	(2,729)			(2,938)	(5,740)
Net income attributable to noncontrolling interests		0	0			0	4
Net income from discontinued operations		1,245	1,546			3,310	3,640
Net income attributable to the Company		(492)	(1,183)			372	(2,104)
Total assets		(94,606)	(69,705)			(94,606)	(69,705)
Amortization of intangibles				$ 0	$ 0	0	0
Depreciation		$ 10	$ 13			$ 21	$ 44

[1] Operating expense includes $0.9 million and $2.0 million of restructuring expense for the three and six months ended June 30, 2016, respectively, related to the Wealth Management and Trust segment. Operating expense includes $0.2 million of restructuring expenses for both the three and six months ended June 30, 2015 related to the Wealth Management and Trust segment.