Fair Value Measurements - Estimated Fair Value (Details) - USD ($) $ in Thousands	Mar. 31, 2016	Dec. 31, 2015	Mar. 31, 2015	Dec. 31, 2014
Financial assets:
Held to maturity securities	$ 313,553	$ 331,468
Non-covered loans, net	5,317,615	5,174,625
Covered loans, net	346,169	378,762
Broker-dealer and clearing organization receivables	1,370,622	1,362,499
FDIC indemnification asset	80,522	91,648	$ 107,567	$ 130,437
Financial liabilities:
Broker-dealer and clearing organization payables	1,284,016	1,338,305
Carrying Amount
Financial assets:
Cash and cash equivalents	527,509	669,445
Securities purchased under agreements to resell	96,646	105,660
Assets segregated for regulatory purposes	120,714	158,613
Held to maturity securities	310,478	332,022
Loans held for sale	58,989	72,843
Non-covered loans, net	5,317,615	5,174,625
Covered loans, net	346,169	378,762
Broker-dealer and clearing organization receivables	1,370,622	1,362,499
FDIC indemnification asset	80,522	91,648
Other assets	60,468	68,786
Financial liabilities:
Deposits	6,984,175	6,952,683
Broker-dealer and clearing organization payables	1,284,016	1,338,305
Short-term borrowings	832,921	947,373
Debt	299,202	305,728
Other liabilities	5,389	3,699
Estimate of Fair Value Measurement
Financial assets:
Cash and cash equivalents	527,509	669,445
Securities purchased under agreements to resell	96,646	105,660
Assets segregated for regulatory purposes	120,714	158,613
Held to maturity securities	313,553	331,468
Loans held for sale	58,989	72,843
Non-covered loans, net	5,355,827	5,203,374
Covered loans, net	484,424	527,201
Broker-dealer and clearing organization receivables	1,370,622	1,362,499
FDIC indemnification asset	77,817	91,648
Other assets	60,468	68,786
Financial liabilities:
Deposits	6,987,315	6,955,919
Broker-dealer and clearing organization payables	1,284,016	1,338,305
Short-term borrowings	832,921	947,373
Debt	292,817	299,257
Other liabilities	5,389	3,699
Estimate of Fair Value Measurement | Level 1
Financial assets:
Cash and cash equivalents	527,509	669,445
Assets segregated for regulatory purposes	120,714	158,613
Estimate of Fair Value Measurement | Level 2
Financial assets:
Securities purchased under agreements to resell	96,646	105,660
Held to maturity securities	313,553	331,468
Loans held for sale	58,989	72,843
Non-covered loans, net	463,987	602,968
Broker-dealer and clearing organization receivables	1,370,622	1,362,499
Other assets	54,412	53,214
Financial liabilities:
Deposits	6,987,315	6,955,919
Broker-dealer and clearing organization payables	1,284,016	1,338,305
Short-term borrowings	832,921	947,373
Debt	292,817	299,257
Other liabilities	5,389	3,699
Estimate of Fair Value Measurement | Level 3
Financial assets:
Non-covered loans, net	4,891,840	4,600,406
Covered loans, net	484,424	527,201
FDIC indemnification asset	77,817	91,648
Other assets	$ 6,056	$ 15,572