Fair Value Measurements - Estimated Fair Value (Details) - USD ($) $ in Thousands	Dec. 31, 2015	Dec. 31, 2014	Dec. 31, 2013
Financial assets:
Held to maturity securities	$ 331,468	$ 118,345
Non-covered loans, net	5,174,625	3,883,435
Covered loans, net	378,762	638,029
Broker-dealer and clearing organization receivables	1,362,499	167,884
FDIC indemnification asset	91,648	130,437	$ 188,291
Financial liabilities:
Broker-dealer and clearing organization payables	1,338,305	179,042
Level 2
Financial assets:
Loans held for sale	1,434,955	1,263,135
Level 3
Financial assets:
Loans held for sale	25,880	9,017
Carrying Amount
Financial assets:
Cash and cash equivalents	669,445	813,075
Securities purchased under agreements to resell	105,660
Assets segregated for regulatory purposes	158,613	76,013
Held to maturity securities	332,022	118,209
Loans held for sale	72,843	37,541
Non-covered loans, net	5,174,625	3,883,435
Covered loans, net	378,762	638,029
Broker-dealer and clearing organization receivables	1,362,499	167,884
FDIC indemnification asset	91,648	130,437
Other assets	68,786	59,432
Financial liabilities:
Deposits	6,952,683	6,369,892
Broker-dealer and clearing organization payables	1,338,305	179,042
Short-term borrowings	947,373	762,696
Debt	305,728	123,696
Other liabilities	3,699	2,144
Estimate of Fair Value Measurement
Financial assets:
Cash and cash equivalents	669,445	813,075
Securities purchased under agreements to resell	105,660
Assets segregated for regulatory purposes	158,613	76,013
Held to maturity securities	331,468	118,345
Loans held for sale	72,843	37,541
Non-covered loans, net	5,203,374	3,907,194
Covered loans, net	527,201	767,751
Broker-dealer and clearing organization receivables	1,362,499	167,884
FDIC indemnification asset	91,648	130,437
Other assets	68,786	59,432
Financial liabilities:
Deposits	6,955,919	6,365,555
Broker-dealer and clearing organization payables	1,338,305	179,042
Short-term borrowings	947,373	762,696
Debt	299,257	117,028
Other liabilities	3,699	2,144
Estimate of Fair Value Measurement | Level 1
Financial assets:
Cash and cash equivalents	669,445	813,075
Assets segregated for regulatory purposes	158,613	76,013
Estimate of Fair Value Measurement | Level 2
Financial assets:
Securities purchased under agreements to resell	105,660
Held to maturity securities	331,468	118,345
Loans held for sale	72,843	37,541
Non-covered loans, net	602,968	378,425
Broker-dealer and clearing organization receivables	1,362,499	167,884
Other assets	53,214	43,937
Financial liabilities:
Deposits	6,955,919	6,365,555
Broker-dealer and clearing organization payables	1,338,305	179,042
Short-term borrowings	947,373	762,696
Debt	299,257	117,028
Other liabilities	3,699	2,144
Estimate of Fair Value Measurement | Level 3
Financial assets:
Non-covered loans, net	4,600,406	3,528,769
Covered loans, net	527,201	767,751
FDIC indemnification asset	91,648	130,437
Other assets	$ 15,572	$ 15,495