Commitments and Contingencies (Details) $ in Thousands, shares in Millions			3 Months Ended
	Sep. 13, 2013 USD ($) item	Nov. 02, 2012 USD ($) item	Mar. 31, 2016 USD ($) shares	Mar. 31, 2015 USD ($)	Mar. 31, 2016 USD ($)	Dec. 31, 2015 USD ($)
Reserve for Indemnification Liability:
FDIC, collections received			$ 111,500
Representation and Warranty Claims
Roll-forward of claims activity for loans put-back to the mortgage origination segment
Balance, beginning of period			57,298	$ 53,906
Claims made			4,548	37,323
Claims resolved with no payment			(6,115)	(11,687)
Repurchases			(1,157)	(5,239)
Indemnification payments			(372)	(203)
Balance, end of period			54,202	74,100
Reserve for Indemnification Liability:
Total			57,298	53,906	$ 54,202	$ 57,298
Reserve for Indemnification liability established by mortgage origination segment
Commitments and Contingencies
Provision for indemnification losses			900	800
Roll-forward of claims activity for loans put-back to the mortgage origination segment
Balance, beginning of period			16,640	17,619
Additions for new sales			878	844
Repurchases			(112)	(498)
Early payment defaults			(90)	(10)
Indemnification payments			(169)	(162)
Change in estimate				(451)
Balance, end of period			17,147	17,342
Reserve for Indemnification Liability:
Specific claims					4,754	5,210
Incurred but not reported claims					12,393	11,430
Total			$ 16,640	$ 17,619	$ 17,147	$ 16,640
Shareholder Class Action Lawsuits
Commitments and Contingencies
Number of shares of SWS common stock held by purported shareholders | shares			5.2
Bank | FNB
Reserve for Indemnification Liability:
Number of loss-sharing agreements | item	2
Loans and OREO acquired	$ 1,200,000
Percentage of net losses to be absorbed by FDIC on the first $240.4 million of net losses incurred as per the loss sharing agreement	80.00%
Threshold amount of net losses incurred for 80% of net losses to be absorbed by FDIC as per the loss sharing agreement, first layer	$ 240,400
Percentage of net losses to be absorbed by FDIC in excess of $240.4 million up to and including $365.7 million of net losses incurred as per the loss sharing agreement	0.00%
Threshold amount of net losses incurred for 0% of net losses to be absorbed by FDIC as per the loss sharing agreement, second layer	$ 365,700
Percentage of net losses to be absorbed by FDIC in excess of $365.7 million of net losses incurred as per the loss sharing agreement	80.00%
Threshold limit of subsequent recoveries reimbursable to the FDIC under the loss share agreement	$ 0
Covered net losses billed to the FDIC			$ 139,400
Percentage of covered net losses reimbursable under the loss-share agreements			80.00%
Amount of covered net losses reimbursable under the loss-share agreements			$ 111,500
FDIC, collections received			111,500
Bank | FNB | Covered
Reserve for Indemnification Liability:
Period for which payment is required to be made to the FDIC of true-up amount	10 years
Bank | FNB | Covered | Commercial loan
Reserve for Indemnification Liability:
Period of loss-sharing agreements in effect	5 years
Period of loss recovery provisions in effect	8 years
Bank | FNB | Covered | Single family residential loans
Reserve for Indemnification Liability:
Period of loss-sharing agreements in effect	10 years
Period of loss recovery provisions in effect	10 years
Bank | FNB | Minimum
Reserve for Indemnification Liability:
Covered losses and reimbursable expenses			240,400
Bank | FNB | Maximum
Reserve for Indemnification Liability:
Covered losses and reimbursable expenses			$ 365,700
First Southwest Company
Commitments and Contingencies
Number of defendants other than reporting entity | item		13
Amount of bonds to finance a loan to 38 Studios		$ 75,000