Commitments and Contingencies (Details) (USD $) Share data in Millions, unless otherwise specified	3 Months Ended			0 Months Ended		12 Months Ended	0 Months Ended
	Mar. 31, 2015	Mar. 31, 2014	Mar. 31, 2013	Sep. 13, 2013	Sep. 13, 2013 item	Dec. 31, 2014	Nov. 02, 2012 item	Apr. 30, 2014 item
Commitments and Contingencies
Number of shares of SWS common stock held by purported shareholders	8.43
Representation and Warranty Claims
Commitments and Contingencies
Indemnification liability reserve	74,100,000	$ 51,929,000
Roll-forward of claims activity for loans put-back to the mortgage origination segment
Balance, beginning of period	53,906,000	51,912,000				51,912,000
Claims made	37,323,000	10,044,000
Claims resolved with no payment	(11,687,000)	(4,458,000)
Repurchases	(5,239,000)	(4,878,000)
Indemnification payments	(203,000)	(691,000)
Balance, end of period	74,100,000	51,929,000
Reserve for Indemnification Liability:
Total	74,100,000	51,929,000
Reserve for Indemnification liability established by mortgage origination segment
Commitments and Contingencies
Indemnification liability reserve	17,342,000	20,975,000
Provision for indemnification losses		800,000	600,000
Roll-forward of claims activity for loans put-back to the mortgage origination segment
Balance, beginning of period	17,619,000	21,121,000				21,121,000
Additions for new sales	844,000	560,000
Repurchases	(498,000)	(504,000)
Early payment defaults	(10,000)	(21,000)
Indemnification payments	(162,000)	(181,000)
Change in estimate	(451,000)
Balance, end of period	17,342,000	20,975,000
Reserve for Indemnification Liability:
Specific claims	7,783,000					7,912,000
Incurred but not reported claims	9,559,000					9,707,000
Total	17,342,000	20,975,000
Shareholder Class Action Lawsuits
Commitments and Contingencies
Number of purported shareholder class action lawsuits								2
Bank | FNB
Reserve for Indemnification Liability:
Number of loss-sharing agreements					2
Loans and OREO acquired				1,200,000,000	1,200,000,000
Percentage of losses to be absorbed by FDIC on the first $240.4 million of losses incurred as per the loss sharing agreement				80.00%	80.00%
Threshold amount of losses incurred for 80% of losses to be absorbed by FDIC as per the loss sharing agreement, first layer				240,400,000	240,400,000
Percentage of losses to be absorbed by FDIC in excess of $240.4 million up to and including $365.7 million of losses incurred as per the loss sharing agreement				0.00%	0.00%
Threshold amount of losses incurred for 0% of losses to be absorbed by FDIC as per the loss sharing agreement, second layer				365,700,000	365,700,000
Percentage of losses to be absorbed by FDIC in excess of $365.7 million of losses incurred as per the loss sharing agreement				80.00%	80.00%
Period for which payment is required to be made to the FDIC of true-up amount					10 years
Covered net losses billed to the FDIC	104,700,000
Percentage of covered net losses reimbursable under the loss-share agreements	80.00%
Amount of covered net losses reimbursable under the loss-share agreements	83,800,000
Aggregate reimbursements received	83,800,000
Bank | FNB | Covered | Commercial loan
Reserve for Indemnification Liability:
Period of loss-sharing agreements in effect					5 years
Period of loss recovery provisions in effect					8 years
Bank | FNB | Covered | Single family residential loans
Reserve for Indemnification Liability:
Period of loss-sharing agreements in effect					10 years
Period of loss recovery provisions in effect					10 years
Bank | FNB | Minimum
Reserve for Indemnification Liability:
Covered losses and reimbursable expenses						240,400,000
Bank | FNB | Maximum
Reserve for Indemnification Liability:
Covered losses and reimbursable expenses						365,700,000
First Southwest Company
Commitments and Contingencies
Number of defendants other than reporting entity							13
Amount of bonds to finance a loan to 38 Studios							$ 75,000,000