Goodwill and Other Intangible Assets - Schedule of Core Deposit and Other Intangibles (Details) - USD ($) $ in Thousands	Dec. 31, 2017	Dec. 31, 2016	Dec. 31, 2015
Finite Lived Intangible Assets [Line Items]
Net Carrying Amount	$ 14,902	$ 4,563
Core Deposits
Finite Lived Intangible Assets [Line Items]
Net Carrying Amount	14,902
Clayton Banks
Finite Lived Intangible Assets [Line Items]
Gross Carrying Amount	42,190
Accumulated Amortization	(27,288)
Net Carrying Amount	14,902
Clayton Banks | Core Deposits
Finite Lived Intangible Assets [Line Items]
Gross Carrying Amount	38,915	29,855	$ 29,855
Accumulated Amortization	(27,121)	(25,292)	(23,160)
Net Carrying Amount	11,794	$ 4,563	$ 6,695
Clayton Banks | Leasehold Intangible
Finite Lived Intangible Assets [Line Items]
Gross Carrying Amount	587
Accumulated Amortization	(38)
Net Carrying Amount	549
Clayton Banks | Customer Base Trust Intangible
Finite Lived Intangible Assets [Line Items]
Gross Carrying Amount	1,600
Accumulated Amortization	(67)
Net Carrying Amount	1,533
Clayton Banks | Manufactured Housing Servicing Intangible
Finite Lived Intangible Assets [Line Items]
Gross Carrying Amount	1,088
Accumulated Amortization	(62)
Net Carrying Amount	$ 1,026