Segment Reporting - Narrative (Details) $ in Thousands	3 Months Ended				6 Months Ended
	Jun. 30, 2020 USD ($)	Mar. 31, 2020 USD ($)	Jun. 30, 2019 USD ($)	Mar. 31, 2019 USD ($)	Jun. 30, 2020 USD ($) segment	Jun. 30, 2019 USD ($)
Segment Reporting Information [Line Items]
Number of reportable segments | segment					3
Net income (loss)	$ 18,034	$ 1,048	$ 15,377	$ 3,192	$ 19,082	$ 18,569
Gain on sale of banking centers	9,631		0		9,631	0
Increase (decrease) in professional fees	2,994		881		4,325	1,709
Net income from continuing operations	18,034		14,931		19,082	18,123
The First State Bank
Segment Reporting Information [Line Items]
Accretion	409				409
Operating Segments | Commercial & Retail Banking
Segment Reporting Information [Line Items]
Net income (loss)	9,215		14,695		12,938	18,895
Increase (decrease) in net interest income	2,100				3,400
Increase (decrease) in loan interest and fee income	638				1,300
Increase (decrease) in deposit interest	(1,500)				(1,700)
Increase (decrease) in average balance of loans	$ 283,500				$ 215,000
Increase (decrease) in basis points	(0.0084)				(0.0061)
Increase (decrease) in average balance of deposits	$ 182,200				$ 135,700
Increase (decrease) in noninterest income	2,100				3,900
Gain on sale of banking centers	9,600				9,600
Bargain purchase gain	4,700				4,700
Increase (Decrease) in consulting income	1,300				2,600
Increase (decrease) in gain on sale of securities	602				998
Increase (decrease) in holding gain on equity securities	(13,600)				(13,700)
Increase (decrease) in noninterest expense	5,600				8,400
Increase (decrease) in salaries and employee benefits expense	2,000				3,400
Increase (decrease) in professional fees	2,000				2,500
Increase (decrease) in data processing expense	721				910
Increase (decrease) in provision expense	6,000				6,900
Increase (decrease) In other operating expenses					729
Increase (decrease) in travel, entertainment, dues, and subscriptions					552
Net income from continuing operations			14,695			18,895
Operating Segments | Mortgage Banking
Segment Reporting Information [Line Items]
Net income (loss)	10,886		1,902		9,764	$ 2,237
Increase (decrease) in net interest income	1,500				2,000
Increase (decrease) in average balance of loans	153,800				119,400
Increase (decrease) in noninterest income	17,600				18,200
Increase (decrease) in noninterest expense	7,000				10,100
Increase (decrease) in salaries and employee benefits expense	6,500				9,300
Increase (decrease) in professional fees	135				382
Increase (decrease) In other operating expenses	146
Increase (decrease) in interest and fees on loans	1,500				2,400
Increase (decrease) in derivative performance	12,300				8,300
Increase (decrease) in mortgage fee income	5,400				10,100
Increase (decrease) in hedge valuation	4,400
Increase (decrease) in mortgage closed loan volume	451,400				623,000
Increase (decrease) in mortgage processing expense	174				216
Increase (decrease) in FHLB and other borrowings interest income					412
Increase (decrease) in average borrowings					$ 119,800
Increase (decrease) in locked mortgage pipeline					212.00%	104.40%
Net income from continuing operations			1,902			$ 2,237
Operating Segments | Financial Holding Company
Segment Reporting Information [Line Items]
Net income (loss)	(2,067)		(1,220)		$ (3,620)	(2,563)
Increase (decrease) in net interest income	(264)				(514)
Increase (decrease) in noninterest income	184				(91)
Increase (decrease) in gain on sale of securities	39				(98)
Increase (decrease) in noninterest expense	1,000				1,300
Increase (decrease) in salaries and employee benefits expense	883				1,100
Increase (decrease) in professional fees	368				394
Net income from continuing operations	(2,100)		$ (1,666)		(3,600)	$ (3,009)
Increase (decrease) in income tax expense	(179)				(260)
Increase (decrease) in interest on subordinated debt	(264)				$ (514)
Increase (decrease) in intercompany services income	$ 143