Segment Reporting (Details) - USD ($) $ in Thousands	3 Months Ended				6 Months Ended
	Jun. 30, 2020	Mar. 31, 2020	Jun. 30, 2019	Mar. 31, 2019	Jun. 30, 2020	Jun. 30, 2019	Dec. 31, 2019	Dec. 31, 2018
Segment Reporting Information [Line Items]
Interest income	$ 21,774		$ 20,470		$ 42,473	$ 40,093
Interest expense	3,316		5,941		7,844	11,592
NET INTEREST INCOME	18,458		14,529		34,629	28,501
Provision for loan losses	6,596		600		7,734	900
Net interest income after provision for loan losses	11,862		13,929		26,895	27,601
Noninterest Income:
Mortgage fee income	14,944		9,864		26,163	16,534
Other income	30,569		16,523		30,200	18,618
Total noninterest income	45,513		26,387		56,363	35,152
Noninterest Expenses:
Salaries and employee benefits	22,659		13,280		38,841	25,014
Other expense	10,674		7,110		19,148	13,824
Total noninterest expenses	33,333		20,390		57,989	38,838
Income from continuing operations, before income taxes	24,042		19,926		25,269	23,915
Income tax expense (benefit)	6,008		4,995		6,187	5,792
Net income from continuing operations	18,034		14,931		19,082	18,123
Income from discontinued operations, before income taxes	0		600		0	600
Income tax expense (benefit) - discontinued operations	0		154		0	154
Net income (loss) from discontinued operations	0		446		0	446
Net income	18,034	$ 1,048	15,377	$ 3,192	19,082	18,569
Preferred dividends	115		122		229	243
Net income available to common shareholders - basic	17,919		15,255		18,853	18,326
Capital expenditures	1,135		514		2,519	629
Total assets	2,215,157		1,833,018		2,215,157	1,833,018	$ 1,944,114	$ 1,750,969
Goodwill	19,232		18,480		19,232	18,480	19,630	18,480
Operating Segments | Commercial & Retail Banking
Segment Reporting Information [Line Items]
Interest income	19,182		18,820		37,956	37,147
Interest expense	3,027		4,743		6,865	9,497
NET INTEREST INCOME	16,155		14,077		31,091	27,650
Provision for loan losses	6,598		625		7,730	872
Net interest income after provision for loan losses	9,557		13,452		23,361	26,778
Noninterest Income:
Mortgage fee income	40		277		150	386
Other income	17,792		15,464		21,138	17,030
Total noninterest income	17,832		15,741		21,288	17,416
Noninterest Expenses:
Salaries and employee benefits	6,170		4,220		12,036	8,615
Other expense	9,124		5,493		15,783	10,845
Total noninterest expenses	15,294		9,713		27,819	19,460
Income from continuing operations, before income taxes	12,095		19,480		16,830	24,734
Income tax expense (benefit)	2,880		4,785		3,892	5,839
Net income from continuing operations			14,695			18,895
Income from discontinued operations, before income taxes			0			0
Income tax expense (benefit) - discontinued operations			0			0
Net income (loss) from discontinued operations			0			0
Net income	9,215		14,695		12,938	18,895
Preferred dividends	0		0		0	0
Net income available to common shareholders - basic	9,215		14,695		12,938	18,895
Capital expenditures	1,105		414		2,400	503
Total assets	2,219,352		1,831,419		2,219,352	1,831,419	1,953,975	1,753,932
Goodwill	2,350		1,598		2,350	1,598	2,748	1,598
Operating Segments | Mortgage Banking
Segment Reporting Information [Line Items]
Interest income	3,538		2,032		5,956	3,570
Interest expense	1,517		1,499		2,904	2,492
NET INTEREST INCOME	2,021		533		3,052	1,078
Provision for loan losses	(2)		(25)		4	28
Net interest income after provision for loan losses	2,023		558		3,048	1,050
Noninterest Income:
Mortgage fee income	15,208		9,792		26,555	16,489
Other income	13,354		1,135		9,792	1,611
Total noninterest income	28,562		10,927		36,347	18,100
Noninterest Expenses:
Salaries and employee benefits	13,584		7,038		21,468	12,197
Other expense	2,315		1,842		4,712	3,867
Total noninterest expenses	15,899		8,880		26,180	16,064
Income from continuing operations, before income taxes	14,686		2,605		13,215	3,086
Income tax expense (benefit)	3,800		703		3,451	849
Net income from continuing operations			1,902			2,237
Income from discontinued operations, before income taxes			0			0
Income tax expense (benefit) - discontinued operations			0			0
Net income (loss) from discontinued operations			0			0
Net income	10,886		1,902		9,764	2,237
Preferred dividends	0		0		0	0
Net income available to common shareholders - basic	10,886		1,902		9,764	2,237
Capital expenditures	30		23		99	27
Total assets	342,497		225,012		342,497	225,012	248,382	165,430
Goodwill	16,882		16,882		16,882	16,882	16,882	16,882
Operating Segments | Financial Holding Company
Segment Reporting Information [Line Items]
Interest income	1		1		2	3
Interest expense	23		287		58	572
NET INTEREST INCOME	(22)		(286)		(56)	(569)
Provision for loan losses	0		0		0	0
Net interest income after provision for loan losses	(22)		(286)		(56)	(569)
Noninterest Income:
Mortgage fee income	0		0		0	0
Other income	1,679		1,495		3,183	3,274
Total noninterest income	1,679		1,495		3,183	3,274
Noninterest Expenses:
Salaries and employee benefits	2,905		2,022		5,337	4,202
Other expense	1,491		1,346		2,566	2,408
Total noninterest expenses	4,396		3,368		7,903	6,610
Income from continuing operations, before income taxes	(2,739)		(2,159)		(4,776)	(3,905)
Income tax expense (benefit)	(672)		(493)		(1,156)	(896)
Net income from continuing operations	(2,100)		(1,666)		(3,600)	(3,009)
Income from discontinued operations, before income taxes			600			600
Income tax expense (benefit) - discontinued operations			154			154
Net income (loss) from discontinued operations			446			446
Net income	(2,067)		(1,220)		(3,620)	(2,563)
Preferred dividends	115		122		229	243
Net income available to common shareholders - basic	(2,182)		(1,342)		(3,849)	(2,806)
Capital expenditures	0		77		20	99
Total assets	232,026		217,217		232,026	217,217	216,411	196,537
Goodwill	0		0		0	0	0	0
Intercompany Eliminations
Segment Reporting Information [Line Items]
Interest income	(947)		(383)		(1,441)	(627)
Interest expense	(1,251)		(588)		(1,983)	(969)
NET INTEREST INCOME	304		205		542	342
Provision for loan losses	0		0		0	0
Net interest income after provision for loan losses	304		205		542	342
Noninterest Income:
Mortgage fee income	(304)		(205)		(542)	(341)
Other income	(2,256)		(1,571)		(3,913)	(3,297)
Total noninterest income	(2,560)		(1,776)		(4,455)	(3,638)
Noninterest Expenses:
Salaries and employee benefits	0		0		0	0
Other expense	(2,256)		(1,571)		(3,913)	(3,296)
Total noninterest expenses	(2,256)		(1,571)		(3,913)	(3,296)
Income from continuing operations, before income taxes	0		0		0	0
Income tax expense (benefit)	0		0		0	0
Net income from continuing operations			0			0
Income from discontinued operations, before income taxes			0			0
Income tax expense (benefit) - discontinued operations			0			0
Net income (loss) from discontinued operations			0			0
Net income	0		0		0	0
Preferred dividends	0		0		0	0
Net income available to common shareholders - basic	0		0		0	0
Capital expenditures	0		0		0	0
Total assets	(578,718)		(440,630)		(578,718)	(440,630)	(474,654)	(364,930)
Goodwill	$ 0		$ 0		$ 0	$ 0	$ 0	$ 0