Consolidated Statements of Cash Flows - USD ($) $ in Thousands	12 Months Ended
	Dec. 31, 2022	Dec. 31, 2021	Dec. 31, 2020
Cash flows from operating activities:
Net income from continuing operations	$ 27,577	$ 52,624	$ 17,837
Net income (loss) from discontinued operations	337	(144)	(140)
Adjustments to reconcile net income to net cash provided by (used in) operating activities:
Depreciation and amortization	1,977	2,144	924
Deferred income tax expense (benefit)	3,025	1,923	(1,680)
Provision for loan losses	17,886	117	10,450
Accretion of fair value adjustments (discounts) on acquired loans	(7,392)	(2,006)	(1,030)
Accretion of fair value adjustments (premiums) on acquired time deposits	(1,480)	(3,225)	(23)
Accretion of fair value adjustments (premiums) on acquired subordinated notes	(101)	(176)	0
Proceeds from sale of mortgage loans held for sale	522,252	1,228,021	1,099,378
Mortgage loans held for sale, originated	(416,772)	(1,171,787)	(1,180,190)
Gain on sale of mortgage loans	(4,451)	(21,432)	(42,140)
Proceeds from sale of guaranteed government loans held for sale	54,271	0	0
Guaranteed government loans held for sale, originated	(88,077)	0	0
Gain on sale of guaranteed government loans	(4,734)	0	0
Gain on sale of Paycheck Protection Program loans	0	(24,315)	0
Realized losses (gains) on sale of available for sale securities	0	144	(211)
(Gain) loss on disposal of premises and equipment	(263)	110	160
Investment amortization expense, net	1,332	1,865	1,138
Amortization of subordinated debt issuance costs	35	206	55
Intangible amortization	1,525	1,671	629
Fair value adjustments of other equity investments	(9,306)	(7,316)	0
Fair value adjustments attributable to mortgage servicing rights	(2,248)	0	0
Fair value adjustments on other real estate owned	0	75	0
Increase in cash surrender value of bank owned life insurance	(1,348)	(932)	(390)
Increase in other assets	(5,431)	12,347	(26,332)
Increase (decrease) in other liabilities	5,262	(10,921)	13,002
Net cash provided by (used in) operating activities - continuing operations	93,876	58,993	(108,563)
Net cash provided by operating activities - discontinued operations	55	220	223
Cash provided by (used in) operating activities	93,931	59,213	(108,340)
Cash flows used in investing activities:
Net increase in loans held for investment	(631,797)	(59,053)	(53,320)
Net decrease (increase) in federal funds sold	42,477	(41,396)	(295)
Purchases of securities available for sale	(68,261)	(264,929)	(44,164)
Proceeds from calls, sales, paydowns and maturities of securities available for sale	32,655	71,804	53,595
Proceeds from calls, sales, paydowns, and maturities of securities held to maturity	0	0	1,212
Proceeds from sale of other real estate owned	70	341	0
Proceeds from sale of Paycheck Protection Program loans	0	705,930	0
Net decrease (increase) in Paycheck Protection Program loans	18,439	(382,830)	(292,068)
Net change in restricted equity and other investments	(13,899)	(78)	(2,943)
Purchases of premises and equipment	(455)	(1,217)	(3,010)
Proceeds from sale of premises and equipment	2,211	547	719
Purchases of bank owned life insurance	0	(9,600)	0
Redemption of bank owned life insurance	414	0	0
Capital calls of small business investment company funds and other investments	(11,310)	(11,582)	(609)
Net cash acquired in acquisition of Bay Banks of Virginia, Inc.	0	44,066	0
Nonincome distributions from SBICs and other investments	1,028	647	94
Net cash (used in) provided by investing activities - continuing operations	(628,428)	52,650	(340,789)
Net cash provided by (used in) investing activities - discontinued operations	245	(166)	(96)
Cash (used in) provided by investing activities	(628,183)	52,484	(340,885)
Cash flows from financing activities:
Net increase in demand, savings and other interest-bearing deposits	312,277	452,173	232,591
Net decrease in time deposits	(106,061)	(127,174)	(9,512)
Common stock dividends paid	(9,175)	(7,183)	(2,436)
FHLB advances	822,900	721,000	676,900
FHLB repayments	(521,200)	(836,000)	(686,700)
FRB advances	0	434,336	363,682
FRB repayments	(17,850)	(722,900)	(82,032)
Stock option exercised	14	804	0
Dividend reinvestment plan issuances	73	0	0
Redemptions of subordinated notes	0	(14,150)	0
Issuance of subordinated notes	0	0	15,000
Payment of subordinated notes issuance costs	0	0	(349)
Net Cash Provided by (Used in) Financing Activities, Continuing Operations	480,978	(99,094)	507,144
Net cash provided by financing activities - discontinued operations	0	0	0
Cash provided by (used in) financing activities	480,978	(99,094)	507,144
Net (decrease) increase in cash and due from banks	(53,274)	12,603	57,919
Cash and due from banks at beginning of period	130,548	117,945	60,026
Cash and due from banks at end of period	77,274	130,548	117,945
Supplemental Schedule of Cash Flow Information
Interest	16,011	11,583	10,030
Income taxes	2,077	10,131	2,000
Non-cash investing and financing activities:
Unrealized (losses) gains on securities available for sale	(53,405)	(6,024)	1,029
Transfers of securities from held to maturity to available for sale	0	0	10,980
Restricted stock awards, net of forfeitures	1,564	1,331	567
Assets acquired in business combination	0	1,224,583	0
Liabilities assumed in business combination	0	1,107,036	0
Effective settlement of subordinated notes in business combination	0	650	0
Change in goodwill	$ 0	$ 7,206	$ 23
Cumulative effect adjustment of change in accounting method	3,542	—	—