Loans and Allowance for Loan Losses - Internal Risk Rating Summary (Details) - USD ($) $ in Thousands	Jun. 30, 2021	Dec. 31, 2020	Jun. 30, 2020
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding PCI	$ 1,676,729	$ 1,415,362	$ 1,448,326
Total Loans	1,699,871	1,454,801
SBA PPP
Classes of the loan portfolio summarized by credit quality indicators:
Total Loans	143,000
Commercial and non-residential real estate
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding PCI	1,385,849	1,141,114	1,127,792
Commercial and non-residential real estate | Commercial business
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding PCI	591,561	523,891
Commercial and non-residential real estate | Commercial real estate
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding PCI	486,980	423,122
Commercial and non-residential real estate | Acquisition and development
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding PCI	100,005	112,126
Commercial and non-residential real estate | SBA PPP
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding PCI	207,303	81,975
Residential real estate
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding PCI	255,935	240,264	279,626
Home Equity
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding PCI	24,833	30,828	37,383
Consumer
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding PCI	10,112	3,156	$ 3,525
Purchased credit impaired loans
Classes of the loan portfolio summarized by credit quality indicators:
Total Loans	23,142	39,439
Purchased credit impaired loans | Commercial and non-residential real estate
Classes of the loan portfolio summarized by credit quality indicators:
Total Loans	13,860	21,008
Purchased credit impaired loans | Commercial and non-residential real estate | Commercial business
Classes of the loan portfolio summarized by credit quality indicators:
Total Loans	11,226	17,604
Purchased credit impaired loans | Commercial and non-residential real estate | Commercial real estate
Classes of the loan portfolio summarized by credit quality indicators:
Total Loans	2,259	1,269
Purchased credit impaired loans | Commercial and non-residential real estate | Acquisition and development
Classes of the loan portfolio summarized by credit quality indicators:
Total Loans	375	2,135
Purchased credit impaired loans | Residential real estate
Classes of the loan portfolio summarized by credit quality indicators:
Total Loans	8,851	16,943
Purchased credit impaired loans | Consumer
Classes of the loan portfolio summarized by credit quality indicators:
Total Loans	431	1,488
Pass
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding PCI	1,560,965	1,285,163
Pass | Commercial and non-residential real estate
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding PCI	1,280,794	1,015,512
Pass | Commercial and non-residential real estate | Commercial business
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding PCI	558,052	496,222
Pass | Commercial and non-residential real estate | Commercial real estate
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding PCI	428,548	356,544
Pass | Commercial and non-residential real estate | Acquisition and development
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding PCI	86,891	80,771
Pass | Commercial and non-residential real estate | SBA PPP
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding PCI	207,303	81,975
Pass | Residential real estate
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding PCI	245,794	236,250
Pass | Home Equity
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding PCI	24,289	30,277
Pass | Consumer
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding PCI	10,088	3,124
Pass | Purchased credit impaired loans
Classes of the loan portfolio summarized by credit quality indicators:
Total Loans	14,650	31,558
Pass | Purchased credit impaired loans | Commercial and non-residential real estate
Classes of the loan portfolio summarized by credit quality indicators:
Total Loans	7,562	15,145
Pass | Purchased credit impaired loans | Commercial and non-residential real estate | Commercial business
Classes of the loan portfolio summarized by credit quality indicators:
Total Loans	6,828	12,263
Pass | Purchased credit impaired loans | Commercial and non-residential real estate | Commercial real estate
Classes of the loan portfolio summarized by credit quality indicators:
Total Loans	594	982
Pass | Purchased credit impaired loans | Commercial and non-residential real estate | Acquisition and development
Classes of the loan portfolio summarized by credit quality indicators:
Total Loans	140	1,900
Pass | Purchased credit impaired loans | Residential real estate
Classes of the loan portfolio summarized by credit quality indicators:
Total Loans	6,659	15,157
Pass | Purchased credit impaired loans | Consumer
Classes of the loan portfolio summarized by credit quality indicators:
Total Loans	429	1,256
Special Mention
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding PCI	52,323	67,935
Special Mention | Commercial and non-residential real estate
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding PCI	51,003	66,574
Special Mention | Commercial and non-residential real estate | Commercial business
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding PCI	18,893	9,529
Special Mention | Commercial and non-residential real estate | Commercial real estate
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding PCI	24,495	32,044
Special Mention | Commercial and non-residential real estate | Acquisition and development
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding PCI	7,615	25,001
Special Mention | Commercial and non-residential real estate | SBA PPP
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding PCI	0	0
Special Mention | Residential real estate
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding PCI	920	948
Special Mention | Home Equity
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding PCI	379	381
Special Mention | Consumer
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding PCI	21	32
Special Mention | Purchased credit impaired loans
Classes of the loan portfolio summarized by credit quality indicators:
Total Loans	703	139
Special Mention | Purchased credit impaired loans | Commercial and non-residential real estate
Classes of the loan portfolio summarized by credit quality indicators:
Total Loans	616	139
Special Mention | Purchased credit impaired loans | Commercial and non-residential real estate | Commercial business
Classes of the loan portfolio summarized by credit quality indicators:
Total Loans	614	136
Special Mention | Purchased credit impaired loans | Commercial and non-residential real estate | Commercial real estate
Classes of the loan portfolio summarized by credit quality indicators:
Total Loans	2	3
Special Mention | Purchased credit impaired loans | Commercial and non-residential real estate | Acquisition and development
Classes of the loan portfolio summarized by credit quality indicators:
Total Loans	0	0
Special Mention | Purchased credit impaired loans | Residential real estate
Classes of the loan portfolio summarized by credit quality indicators:
Total Loans	87	0
Special Mention | Purchased credit impaired loans | Consumer
Classes of the loan portfolio summarized by credit quality indicators:
Total Loans	0	0
Substandard
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding PCI	60,514	58,270
Substandard | Commercial and non-residential real estate
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding PCI	51,775	55,230
Substandard | Commercial and non-residential real estate | Commercial business
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding PCI	13,679	17,045
Substandard | Commercial and non-residential real estate | Commercial real estate
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding PCI	33,781	34,001
Substandard | Commercial and non-residential real estate | Acquisition and development
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding PCI	4,315	4,184
Substandard | Commercial and non-residential real estate | SBA PPP
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding PCI	0	0
Substandard | Residential real estate
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding PCI	8,666	2,896
Substandard | Home Equity
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding PCI	70	144
Substandard | Consumer
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding PCI	3	0
Substandard | Purchased credit impaired loans
Classes of the loan portfolio summarized by credit quality indicators:
Total Loans	1,556	2,273
Substandard | Purchased credit impaired loans | Commercial and non-residential real estate
Classes of the loan portfolio summarized by credit quality indicators:
Total Loans	426	608
Substandard | Purchased credit impaired loans | Commercial and non-residential real estate | Commercial business
Classes of the loan portfolio summarized by credit quality indicators:
Total Loans	426	345
Substandard | Purchased credit impaired loans | Commercial and non-residential real estate | Commercial real estate
Classes of the loan portfolio summarized by credit quality indicators:
Total Loans	0	263
Substandard | Purchased credit impaired loans | Commercial and non-residential real estate | Acquisition and development
Classes of the loan portfolio summarized by credit quality indicators:
Total Loans	0	0
Substandard | Purchased credit impaired loans | Residential real estate
Classes of the loan portfolio summarized by credit quality indicators:
Total Loans	1,130	1,665
Substandard | Purchased credit impaired loans | Consumer
Classes of the loan portfolio summarized by credit quality indicators:
Total Loans	0	0
Doubtful
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding PCI	2,927	3,994
Doubtful | Commercial and non-residential real estate
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding PCI	2,277	3,798
Doubtful | Commercial and non-residential real estate | Commercial business
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding PCI	937	1,095
Doubtful | Commercial and non-residential real estate | Commercial real estate
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding PCI	156	533
Doubtful | Commercial and non-residential real estate | Acquisition and development
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding PCI	1,184	2,170
Doubtful | Commercial and non-residential real estate | SBA PPP
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding PCI	0	0
Doubtful | Residential real estate
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding PCI	555	170
Doubtful | Home Equity
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding PCI	95	26
Doubtful | Consumer
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding PCI	0	0
Doubtful | Purchased credit impaired loans
Classes of the loan portfolio summarized by credit quality indicators:
Total Loans	6,233	5,469
Doubtful | Purchased credit impaired loans | Commercial and non-residential real estate
Classes of the loan portfolio summarized by credit quality indicators:
Total Loans	5,256	5,116
Doubtful | Purchased credit impaired loans | Commercial and non-residential real estate | Commercial business
Classes of the loan portfolio summarized by credit quality indicators:
Total Loans	3,358	4,860
Doubtful | Purchased credit impaired loans | Commercial and non-residential real estate | Commercial real estate
Classes of the loan portfolio summarized by credit quality indicators:
Total Loans	1,663	21
Doubtful | Purchased credit impaired loans | Commercial and non-residential real estate | Acquisition and development
Classes of the loan portfolio summarized by credit quality indicators:
Total Loans	235	235
Doubtful | Purchased credit impaired loans | Residential real estate
Classes of the loan portfolio summarized by credit quality indicators:
Total Loans	975	121
Doubtful | Purchased credit impaired loans | Consumer
Classes of the loan portfolio summarized by credit quality indicators:
Total Loans	$ 2	$ 232