Loans and Allowance for Loan Losses - Internal Risk Rating Summary (Details) - USD ($) $ in Thousands	Jun. 30, 2020	Dec. 31, 2019	Jun. 30, 2019
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding acquired	$ 1,448,326	$ 1,374,235	$ 1,326,399
Total Loans	1,496,761	1,374,235
SBA Paycheck Protection Program
Classes of the loan portfolio summarized by credit quality indicators:
Total Loans	89,800
Commercial and Non-Residential Real Estate
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding acquired	1,127,792	1,063,828	984,830
Total Loans	1,127,792	1,063,828
Commercial and Non-Residential Real Estate | Commercial Business
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding acquired	481,849	540,882
Commercial and Non-Residential Real Estate | Commercial Real Estate
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding acquired	426,720	411,770
Commercial and Non-Residential Real Estate | Acquisition & Development
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding acquired	129,461	111,176
Commercial and Non-Residential Real Estate | SBA Paycheck Protection Program
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding acquired	89,762	0
Residential Real Estate
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding acquired	279,626	271,604	273,827
Total Loans	279,626	271,604
Home Equity
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding acquired	37,383	35,106	58,893
Consumer
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding acquired	3,525	3,697	$ 8,849
Total Loans	3,525	3,697
Purchased credit impaired loans
Classes of the loan portfolio summarized by credit quality indicators:
Total Loans	48,435
Purchased credit impaired loans | Commercial and Non-Residential Real Estate
Classes of the loan portfolio summarized by credit quality indicators:
Total Loans	19,799	0
Purchased credit impaired loans | Commercial and Non-Residential Real Estate | Commercial Business
Classes of the loan portfolio summarized by credit quality indicators:
Total Loans	16,450
Purchased credit impaired loans | Commercial and Non-Residential Real Estate | Commercial Real Estate
Classes of the loan portfolio summarized by credit quality indicators:
Total Loans	1,356
Purchased credit impaired loans | Commercial and Non-Residential Real Estate | Acquisition & Development
Classes of the loan portfolio summarized by credit quality indicators:
Total Loans	1,993
Purchased credit impaired loans | Residential Real Estate
Classes of the loan portfolio summarized by credit quality indicators:
Total Loans	27,033	0
Purchased credit impaired loans | Home Equity
Classes of the loan portfolio summarized by credit quality indicators:
Total Loans	0
Purchased credit impaired loans | Consumer
Classes of the loan portfolio summarized by credit quality indicators:
Total Loans	1,603	0
Pass
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding acquired	1,349,222	1,330,351
Pass | Commercial and Non-Residential Real Estate
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding acquired	1,033,477	1,024,730
Pass | Commercial and Non-Residential Real Estate | Commercial Business
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding acquired	463,570	511,590
Pass | Commercial and Non-Residential Real Estate | Commercial Real Estate
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding acquired	365,802	406,712
Pass | Commercial and Non-Residential Real Estate | Acquisition & Development
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding acquired	114,343	106,428
Pass | Commercial and Non-Residential Real Estate | SBA Paycheck Protection Program
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding acquired	89,762	0
Pass | Residential Real Estate
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding acquired	275,357	267,367
Pass | Home Equity
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding acquired	36,899	34,641
Pass | Consumer
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding acquired	3,489	3,613
Pass | Purchased credit impaired loans
Classes of the loan portfolio summarized by credit quality indicators:
Total Loans	44,957
Pass | Purchased credit impaired loans | Commercial and Non-Residential Real Estate
Classes of the loan portfolio summarized by credit quality indicators:
Total Loans	17,816
Pass | Purchased credit impaired loans | Commercial and Non-Residential Real Estate | Commercial Business
Classes of the loan portfolio summarized by credit quality indicators:
Total Loans	14,623
Pass | Purchased credit impaired loans | Commercial and Non-Residential Real Estate | Commercial Real Estate
Classes of the loan portfolio summarized by credit quality indicators:
Total Loans	1,340
Pass | Purchased credit impaired loans | Commercial and Non-Residential Real Estate | Acquisition & Development
Classes of the loan portfolio summarized by credit quality indicators:
Total Loans	1,853
Pass | Purchased credit impaired loans | Residential Real Estate
Classes of the loan portfolio summarized by credit quality indicators:
Total Loans	26,013
Pass | Purchased credit impaired loans | Home Equity
Classes of the loan portfolio summarized by credit quality indicators:
Total Loans	0
Pass | Purchased credit impaired loans | Consumer
Classes of the loan portfolio summarized by credit quality indicators:
Total Loans	1,128
Special Mention
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding acquired	74,977	25,216
Special Mention | Commercial and Non-Residential Real Estate
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding acquired	73,159	22,831
Special Mention | Commercial and Non-Residential Real Estate | Commercial Business
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding acquired	8,782	17,398
Special Mention | Commercial and Non-Residential Real Estate | Commercial Real Estate
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding acquired	53,890	3,564
Special Mention | Commercial and Non-Residential Real Estate | Acquisition & Development
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding acquired	10,487	1,869
Special Mention | Commercial and Non-Residential Real Estate | SBA Paycheck Protection Program
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding acquired	0	0
Special Mention | Residential Real Estate
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding acquired	1,400	1,946
Special Mention | Home Equity
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding acquired	382	383
Special Mention | Consumer
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding acquired	36	56
Special Mention | Purchased credit impaired loans
Classes of the loan portfolio summarized by credit quality indicators:
Total Loans	1,658
Special Mention | Purchased credit impaired loans | Commercial and Non-Residential Real Estate
Classes of the loan portfolio summarized by credit quality indicators:
Total Loans	1,425
Special Mention | Purchased credit impaired loans | Commercial and Non-Residential Real Estate | Commercial Business
Classes of the loan portfolio summarized by credit quality indicators:
Total Loans	1,425
Special Mention | Purchased credit impaired loans | Commercial and Non-Residential Real Estate | Commercial Real Estate
Classes of the loan portfolio summarized by credit quality indicators:
Total Loans	0
Special Mention | Purchased credit impaired loans | Commercial and Non-Residential Real Estate | Acquisition & Development
Classes of the loan portfolio summarized by credit quality indicators:
Total Loans	0
Special Mention | Purchased credit impaired loans | Residential Real Estate
Classes of the loan portfolio summarized by credit quality indicators:
Total Loans	0
Special Mention | Purchased credit impaired loans | Home Equity
Classes of the loan portfolio summarized by credit quality indicators:
Total Loans	0
Special Mention | Purchased credit impaired loans | Consumer
Classes of the loan portfolio summarized by credit quality indicators:
Total Loans	233
Substandard
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding acquired	24,016	18,554
Substandard | Commercial and Non-Residential Real Estate
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding acquired	21,156	16,267
Substandard | Commercial and Non-Residential Real Estate | Commercial Business
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding acquired	9,497	11,894
Substandard | Commercial and Non-Residential Real Estate | Commercial Real Estate
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding acquired	7,028	1,494
Substandard | Commercial and Non-Residential Real Estate | Acquisition & Development
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding acquired	4,631	2,879
Substandard | Commercial and Non-Residential Real Estate | SBA Paycheck Protection Program
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding acquired	0	0
Substandard | Residential Real Estate
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding acquired	2,758	2,177
Substandard | Home Equity
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding acquired	102	82
Substandard | Consumer
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding acquired	0	28
Substandard | Purchased credit impaired loans
Classes of the loan portfolio summarized by credit quality indicators:
Total Loans	1,820
Substandard | Purchased credit impaired loans | Commercial and Non-Residential Real Estate
Classes of the loan portfolio summarized by credit quality indicators:
Total Loans	558
Substandard | Purchased credit impaired loans | Commercial and Non-Residential Real Estate | Commercial Business
Classes of the loan portfolio summarized by credit quality indicators:
Total Loans	402
Substandard | Purchased credit impaired loans | Commercial and Non-Residential Real Estate | Commercial Real Estate
Classes of the loan portfolio summarized by credit quality indicators:
Total Loans	16
Substandard | Purchased credit impaired loans | Commercial and Non-Residential Real Estate | Acquisition & Development
Classes of the loan portfolio summarized by credit quality indicators:
Total Loans	140
Substandard | Purchased credit impaired loans | Residential Real Estate
Classes of the loan portfolio summarized by credit quality indicators:
Total Loans	1,020
Substandard | Purchased credit impaired loans | Home Equity
Classes of the loan portfolio summarized by credit quality indicators:
Total Loans	0
Substandard | Purchased credit impaired loans | Consumer
Classes of the loan portfolio summarized by credit quality indicators:
Total Loans	242
Doubtful
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding acquired	111	114
Doubtful | Commercial and Non-Residential Real Estate
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding acquired	0	0
Doubtful | Commercial and Non-Residential Real Estate | Commercial Business
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding acquired	0	0
Doubtful | Commercial and Non-Residential Real Estate | Commercial Real Estate
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding acquired	0	0
Doubtful | Commercial and Non-Residential Real Estate | Acquisition & Development
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding acquired	0	0
Doubtful | Commercial and Non-Residential Real Estate | SBA Paycheck Protection Program
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding acquired	0	0
Doubtful | Residential Real Estate
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding acquired	111	114
Doubtful | Home Equity
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding acquired	0	0
Doubtful | Consumer
Classes of the loan portfolio summarized by credit quality indicators:
Total loans, excluding acquired	0	$ 0
Doubtful | Purchased credit impaired loans
Classes of the loan portfolio summarized by credit quality indicators:
Total Loans	0
Doubtful | Purchased credit impaired loans | Commercial and Non-Residential Real Estate
Classes of the loan portfolio summarized by credit quality indicators:
Total Loans	0
Doubtful | Purchased credit impaired loans | Commercial and Non-Residential Real Estate | Commercial Business
Classes of the loan portfolio summarized by credit quality indicators:
Total Loans	0
Doubtful | Purchased credit impaired loans | Commercial and Non-Residential Real Estate | Commercial Real Estate
Classes of the loan portfolio summarized by credit quality indicators:
Total Loans	0
Doubtful | Purchased credit impaired loans | Commercial and Non-Residential Real Estate | Acquisition & Development
Classes of the loan portfolio summarized by credit quality indicators:
Total Loans	0
Doubtful | Purchased credit impaired loans | Residential Real Estate
Classes of the loan portfolio summarized by credit quality indicators:
Total Loans	0
Doubtful | Purchased credit impaired loans | Home Equity
Classes of the loan portfolio summarized by credit quality indicators:
Total Loans	0
Doubtful | Purchased credit impaired loans | Consumer
Classes of the loan portfolio summarized by credit quality indicators:
Total Loans	$ 0