Loans and Allowance for Loan Losses - Narrative (Details)	3 Months Ended			9 Months Ended
	Sep. 30, 2020 USD ($) loan	Sep. 30, 2019 USD ($) loan	Mar. 31, 2018 USD ($)	Sep. 30, 2020 USD ($) loan	Sep. 30, 2019 USD ($) loan	Sep. 30, 2020 loan	Sep. 30, 2020 hotel	Jun. 30, 2020 USD ($) loan	Dec. 31, 2019 USD ($) loan
Loans and Leases Receivable Disclosure [Line Items]
Servicing agent fee, percentage				0.75%
Financing receivable, before allowance for credit loss	$ 1,050,885,000			$ 1,050,885,000					$ 952,496,000
Total unamortized net deferred loan fees	2,500,000			2,500,000					907,000
Real estate loans services for others	106,000,000.0			106,000,000.0					100,000,000.0
Mortgage loans in process of foreclosure, amount	805,000,000			805,000,000					$ 1,100,000
Financing receivable, TDRs, number of contracts | loan						18			16
Financing receivable, troubled debt restructuring	$ 3,600,000			$ 3,600,000					$ 3,000,000.0
Financing receivable, troubled debt restructuring, subsequent default, number of contracts | loan	0	0		0	0
Increase (decrease) in impaired financing receivable, recorded investment				$ 26,900,000
Impaired financing receivable, with no related allowance, recorded investment	$ 23,634,000			23,634,000					3,740,000
Provision for Loan Losses	1,200,000	$ 175,000		4,000,000	$ 550,000
Charge-offs	114,000	209,000		275,000	538,000
Non-TDR Loan Modifications, CARES Act
Loans and Leases Receivable Disclosure [Line Items]
Number of loans in forbearance | loan						34		527
Financing receivable, before allowance for credit loss, in forbearance	22,706,000			$ 22,706,000				$ 165,936,000
COVID-19
Loans and Leases Receivable Disclosure [Line Items]
Financing receivable, non-TDR modification period, deferred payments				3 months
Financing receivable, non-TDR modification period, interest only payments				3 months
Financing receivable, non-TDR modification period, maturity date extension				3 months
Provision for Loan Losses	1,200,000			$ 4,000,000.0
Unlikely to be Collected Financing Receivable
Loans and Leases Receivable Disclosure [Line Items]
Financing receivable, before allowance for credit loss	0			0					0
Special Mention
Loans and Leases Receivable Disclosure [Line Items]
Financing receivable, before allowance for credit loss	38,084,000			38,084,000					24,585,000
Increase (decrease) in finance receivables				13,500,000
Substandard
Loans and Leases Receivable Disclosure [Line Items]
Financing receivable, before allowance for credit loss	18,189,000			18,189,000					7,383,000
Increase (decrease) in finance receivables				$ 10,800,000
Minimum
Loans and Leases Receivable Disclosure [Line Items]
Loan processing fee, percentage				1.00%
Maximum
Loans and Leases Receivable Disclosure [Line Items]
Loan processing fee, percentage				5.00%
Real Estate | Commercial
Loans and Leases Receivable Disclosure [Line Items]
Financing receivable, before allowance for credit loss	353,904,000			$ 353,904,000					351,360,000
Impaired financing receivable, with no related allowance, recorded investment	20,896,000			20,896,000					3,058,000
Provision for Loan Losses	1,711,000	(508,000)		3,634,000	224,000
Charge-offs	0	0		0	$ 0
Real Estate | Commercial | Non-TDR Loan Modifications, CARES Act
Loans and Leases Receivable Disclosure [Line Items]
Number of loans in forbearance | loan						9		111
Financing receivable, before allowance for credit loss, in forbearance	13,885,000			13,885,000				$ 105,117,000
Real Estate | Commercial | Hotel
Loans and Leases Receivable Disclosure [Line Items]
Impaired financing receivable, with no related allowance, recorded investment									16,100,000
Real Estate | Commercial | Hotel | Non-TDR Loan Modifications, CARES Act
Loans and Leases Receivable Disclosure [Line Items]
Number of loans in forbearance | loan								5
Financing receivable, before allowance for credit loss, in forbearance								$ 10,300,000
Real Estate | Commercial | COVID-19 | Hotel
Loans and Leases Receivable Disclosure [Line Items]
Financing receivable, before allowance for credit loss									7,900,000
Financing receivable nonacrrual, number of loans impaired						2	2
Increase (decrease) in impaired financing receivable, unpaid principal balance				7,900,000
Real Estate | Commercial | Special Mention
Loans and Leases Receivable Disclosure [Line Items]
Financing receivable, before allowance for credit loss	29,383,000			29,383,000					12,260,000
Real Estate | Commercial | Substandard
Loans and Leases Receivable Disclosure [Line Items]
Financing receivable, before allowance for credit loss	13,396,000			13,396,000					3,664,000
Real Estate | Residential
Loans and Leases Receivable Disclosure [Line Items]
Financing receivable, before allowance for credit loss	343,955,000			$ 343,955,000					347,766,000
Financing receivable, modifications, number of contracts paid off | loan				1	1
Financing receivable, modifications, troubled debt restructuring, paid off				$ 60,000	$ 851,000
Impaired financing receivable, with no related allowance, recorded investment	1,216,000			1,216,000					549,000
Provision for Loan Losses	(506,000)	582,000		180,000	662,000
Charge-offs	11,000	28,000		36,000	71,000
Real Estate | Residential | Non-TDR Loan Modifications, CARES Act
Loans and Leases Receivable Disclosure [Line Items]
Number of loans in forbearance | loan						11		163
Financing receivable, before allowance for credit loss, in forbearance	1,242,000			1,242,000				$ 23,653,000
Real Estate | Residential | Special Mention
Loans and Leases Receivable Disclosure [Line Items]
Financing receivable, before allowance for credit loss	1,045,000			1,045,000					1,997,000
Real Estate | Residential | Substandard
Loans and Leases Receivable Disclosure [Line Items]
Financing receivable, before allowance for credit loss	2,489,000			2,489,000					1,918,000
Real Estate | Construction
Loans and Leases Receivable Disclosure [Line Items]
Financing receivable, before allowance for credit loss	69,178,000			69,178,000					35,605,000
Impaired financing receivable, with no related allowance, recorded investment	641,000			641,000
Provision for Loan Losses	71,000	49,000		606,000	142,000
Charge-offs	0	0		0	0
Real Estate | Construction | Non-TDR Loan Modifications, CARES Act
Loans and Leases Receivable Disclosure [Line Items]
Number of loans in forbearance | loan						1		6
Financing receivable, before allowance for credit loss, in forbearance	7,162,000			7,162,000				$ 15,518,000
Real Estate | Construction | Special Mention
Loans and Leases Receivable Disclosure [Line Items]
Financing receivable, before allowance for credit loss	3,379,000			3,379,000					2,263,000
Real Estate | Construction | Substandard
Loans and Leases Receivable Disclosure [Line Items]
Financing receivable, before allowance for credit loss	641,000			641,000					0
Commercial and Industrial
Loans and Leases Receivable Disclosure [Line Items]
Financing receivable, before allowance for credit loss	144,315,000			144,315,000					85,586,000
Increase (decrease) in impaired financing receivable, unpaid principal balance				1,400,000
Impaired financing receivable, with no related allowance, recorded investment	881,000			881,000					133,000
Provision for Loan Losses	170,000	(278,000)		(691,000)	(369,000)
Charge-offs	0	16,000	$ 1,400,000	0	16,000
Commercial and Industrial | Non-TDR Loan Modifications, CARES Act
Loans and Leases Receivable Disclosure [Line Items]
Number of loans in forbearance | loan						1		76
Financing receivable, before allowance for credit loss, in forbearance	122,000			$ 122,000				$ 15,697,000
Commercial and Industrial | Paycheck Protection Program, CARES Act
Loans and Leases Receivable Disclosure [Line Items]
Financing receivable, number of contracts | loan				638
Financing receivable, before allowance for credit loss	71,000,000.0			$ 71,000,000.0
Financing receivable, before allowance for credit loss, median loan	35,000			35,000
Financing receivable, unamortized loan fee	2,200,000			2,200,000
Amortization of deferred loan origination fees, net	274,000			465,000
Commercial and Industrial | Paycheck Protection Program, CARES Act | Healthcare Sector
Loans and Leases Receivable Disclosure [Line Items]
Financing receivable, before allowance for credit loss	15,600,000			15,600,000
Commercial and Industrial | Paycheck Protection Program, CARES Act | Construction and Specialty-Trade Contractors Sector
Loans and Leases Receivable Disclosure [Line Items]
Financing receivable, before allowance for credit loss	12,600,000			12,600,000
Commercial and Industrial | Paycheck Protection Program, CARES Act | Professional and Technical Services Sector
Loans and Leases Receivable Disclosure [Line Items]
Financing receivable, before allowance for credit loss	6,100,000			6,100,000
Commercial and Industrial | Paycheck Protection Program, CARES Act | Retail Sector
Loans and Leases Receivable Disclosure [Line Items]
Financing receivable, before allowance for credit loss	6,100,000			6,100,000
Commercial and Industrial | Paycheck Protection Program, CARES Act | Wholesale Trade Sector
Loans and Leases Receivable Disclosure [Line Items]
Financing receivable, before allowance for credit loss	5,100,000			5,100,000
Commercial and Industrial | Paycheck Protection Program, CARES Act | Manufacturing Sector
Loans and Leases Receivable Disclosure [Line Items]
Financing receivable, before allowance for credit loss	4,600,000			4,600,000
Commercial and Industrial | Paycheck Protection Program, CARES Act | Restaurant and Food Service Sector
Loans and Leases Receivable Disclosure [Line Items]
Financing receivable, before allowance for credit loss	3,400,000			3,400,000
Commercial and Industrial | Special Mention
Loans and Leases Receivable Disclosure [Line Items]
Financing receivable, before allowance for credit loss	4,197,000			4,197,000					7,975,000
Commercial and Industrial | Substandard
Loans and Leases Receivable Disclosure [Line Items]
Financing receivable, before allowance for credit loss	1,614,000			1,614,000					1,691,000
Consumer
Loans and Leases Receivable Disclosure [Line Items]
Financing receivable, before allowance for credit loss	117,364,000			117,364,000					113,637,000
Provision for Loan Losses	(290,000)	76,000		47,000	(220,000)
Charge-offs	103,000	$ 165,000		239,000	$ 451,000
Consumer | Non-TDR Loan Modifications, CARES Act
Loans and Leases Receivable Disclosure [Line Items]
Number of loans in forbearance | loan						12		170
Financing receivable, before allowance for credit loss, in forbearance	295,000			295,000				$ 3,447,000
Consumer | Special Mention
Loans and Leases Receivable Disclosure [Line Items]
Financing receivable, before allowance for credit loss	0			0					0
Consumer | Substandard
Loans and Leases Receivable Disclosure [Line Items]
Financing receivable, before allowance for credit loss	$ 49,000			$ 49,000					$ 110,000