Credit Quality and Nonperforming Assets - Credit Quality Classifications (Details) - USD ($) $ in Thousands	Sep. 30, 2019	Dec. 31, 2018
Financing Receivable Recorded Investment [Line Items]
Total gross loans and leases	$ 1,797,660	$ 1,731,928
Real Estate Sector [Member]
Financing Receivable Recorded Investment [Line Items]
Total gross loans and leases	1,407,754	1,453,930
Real Estate Sector [Member] | 1-4 Family Residential Construction [Member]
Financing Receivable Recorded Investment [Line Items]
Total gross loans and leases	115,821	105,676
Real Estate Sector [Member] | Other construction/land [Member]
Financing Receivable Recorded Investment [Line Items]
Total gross loans and leases	93,117	109,023
Real Estate Sector [Member] | 1-4 Family - Closed-End [Member]
Financing Receivable Recorded Investment [Line Items]
Total gross loans and leases	210,727	236,825
Real Estate Sector [Member] | Equity Lines [Member]
Financing Receivable Recorded Investment [Line Items]
Total gross loans and leases	51,207	56,320
Real Estate Sector [Member] | Multi-Family Residential [Member]
Financing Receivable Recorded Investment [Line Items]
Total gross loans and leases	52,302	54,877
Real Estate Sector [Member] | Commercial Real Estate Owner Occupied [Member]
Financing Receivable Recorded Investment [Line Items]
Total gross loans and leases	322,538	301,324
Real Estate Sector [Member] | Commercial Real Estate Non Owner Occupied [Member]
Financing Receivable Recorded Investment [Line Items]
Total gross loans and leases	417,424	438,344
Real Estate Sector [Member] | Farmland [Member]
Financing Receivable Recorded Investment [Line Items]
Total gross loans and leases	144,618	151,541
Agricultural Sector [Member]
Financing Receivable Recorded Investment [Line Items]
Total gross loans and leases	49,105	49,103
Commercial and Industrial Sector [Member]
Financing Receivable Recorded Investment [Line Items]
Total gross loans and leases	115,737	128,220
Mortgage Warehouse Sector [Member]
Financing Receivable Recorded Investment [Line Items]
Total gross loans and leases	216,913	91,813
Consumer Loans Sector [Member]
Financing Receivable Recorded Investment [Line Items]
Total gross loans and leases	8,151	8,862
Pass [Member]
Financing Receivable Recorded Investment [Line Items]
Total gross loans and leases	1,740,032	1,677,847
Pass [Member] | Real Estate Sector [Member]
Financing Receivable Recorded Investment [Line Items]
Total gross loans and leases	1,375,495	1,419,253
Pass [Member] | Real Estate Sector [Member] | 1-4 Family Residential Construction [Member]
Financing Receivable Recorded Investment [Line Items]
Total gross loans and leases	115,821	105,676
Pass [Member] | Real Estate Sector [Member] | Other construction/land [Member]
Financing Receivable Recorded Investment [Line Items]
Total gross loans and leases	92,008	108,304
Pass [Member] | Real Estate Sector [Member] | 1-4 Family - Closed-End [Member]
Financing Receivable Recorded Investment [Line Items]
Total gross loans and leases	204,871	230,022
Pass [Member] | Real Estate Sector [Member] | Equity Lines [Member]
Financing Receivable Recorded Investment [Line Items]
Total gross loans and leases	44,352	49,346
Pass [Member] | Real Estate Sector [Member] | Multi-Family Residential [Member]
Financing Receivable Recorded Investment [Line Items]
Total gross loans and leases	51,945	54,504
Pass [Member] | Real Estate Sector [Member] | Commercial Real Estate Owner Occupied [Member]
Financing Receivable Recorded Investment [Line Items]
Total gross loans and leases	310,413	292,886
Pass [Member] | Real Estate Sector [Member] | Commercial Real Estate Non Owner Occupied [Member]
Financing Receivable Recorded Investment [Line Items]
Total gross loans and leases	412,689	429,835
Pass [Member] | Real Estate Sector [Member] | Farmland [Member]
Financing Receivable Recorded Investment [Line Items]
Total gross loans and leases	143,396	148,680
Pass [Member] | Agricultural Sector [Member]
Financing Receivable Recorded Investment [Line Items]
Total gross loans and leases	39,165	48,517
Pass [Member] | Commercial and Industrial Sector [Member]
Financing Receivable Recorded Investment [Line Items]
Total gross loans and leases	101,108	110,413
Pass [Member] | Mortgage Warehouse Sector [Member]
Financing Receivable Recorded Investment [Line Items]
Total gross loans and leases	216,913	91,813
Pass [Member] | Consumer Loans Sector [Member]
Financing Receivable Recorded Investment [Line Items]
Total gross loans and leases	7,351	7,851
Special Mention [Member]
Financing Receivable Recorded Investment [Line Items]
Total gross loans and leases	35,666	28,698
Special Mention [Member] | Real Estate Sector [Member]
Financing Receivable Recorded Investment [Line Items]
Total gross loans and leases	12,494	12,281
Special Mention [Member] | Real Estate Sector [Member] | 1-4 Family Residential Construction [Member]
Financing Receivable Recorded Investment [Line Items]
Total gross loans and leases	0	0
Special Mention [Member] | Real Estate Sector [Member] | Other construction/land [Member]
Financing Receivable Recorded Investment [Line Items]
Total gross loans and leases	704	231
Special Mention [Member] | Real Estate Sector [Member] | 1-4 Family - Closed-End [Member]
Financing Receivable Recorded Investment [Line Items]
Total gross loans and leases	1,618	1,861
Special Mention [Member] | Real Estate Sector [Member] | Equity Lines [Member]
Financing Receivable Recorded Investment [Line Items]
Total gross loans and leases	1,971	2,194
Special Mention [Member] | Real Estate Sector [Member] | Multi-Family Residential [Member]
Financing Receivable Recorded Investment [Line Items]
Total gross loans and leases	0	0
Special Mention [Member] | Real Estate Sector [Member] | Commercial Real Estate Owner Occupied [Member]
Financing Receivable Recorded Investment [Line Items]
Total gross loans and leases	5,751	4,192
Special Mention [Member] | Real Estate Sector [Member] | Commercial Real Estate Non Owner Occupied [Member]
Financing Receivable Recorded Investment [Line Items]
Total gross loans and leases	1,390	2,730
Special Mention [Member] | Real Estate Sector [Member] | Farmland [Member]
Financing Receivable Recorded Investment [Line Items]
Total gross loans and leases	1,060	1,073
Special Mention [Member] | Agricultural Sector [Member]
Financing Receivable Recorded Investment [Line Items]
Total gross loans and leases	9,934	580
Special Mention [Member] | Commercial and Industrial Sector [Member]
Financing Receivable Recorded Investment [Line Items]
Total gross loans and leases	13,162	15,686
Special Mention [Member] | Mortgage Warehouse Sector [Member]
Financing Receivable Recorded Investment [Line Items]
Total gross loans and leases	0	0
Special Mention [Member] | Consumer Loans Sector [Member]
Financing Receivable Recorded Investment [Line Items]
Total gross loans and leases	76	151
Substandard [Member]
Financing Receivable Recorded Investment [Line Items]
Total gross loans and leases	6,175	9,311
Substandard [Member] | Real Estate Sector [Member]
Financing Receivable Recorded Investment [Line Items]
Total gross loans and leases	5,500	8,895
Substandard [Member] | Real Estate Sector [Member] | 1-4 Family Residential Construction [Member]
Financing Receivable Recorded Investment [Line Items]
Total gross loans and leases	0	0
Substandard [Member] | Real Estate Sector [Member] | Other construction/land [Member]
Financing Receivable Recorded Investment [Line Items]
Total gross loans and leases	0	0
Substandard [Member] | Real Estate Sector [Member] | 1-4 Family - Closed-End [Member]
Financing Receivable Recorded Investment [Line Items]
Total gross loans and leases	424	1,310
Substandard [Member] | Real Estate Sector [Member] | Equity Lines [Member]
Financing Receivable Recorded Investment [Line Items]
Total gross loans and leases	89	64
Substandard [Member] | Real Estate Sector [Member] | Multi-Family Residential [Member]
Financing Receivable Recorded Investment [Line Items]
Total gross loans and leases	0	0
Substandard [Member] | Real Estate Sector [Member] | Commercial Real Estate Owner Occupied [Member]
Financing Receivable Recorded Investment [Line Items]
Total gross loans and leases	4,324	3,021
Substandard [Member] | Real Estate Sector [Member] | Commercial Real Estate Non Owner Occupied [Member]
Financing Receivable Recorded Investment [Line Items]
Total gross loans and leases	526	4,354
Substandard [Member] | Real Estate Sector [Member] | Farmland [Member]
Financing Receivable Recorded Investment [Line Items]
Total gross loans and leases	137	146
Substandard [Member] | Agricultural Sector [Member]
Financing Receivable Recorded Investment [Line Items]
Total gross loans and leases	0	0
Substandard [Member] | Commercial and Industrial Sector [Member]
Financing Receivable Recorded Investment [Line Items]
Total gross loans and leases	657	377
Substandard [Member] | Mortgage Warehouse Sector [Member]
Financing Receivable Recorded Investment [Line Items]
Total gross loans and leases	0	0
Substandard [Member] | Consumer Loans Sector [Member]
Financing Receivable Recorded Investment [Line Items]
Total gross loans and leases	18	39
Impaired [Member]
Financing Receivable Recorded Investment [Line Items]
Total gross loans and leases	15,787	16,072
Impaired [Member] | Real Estate Sector [Member]
Financing Receivable Recorded Investment [Line Items]
Total gross loans and leases	14,265	13,501
Impaired [Member] | Real Estate Sector [Member] | 1-4 Family Residential Construction [Member]
Financing Receivable Recorded Investment [Line Items]
Total gross loans and leases	0	0
Impaired [Member] | Real Estate Sector [Member] | Other construction/land [Member]
Financing Receivable Recorded Investment [Line Items]
Total gross loans and leases	405	488
Impaired [Member] | Real Estate Sector [Member] | 1-4 Family - Closed-End [Member]
Financing Receivable Recorded Investment [Line Items]
Total gross loans and leases	3,814	3,632
Impaired [Member] | Real Estate Sector [Member] | Equity Lines [Member]
Financing Receivable Recorded Investment [Line Items]
Total gross loans and leases	4,795	4,716
Impaired [Member] | Real Estate Sector [Member] | Multi-Family Residential [Member]
Financing Receivable Recorded Investment [Line Items]
Total gross loans and leases	357	373
Impaired [Member] | Real Estate Sector [Member] | Commercial Real Estate Owner Occupied [Member]
Financing Receivable Recorded Investment [Line Items]
Total gross loans and leases	2,050	1,225
Impaired [Member] | Real Estate Sector [Member] | Commercial Real Estate Non Owner Occupied [Member]
Financing Receivable Recorded Investment [Line Items]
Total gross loans and leases	2,819	1,425
Impaired [Member] | Real Estate Sector [Member] | Farmland [Member]
Financing Receivable Recorded Investment [Line Items]
Total gross loans and leases	25	1,642
Impaired [Member] | Agricultural Sector [Member]
Financing Receivable Recorded Investment [Line Items]
Total gross loans and leases	6	6
Impaired [Member] | Commercial and Industrial Sector [Member]
Financing Receivable Recorded Investment [Line Items]
Total gross loans and leases	810	1,744
Impaired [Member] | Mortgage Warehouse Sector [Member]
Financing Receivable Recorded Investment [Line Items]
Total gross loans and leases	0	0
Impaired [Member] | Consumer Loans Sector [Member]
Financing Receivable Recorded Investment [Line Items]
Total gross loans and leases	$ 706	$ 821