Investment Securities Security Type - Available for Sale (Details) - USD ($) $ in Thousands	Mar. 31, 2020	Dec. 31, 2019
Debt Securities, Available-for-sale [Line Items]
Investment securities available for sale, amortized cost	$ 370,813	$ 397,169
Investment securities available for sale, gross unrealized gains	7,611	3,699
Investment securities available for sale, gross unrealized (losses)	(6,004)	(2,290)
Investment securities available for sale, at fair value	372,420	398,578
State and political subdivisions [Member]
Debt Securities, Available-for-sale [Line Items]
Investment securities available for sale, amortized cost	35,847	45,442
Investment securities available for sale, gross unrealized gains	1,380	1,736
Investment securities available for sale, gross unrealized (losses)	0	0
Investment securities available for sale, at fair value	37,227	47,178
Collateralized mortgage obligations [Member]
Debt Securities, Available-for-sale [Line Items]
Investment securities available for sale, amortized cost	188,882	180,899
Investment securities available for sale, gross unrealized gains	4,414	1,651
Investment securities available for sale, gross unrealized (losses)	(631)	(629)
Investment securities available for sale, at fair value	192,665	181,921
Mortgage-backed securities [Member]
Debt Securities, Available-for-sale [Line Items]
Investment securities available for sale, amortized cost	71,717	73,038
Investment securities available for sale, gross unrealized gains	1,639	225
Investment securities available for sale, gross unrealized (losses)	(40)	(233)
Investment securities available for sale, at fair value	73,316	73,030
Asset-backed securities
Debt Securities, Available-for-sale [Line Items]
Investment securities available for sale, amortized cost	16,205	17,551
Investment securities available for sale, gross unrealized gains	97	66
Investment securities available for sale, gross unrealized (losses)	0	(17)
Investment securities available for sale, at fair value	16,302	17,600
Collateralized loan obligations [Member]
Debt Securities, Available-for-sale [Line Items]
Investment securities available for sale, amortized cost	57,862	64,939
Investment securities available for sale, gross unrealized gains	81	21
Investment securities available for sale, gross unrealized (losses)	(5,333)	(128)
Investment securities available for sale, at fair value	52,610	64,832
Corporate notes
Debt Securities, Available-for-sale [Line Items]
Investment securities available for sale, amortized cost	300	15,300
Investment securities available for sale, gross unrealized gains	0	0
Investment securities available for sale, gross unrealized (losses)	0	(1,283)
Investment securities available for sale, at fair value	$ 300	$ 14,017