Portfolio Loans - Summary of Recorded Investment in Impaired Portfolio Loans by Category (Details) - Non-Covered Loans - USD ($) $ in Thousands	Dec. 31, 2018	Dec. 31, 2017
Financing Receivable, Recorded Investment, Past Due [Line Items]
Non-accrual	$ 16,520	$ 14,968
Restructured, not on non-accrual	225	719
Total Recorded Investment	16,745	15,687
Commercial and industrial
Financing Receivable, Recorded Investment, Past Due [Line Items]
Non-accrual	12,805	11,946
Restructured, not on non-accrual	145	719
Total Recorded Investment	12,950	12,665
CRE - investor owned
Financing Receivable, Recorded Investment, Past Due [Line Items]
Non-accrual	398	422
Restructured, not on non-accrual	0	0
Total Recorded Investment	398	422
CRE - owner occupied
Financing Receivable, Recorded Investment, Past Due [Line Items]
Non-accrual	808	487
Restructured, not on non-accrual	0	0
Total Recorded Investment	808	487
Construction and land development
Financing Receivable, Recorded Investment, Past Due [Line Items]
Non-accrual	0	136
Restructured, not on non-accrual	0	0
Total Recorded Investment	0	136
Residential real estate
Financing Receivable, Recorded Investment, Past Due [Line Items]
Non-accrual	2,197	1,602
Restructured, not on non-accrual	80	0
Total Recorded Investment	2,277	1,602
Consumer and other
Financing Receivable, Recorded Investment, Past Due [Line Items]
Non-accrual	312	375
Restructured, not on non-accrual	0	0
Total Recorded Investment	$ 312	$ 375