Loans and Allowance for Loan Losses - Geographic and Collateral Concentration (Details)	3 Months Ended	12 Months Ended
	Mar. 31, 2020	Dec. 31, 2019
Concentration risk
Percentage of SBA loan	100.00%	100.00%
Lodging
Concentration risk
Percentage of SBA loan	19.40%	17.30%
Offices of Physicians
Concentration risk
Percentage of SBA loan	13.90%	14.10%
Child Day Care Services
Concentration risk
Percentage of SBA loan	7.60%	8.10%
Eating Places
Concentration risk
Percentage of SBA loan	6.30%	6.10%
Veterinarians
Concentration risk
Percentage of SBA loan	3.90%	4.10%
Gasoline Service Stations
Concentration risk
Percentage of SBA loan	3.80%	3.70%
Funeral Service and Crematories
Concentration risk
Percentage of SBA loan	2.10%	2.00%
Grocery Stores
Concentration risk
Percentage of SBA loan	2.10%	2.00%
Auto
Concentration risk
Percentage of SBA loan	1.20%	1.30%
Other
Concentration risk
Percentage of SBA loan	39.70%	41.30%
Geographical concentration
Concentration risk
Percentage of loan	100.00%	100.00%
Geographical concentration | California
Concentration risk
Percentage of loan	17.70%	16.90%
Geographical concentration | Texas
Concentration risk
Percentage of loan	15.20%	15.20%
Geographical concentration | New York
Concentration risk
Percentage of loan	8.40%	8.30%
Geographical concentration | Florida
Concentration risk
Percentage of loan	8.10%	8.30%
Geographical concentration | Illinois
Concentration risk
Percentage of loan	5.30%	5.20%
Geographical concentration | Georgia
Concentration risk
Percentage of loan	4.60%	4.80%
Geographical concentration | Arizona
Concentration risk
Percentage of loan	3.40%	3.40%
Geographical concentration | North Carolina
Concentration risk
Percentage of loan	3.20%	3.20%
Geographical concentration | Washington
Concentration risk
Percentage of loan	2.90%	2.80%
Geographical concentration | Colorado
Concentration risk
Percentage of loan	2.60%	2.80%
Geographical concentration | Other
Concentration risk
Percentage of loan	28.60%	29.10%
Collateral concentration
Concentration risk
Percentage of loan	100.00%	100.00%
Collateral concentration | Multi-family
Concentration risk
Percentage of loan	26.60%	26.60%
Collateral concentration | Retail
Concentration risk
Percentage of loan	17.40%	17.50%
Collateral concentration | SBA
Concentration risk
Percentage of loan	16.20%	17.60%
Collateral concentration | Office
Concentration risk
Percentage of loan	12.20%	12.90%
Collateral concentration | Mixed Use
Concentration risk
Percentage of loan	12.00%	10.40%
Collateral concentration | Industrial
Concentration risk
Percentage of loan	7.00%	6.40%
Collateral concentration | Lodging/Residential
Concentration risk
Percentage of loan	3.20%	3.30%
Collateral concentration | Other
Concentration risk
Percentage of loan	5.40%	5.30%