Income Taxes - Summary of loss carryforwards by jurisdiction (Details) - USD ($) $ in Thousands	Dec. 31, 2019	Dec. 31, 2018
Federal
Operating Loss Carryforwards [Line Items]
Net operating loss carry-forwards, remaining amount	$ 7,737	$ 8,877
Net operating loss carry-forwards, annual limitation	646	1,143
Federal | Saehan Bank (acquired by Wilshire) | Ownership Change
Operating Loss Carryforwards [Line Items]
Net operating loss carry-forwards, remaining amount	2,488	2,714
Net operating loss carry-forwards, annual limitation	226	226
Federal | Korea First Bank of New York
Operating Loss Carryforwards [Line Items]
Net operating loss carry-forwards, remaining amount		494
Net operating loss carry-forwards, annual limitation		497
Federal | Pacific International Bank
Operating Loss Carryforwards [Line Items]
Net operating loss carry-forwards, remaining amount	5,249	5,669
Net operating loss carry-forwards, annual limitation	420	420
State
Operating Loss Carryforwards [Line Items]
Net operating loss carry-forwards, remaining amount	2,488	2,714
Net operating loss carry-forwards, annual limitation	226	226
State | Saehan Bank (acquired by Wilshire) | Ownership Change
Operating Loss Carryforwards [Line Items]
Net operating loss carry-forwards, remaining amount	2,488	2,714
Net operating loss carry-forwards, annual limitation	226	226
State | Korea First Bank of New York
Operating Loss Carryforwards [Line Items]
Net operating loss carry-forwards, remaining amount		0
Net operating loss carry-forwards, annual limitation		0
State | Pacific International Bank
Operating Loss Carryforwards [Line Items]
Net operating loss carry-forwards, remaining amount	0	0
Net operating loss carry-forwards, annual limitation	$ 0	$ 0