Income Taxes - Additional Information (Details) - USD ($)	12 Months Ended
	Dec. 31, 2020	Dec. 31, 2019	Dec. 31, 2018	Dec. 31, 2017
Operating Loss Carryforwards [Line Items]
Unrealized gains (loss) on securities, tax expense	$ 721,000	$ 874,000	$ 1,100,000
Purchase accounting adjustments	(602,000)	8,475,000
Retained earnings not provided for provision for income tax	10,800,000
Unrecognized deferred tax liability	2,800,000
Unrecognized tax benefits	0	0	0
Income tax benefit from revaluation of state deferred tax assets		2,200,000
Tax cuts and jobs act of 2017, change in tax rate, income tax expense (benefit)			$ 1,900,000	$ 3,600,000
Colonial American Bank
Operating Loss Carryforwards [Line Items]
Operating loss carryforwards	4,600,000	4,900,000
Operating loss carryforwards, subject to expiration	330,000
Cape Bancorp, Inc.
Operating Loss Carryforwards [Line Items]
Tax credit	0	1,000,000.0
Sun Bancorp, Inc.
Operating Loss Carryforwards [Line Items]
Purchase accounting adjustments	4,500,000
Operating loss carryforwards	152,600,000	175,900,000
Operating loss carryforwards, subject to expiration	23,300,000	9,300,000
Tax credit	2,300,000	2,300,000
New Jersey
Operating Loss Carryforwards [Line Items]
Tax credit	1,800,000	$ 1,800,000
Other Comprehensive Income
Operating Loss Carryforwards [Line Items]
Tax cuts and jobs act of 2017, change in tax rate, income tax expense (benefit)	1,800,000
Accumulated Other Comprehensive (Loss) Gain
Operating Loss Carryforwards [Line Items]
Tax cuts and jobs act of 2017, change in tax rate, income tax expense (benefit)	$ 786,000