Securities - Schedule of Contractual Maturity of Securities Available for Sale, Excluding Equity Securities (Details) - USD ($) $ in Thousands	Jun. 30, 2020	Dec. 31, 2019
Amortized Cost
1 year or less	$ 53,898
1 year – 5 years	85,070
5 years – 10 years	65,052
After 10 years	6,294
Total, Amortized Cost	210,314
Total debt securities	527,336	$ 533,203
Fair Value
1 year or less	54,102
1 year – 5 years	89,212
5 years – 10 years	70,171
After 10 years	6,497
Total, Fair Value	219,982
Total debt securities	549,094	542,313
Residential & multi-family mortgage
Amortized Cost
Mortgage & asset backed securities, Amortized Cost	293,605
Total debt securities	293,605	273,404
Fair Value
Mortgage & asset backed securities, Fair Value	304,771
Total debt securities	304,771	276,636
Pooled SBA
Amortized Cost
Mortgage & asset backed securities, Amortized Cost	22,397
Total debt securities	22,397	25,063
Fair Value
Mortgage & asset backed securities, Fair Value	23,358
Total debt securities	23,358	25,174
Other
Amortized Cost
Mortgage & asset backed securities, Amortized Cost	1,020
Total debt securities	1,020	1,020
Fair Value
Mortgage & asset backed securities, Fair Value	983
Total debt securities	$ 983	$ 964