Securities - Schedule of Contractual Maturity of Securities Available for Sale, Excluding Equity Securities (Details) - USD ($) $ in Thousands	Mar. 31, 2021	Dec. 31, 2020
Amortized Cost
1 year or less	$ 51,119
1 year – 5 years	103,898
5 years – 10 years	89,159
After 10 years	9,046
Total, Amortized Cost	253,222
Total debt securities	605,711	$ 565,493
Fair Value
1 year or less	51,083
1 year – 5 years	106,263
5 years – 10 years	91,195
After 10 years	9,074
Total, Fair Value	257,615
Total debt securities	613,461	584,908
Residential & multi-family mortgage
Amortized Cost
Mortgage & asset backed securities, Amortized Cost	327,536
Total debt securities	327,536	306,054
Fair Value
Mortgage & asset backed securities, Fair Value	330,291
Total debt securities	330,291	315,192
Pooled SBA
Amortized Cost
Mortgage & asset backed securities, Amortized Cost	23,933
Total debt securities	23,933	24,975
Fair Value
Mortgage & asset backed securities, Fair Value	24,589
Total debt securities	24,589	25,886
Other
Amortized Cost
Mortgage & asset backed securities, Amortized Cost	1,020
Total debt securities	1,020	1,020
Fair Value
Mortgage & asset backed securities, Fair Value	966
Total debt securities	$ 966	$ 979