First Guaranty Bancshares, Inc. S-4

Exhibit 23.4

Consent of Independent Registered Public Accounting Firm

We consent to the incorporation by reference in the Registration Statement on Form S-4 of First Guaranty Bancshares, Inc. of our report dated March 16, 2022, with respect to the consolidated financial statements of First Guaranty Bancshares, Inc. and subsidiary for the year end December 31, 2021, included in the Annual Report on Form 10-K of First Guaranty Bancshares, Inc. for the year ended December 31, 2022, filed with the Securities and Exchange Commission.

We also consent to the reference to our firm under the caption “Experts” in the Proxy Statement/Prospectus, which is part of this Registration Statement.

/s/ Castaing, Hussey & Lolan, LLC

New Iberia, Louisiana

March 23, 2023