Exhibit 23.1

CONSENT OF ERNST & YOUNG LLP

We consent to the incorporation by reference in the Registration Statement (Form S-8) pertaining to the Oxford Industries, Inc. Employee Stock Purchase Plan of our report dated July 23, 2004, with respect to the consolidated financial statements of Oxford Industries, Inc. and Subsidiaries included in its Annual Report (Form 10-K) for the year ended May 28, 2004, filed with the Securities and Exchange Commission.

/S/ Ernst & Young LLP

Atlanta, Georgia

December 17, 2004