Exhibit 23.1

CONSENT OF INDEPENDENT REGISTERED PUBLIC ACCOUNTING FIRM

We consent to the incorporation by reference in Registration Statement Nos. 333-114933, 333-133846, 333-152043, 333-167728, and 333-175911 each on Form S-8 of our reports dated April 16, 2013, relating to the consolidated financial statements and financial statement schedule of Perry Ellis International, Inc., and the effectiveness of Perry Ellis International Inc.’s internal control over financial reporting, appearing in this Annual Report on Form 10-K of Perry Ellis International, Inc. for the year ended February 2, 2013.

/s/ Deloitte & Touche LLP

Miami, Florida

April 16, 2013