EXHIBIT 23.1

CONSENT OF INDEPENDENT REGISTERED PUBLIC ACCOUNTING FIRM

We consent to the incorporation by reference in Registration Statement No. 333-191917 on Form S-8 of our report dated March 5, 2014, relating to the consolidated financial statements of Potbelly Corporation and subsidiaries appearing in this Annual Report on Form 10-K of Potbelly Corporation and subsidiaries for the year ended December 31, 2013.

/s/ Deloitte & Touche LLP

Chicago, Illinois

March 5, 2014