Exhibit 23.1

CONSENT OF INDEPENDENT REGISTERED PUBLIC ACCOUNTING FIRM

We consent to the incorporation by reference in the Registration Statement (Form S-8 No. 333-170414) pertaining to the 2010 Omnibus Incentive Compensation Plan, the Employee Stock Purchase Plan, and the Deferred Compensation Plan Amended and Restated Effective March 1, 2010 of The Fresh Market, Inc. of our report dated March 22, 2011, with respect to the financial statements of The Fresh Market, Inc. included in this Annual Report (Form 10-K) for the year ended December 31, 2010 and the one month period ended January 30, 2011.

/s/ Ernst & Young LLP

Greensboro, North Carolina

March 22, 2011