Exhibit 23.1

Consent of Independent Registered Public Accounting Firm

The Board of Directors

Ruth’s Hospitality Group, Inc.:

We consent to the incorporation by reference in the registration statements on Forms S-3 (No. 333-171485 and 333-160231) and Form S-8 (No. 333-127681 and 333-182845) of Ruth’s Hospitality Group, Inc. of our reports dated March 11, 2014 with respect to the consolidated balance sheets of Ruth’s Hospitality Group, Inc. and subsidiaries as of December 29, 2013 and December 30, 2012, and the related consolidated statements of income, shareholders’ equity and cash flows for each of the years in the three-year period ended December 29, 2013, and the effectiveness of internal control over financial reporting as of December 29, 2013, which reports appear in the December 29, 2013 annual report on Form 10-K of Ruth’s Hospitality Group, Inc.

Our report refers to a change in the method of accounting for the recognition of gift card breakage income.

/s/ KPMG LLP

Orlando, Florida

March 11, 2014

Certified Public Accountants