Exhibit 23.1

Consent of Independent Registered Public Accounting Firm

We consent to the reference to our firm under the caption “Experts” in this Registration Statement (Form S-3) and related Prospectus of City Holding Company and subsidiaries for the registration of $100,000,000 aggregate amount of its common stock, preferred stock, warrants to purchase common or preferred stock and units that include any of these securities, and to the incorporation by reference therein of our report dated March 15, 2012, with respect to the consolidated financial statements of City Holding Company, and the effectiveness of internal control over financial reporting of City Holding Company, incorporated by reference in its Annual Report (Form 10-K) for the year ended December 31, 2011, filed with the Securities and Exchange Commission.

/s/ Ernst & Young LLP

Charleston, West Virginia

December 5, 2012