Exhibit 23

Consent of Independent Registered Public Accounting Firm

We consent to the incorporation by reference in the following Registration Statements:

(1) Registration Statement (Form S-8 No. 33-335311) of Harley-Davidson, Inc. pertaining to the Harley-Davidson Retirement Savings Plan for Salaried Employees, the Harley-Davidson Retirement Savings Plan for Milwaukee and Tomahawk Hourly Bargaining Unit Employees, and the Holiday Rambler LLC Employees Retirement Plan;

(2) Registration Statement (Form S-8 No. 333-07551) pertaining to the Harley-Davidson, Inc. 1995 Stock Option Plan;

(3) Registration Statement (Form S-8 No. 333-51741) pertaining to the Harley-Davidson, Inc. Director Stock Plan;

(4) Registration Statement (Form S-8 No. 333-75347) pertaining to the Harley-Davidson, Inc. 1998 Non-Exempt Employee Stock Option Plan;

(5) Registration Statement (Form S-8 Nos. 333-93879 and 333-123406) of Harley-Davidson, Inc. pertaining to the Harley-Davidson Retirement Savings Plan for Salaried Employees, the Harley-Davidson Retirement Savings Plan for Milwaukee and Tomahawk Hourly Bargaining Unit Employees, the Harley-Davidson Retirement Savings plan for Kansas City Hourly Bargaining Unit Employees, the Harley-Davidson Retirement Savings Plan for York Hourly Bargaining Unit Employees and the Buell Motorcycle Company Retirements Savings Plan;

(6) Registration Statement (Form S-8 No. 333-60840) pertaining to the Harley-Davidson, Inc. 2001 York Hourly-Paid Employees Stock Option Plan;

(7) Registration Statement (Form S-8 No. 333-123405) pertaining to the Harley-Davidson, Inc. 2004 Incentive Stock Plan;

(8) Registration Statement (Form S-3 No. 333-156060) of Harley-Davidson, Inc. and the related Prospectus;

(9) Registration Statement (Form S-8 No.333-166549) pertaining to the Harley-Davidson, Inc. 2009 Incentive Stock Plan; and

(10) Registration Statement (Form S-8 No. 333-171813) pertaining to the Harley-Davidson, Inc. Stock Purchase Plan;

(11) Registration Statement (Form S-3 No.333-177767) of Harley-Davidson, Inc. and the related Prospectus;

of our reports dated February 23, 2012, with respect to the consolidated financial statements and schedule of Harley-Davidson, Inc. and the effectiveness of internal control over financial reporting of Harley-Davidson, Inc., included in this Annual Report (Form 10-K) for the year ended December 31, 2011.

/s/ Ernst & Young LLP

Milwaukee, WI

February 23, 2012