EXHIBIT 23

Consent of Independent Registered Public Accounting Firm

We consent to the incorporation by reference in

(i) Post-Effective Amendment No. 2 to Registration Statement (Form S-8, No. 2-73803), which relates to the Phillips-Van Heusen Corporation Employee Savings and Retirement Plan,

(ii) Registration Statement (Form S-8, No. 33-50841), which relates to the Phillips-Van Heusen Corporation Associates Investment Plan for Residents of the Commonwealth of Puerto Rico,

(iii) Registration Statement (Form S-8, No. 333-29765), which relates to the Phillips-Van Heusen Corporation 1997 Stock Option Plan,

(iv) Registration Statement (Form S-8, No. 333-41068), which relates to the Phillips-Van Heusen Corporation 2000 Stock Option Plan,

(v) Registration Statement (Form S-8, No. 333-109000), which relates to the Phillips-Van Heusen Corporation 2003 Stock Option Plan,

(vi) Registration Statement (Form S-8, No. 333-125694), which relates to the Phillips-Van Heusen Corporation Associates Investment Plan for Residents of the Commonwealth of Puerto Rico,

(vii) Registration Statement (Form S-8, No. 333-143921), Registration Statement (Form S-8, No. 333-151966) and Registration Statement (Form S-8, No. 333-160382), each of which relates to the Phillips-Van Heusen Corporation 2006 Stock Incentive Plan,

(viii)         Registration Statement (Form S-8, No. 333-158327), which relates to the Phillips-Van Heusen Corporation Associates Investment Plan for Salaried Associates, and

(ix)          Registration Statement (Form S-3, No. 333-166190), which relates to an indeterminate amount of debt securities, common stock and preferred stock and pursuant to which Phillips-Van Heusen Corporation has issued (i) 7 3/8% Senior Notes due 2020; and (ii) 5,750,000 shares of common stock to the public,

of Phillips-Van Heusen Corporation and in the related Prospectuses of our reports dated March 30, 2011 with respect to the consolidated financial statements and financial statement schedule of Phillips-Van Heusen Corporation and the effectiveness of internal control over financial reporting of Phillips-Van Heusen Corporation, included in this Annual Report (Form 10-K) for the year ended January 30, 2011 as filed with the Securities and Exchange Commission.

/s/ ERNST & YOUNG LLP

New York, New York

March 30, 2011