Exhibit 4.2

Description of Registrant’s Securities

Unless otherwise indicated or the context otherwise requires, references in this Exhibit 4.2 to “we, “us” and “our” refer collectively to Provident Bancorp, Inc. and The Provident Bank or to any of those entities, depending on the context. In addition, we may refer to Provident Bancorp, Inc. as “Provident Bancorp”.

General

Provident Bancorp is authorized to issue 100,000,000 shares of common stock, par value of $0.01 per share, and 50,000,000 shares of preferred stock, par value $0.01 per share. Each share of common stock has the same relative rights as, and is identical in all respects to, each other share of common stock. All of our shares of common stock are duly authorized, fully paid and nonassessable.

Common Stock

Dividends. Provident Bancorp may pay dividends to an amount equal to the excess of our capital surplus over payments that would be owed upon dissolution to stockholders whose preferential rights upon dissolution are superior to those receiving the dividend, and up to an amount that would not make us insolvent, as and when declared by our Board of Directors. The payment of dividends by Provident Bancorp is also subject to limitations that are imposed by law and applicable regulation, including restrictions on payments of dividends that would reduce Provident Bancorp’s assets below the then-adjusted balance of its liquidation account. The holders of common stock of Provident Bancorp will be entitled to receive and share equally in dividends as may be declared by our Board of Directors out of funds legally available therefor. If Provident Bancorp issues shares of preferred stock, the holders thereof may have a priority over the holders of the common stock with respect to dividends.

Voting Rights. The holders of common stock of Provident Bancorp will have exclusive voting rights in Provident Bancorp. They elect Provident Bancorp’s Board of Directors and act on other matters as are required to be presented to them under Maryland law or as are otherwise presented to them by the Board of Directors. Generally, each holder of common stock is entitled to one vote per share and does not have any right to cumulate votes in the election of directors. Any person who beneficially owns more than 10% of the outstanding shares of Provident Bancorp’s common stock, however, will not be entitled or permitted to vote any shares of common stock held in excess of the 10% limit. If Provident Bancorp issues shares of preferred stock, holders of the preferred stock may also possess voting rights. Certain matters require the approval of 80% of our outstanding common stock.

Liquidation. In the event of any liquidation, dissolution or winding up of The Provident Bank, Provident Bancorp, as the holder of 100% of The Provident Bank’s capital stock, would be entitled to receive all assets of The Provident Bank available for distribution, after payment or provision for payment of all debts and liabilities of The Provident Bank, including all deposit accounts and accrued interest thereon, and after distribution of the balance in the liquidation account to Eligible Account Holders (as defined in the Plan of Conversion of the former mutual holding company, Provident Bancorp). In the event of liquidation, dissolution or winding up of Provident Bancorp, the holders of its common stock would be entitled to receive, after payment or provision for payment of all its debts and liabilities (including payments with respect to its liquidation account), all of the assets of Provident Bancorp available for distribution. If preferred stock is issued, the holders thereof may have a priority over the holders of the common stock in the event of liquidation or dissolution.

Preemptive Rights. Holders of the common stock of Provident Bancorp are not entitled to preemptive rights with respect to any shares that may be issued. The common stock is not subject to redemption.

Preferred Stock

None of the shares of Provident Bancorp’s authorized preferred stock are outstanding. Preferred stock may be issued with preferences and designations as our Board of Directors may from time to time determine. Our Board of Directors may, without stockholder approval, issue shares of preferred stock with voting, dividend, liquidation and conversion rights that could dilute the voting strength of the holders of the common stock and may assist management in impeding an unfriendly takeover or attempted change in control.

Maryland Law and Articles of Incorporation and Bylaws of Provident Bancorp

Maryland law, as well as Provident Bancorp’s articles of incorporation and bylaws, contain a number of provisions relating to corporate governance and rights of stockholders that may discourage future takeover attempts. As a result, stockholders who might desire to participate in such transactions may not have an opportunity to do so. In addition, these provisions will also render the removal of the Board of Directors or management of Provident Bancorp more difficult.

Directors. The Board of Directors is divided into three classes. The members of each class are elected for a term of three years and only one class of directors is elected annually. Thus, it would take at least two annual elections to replace a majority of the Board of Directors. The bylaws establish qualifications for board members, including restrictions on affiliations with competitors of The Provident Bank and restrictions based upon prior legal or regulatory violations. Further, the bylaws impose notice and information requirements in connection with the nomination by stockholders of candidates for election to the Board of Directors or the proposal by stockholders of business to be acted upon at an annual meeting of stockholders. Such notice and information requirements are applicable to all stockholder business proposals and nominations, and are in addition to any requirements under the federal securities laws.

Restrictions on Call of Special Meetings. The articles of incorporation and bylaws provide that special meetings of stockholders can be called by the President, the Chief Executive Officer, the Chairman, by a majority of the whole Board of Directors or upon the written request of stockholders entitled to cast at least a majority of all votes entitled to vote at the meeting.

Prohibition of Cumulative Voting. The articles of incorporation prohibit cumulative voting for the election of directors.

Limitation of Voting Rights. The articles of incorporation provide that in no event will any person who beneficially owns more than 10% of the then-outstanding shares of common stock, be entitled or permitted to vote any of the shares of common stock held in excess of the 10% limit.

Restrictions on Removing Directors from Office. The articles of incorporation provide that directors may be removed only for cause, and only by the affirmative vote of the holders of at least two-thirds of the voting power of all of Provident Bancorp’s then-outstanding common stock entitled to vote (after giving effect to the limitation on voting rights discussed above in “—Limitation of Voting Rights”).

Forum Selection for Certain Stockholder Lawsuits. The articles of incorporation provide that, unless Provident Bancorp consents in writing to the selection of an alternative forum, the sole and exclusive forum for (i) any derivative action or proceeding brought on behalf of Provident Bancorp, (ii) any action asserting a claim of breach of a fiduciary duty owed by any director, officer or other employee of Provident Bancorp to Provident Bancorp or Provident Bancorp stockholders, (iii) any action asserting a claim arising pursuant to any provision of the Maryland General Corporation Law, or (iv) any action asserting a claim governed by the internal affairs doctrine shall be a state or federal court located within the state of Maryland, in all cases subject to the court’s having personal jurisdiction over the indispensible parties named as defendants. Under the articles of incorporation, any person or entity purchasing or otherwise acquiring any interest in shares of capital stock of Provident Bancorp shall be deemed to have notice of and consented to the exclusive forum provisions of the articles of incorporation.

Authorized but Unissued Shares. Provident Bancorp has authorized but unissued shares of common and preferred stock. The articles of incorporation authorize 50,000,000 shares of serial preferred stock. Provident Bancorp is authorized to issue preferred stock from time to time in one or more series subject to applicable provisions of law, and the Board of Directors is authorized to fix the designations, and relative preferences, limitations, voting rights, if any, including without limitation, offering rights of such shares (which could be multiple or as a separate class). In the event of a proposed merger, tender offer or other attempt to gain control of Provident Bancorp that the Board of Directors does not approve, it may be possible for the Board of Directors to authorize the issuance of a series of preferred stock with rights and preferences that would impede the completion of the transaction. An effect of the possible issuance of preferred stock therefore may be to deter a future attempt to gain control of Provident Bancorp.

Amendments to Articles of Incorporation and Bylaws. Amendments to the articles of incorporation must be approved by the Board of Directors and by the affirmative vote of at least two-thirds of the outstanding shares of common stock, or by the affirmative vote of a majority of the outstanding shares of common stock if at least two-thirds of the members of the whole Board of Directors approves such amendment; provided, however, that approval by at least 80% of the outstanding voting stock is generally required to amend certain provisions.

The articles of incorporation also provide that the bylaws may be amended by the affirmative vote of a majority of Provident Bancorp’s directors or by the stockholders by the affirmative vote of at least 80% of the total votes eligible to be cast at a duly constituted meeting of stockholders. Any amendment of this super-majority requirement for amendment of the bylaws would also require the approval of 80% of the total votes eligible to be cast.

Business Combinations. Maryland law restricts mergers, consolidations, sales of assets and other business combinations between Provident Bancorp and an “interested stockholder,” and provides certain vote standards for other mergers, consolidations, sales of assets and other business combinations.

Evaluation of Offers. The articles of incorporation provide that Provident Bancorp’s Board of Directors, when evaluating a transaction that would or may involve a change in control of Provident Bancorp (whether by purchases of its securities, merger, consolidation, share exchange, dissolution, liquidation, sale of all or substantially all of its assets, proxy solicitation or otherwise), may, in connection with the exercise of its business judgment in determining what is in the best interests of Provident Bancorp and its stockholders and in making any recommendation to the stockholders, give due consideration to all relevant factors, including, but not limited to, certain enumerated factors.

Purpose and Anti-Takeover Effects of Provident Bancorp’s Articles of Incorporation and Bylaws. Our Board of Directors believes that the provisions described above are prudent and reduce our vulnerability to takeover attempts and certain other transactions that have not been negotiated with and approved by our Board of Directors. We believe these provisions are in the best interests of Provident Bancorp and its stockholders. Our Board of Directors believes that it is in the best position to determine the true value of Provident Bancorp and to negotiate more effectively for what may be in the best interests of all our stockholders. Accordingly, our Board of Directors believes that it is in the best interests of Provident Bancorp and all of our stockholders to encourage potential acquirers to negotiate directly with the Board of Directors and that these provisions will encourage such negotiations and discourage hostile takeover attempts. It is also the view of our Board of Directors that these provisions should not discourage persons from proposing a merger or other transaction at a price reflective of the true value of Provident Bancorp and that is in the best interests of all our stockholders.

Takeover attempts that have not been negotiated with and approved by our Board of Directors present the risk of a takeover on terms that may be less favorable than might otherwise be available. A transaction that is negotiated and approved by our Board of Directors, on the other hand, can be carefully planned and undertaken at an opportune time in order to obtain maximum value for our stockholders, with due consideration given to matters such as the management and business of the acquiring corporation.

Although a tender offer or other takeover attempt may be made at a price substantially above the current market price, such offers are sometimes made for less than all of the outstanding shares of a target company. As a result, stockholders may be presented with the alternative of partially liquidating their investment at a time that may be disadvantageous, or retaining their investment in an enterprise that is under different management and whose objectives may not be similar to those of the remaining stockholders.

Despite our belief as to the benefits to stockholders of these provisions of Provident Bancorp’s articles of incorporation and bylaws, these provisions also may have the effect of discouraging a future takeover attempt that would not be approved by our Board of Directors, but pursuant to which stockholders may receive a substantial premium for their shares over then current market prices. As a result, stockholders who might desire to participate in such a transaction may not have any opportunity to do so. Such provisions will also make it more difficult to remove our Board of Directors and management. Our Board of Directors, however, has concluded that the potential benefits outweigh the possible disadvantages.

Federal Conversion Regulations

Federal Reserve Board regulations prohibit any person from making an offer, announcing an intent to make an offer or participating in any other arrangement to purchase stock or acquire stock or subscription rights in a converting institution or its holding company from another person prior to completion of its conversion. Further, without the prior written approval of the Federal Reserve Board, no person may make an offer or announcement of an offer to purchase shares or actually acquire shares of a converted institution or its holding company for a period of three years from the date of the completion of the conversion if, upon the completion of such offer, announcement or acquisition, the person would become the beneficial owner of more than 10% of the outstanding stock of the institution or its holding company. The Federal Reserve Board has defined “person” to include any individual, group acting in concert, corporation, partnership, association, joint stock company, trust, unincorporated organization or similar company, a syndicate or any other group formed for the purpose of acquiring, holding or disposing of securities of an insured institution. However, offers made exclusively to a bank or its holding company, or to an underwriter or member of a selling group acting on the converting institution’s or its holding company’s behalf for resale to the general public, are excepted. The regulation also provides civil penalties for willful violation or assistance in any such violation of the regulation by any person connected with the management of the converting institution or its holding company or who controls more than 10% of the outstanding shares or voting rights of a converted institution or its holding company.

Massachusetts Conversion Regulations

Massachusetts regulations provide that, without prior written notice to us and the prior written approval of the Massachusetts Commissioner of Banks, no person may directly or indirectly offer to acquire the beneficial ownership of more than 10% of a converted holding company for a period of three years from the date of the completion of the conversion. Where a person, directly or indirectly, acquires beneficial ownership of more than 10% of a converted holding company, without prior written notice to the converted holding company and the prior written approval of the Massachusetts Commissioner of Banks, the securities beneficially owned by such person in excess of 10% shall not be counted as shares entitled to vote, shall not be voted by any person or counted as voting shares in connection with any matter submitted to the stockholders for a vote, and shall not be counted as outstanding for purposes of determining the affirmative vote necessary to approve any matter submitted to stockholders for a vote, and the Massachusetts Commissioner of Banks may take any further action he may deem appropriate. The regulation provides for civil penalties for a violation of this regulation.

Change in Control Law and Regulations

Under the Change in Bank Control Act, no person, or group of persons acting in concert, may acquire control of a bank holding company such as Provident Bancorp unless the Federal Reserve Board has been given 60 days’ prior written notice and not disapproved the proposed acquisition. The Federal Reserve Board considers several factors in evaluating a notice, including the financial and managerial resources of the acquirer and competitive effects. Control, as defined under the applicable regulations, means the power, directly or indirectly, to direct the management or policies of the company or to vote 25% or more of any class of voting securities of the company. Acquisition of more than 10% of any class of a bank holding company’s voting securities constitutes a rebuttable presumption of control under certain circumstances, including where, as is the case with Provident Bancorp, the issuer has registered securities under Section 12 of the Securities Exchange Act of 1934.

In addition, federal regulations provide that no company may acquire control (as defined in the Bank Holding Company Act) of a bank holding company without the prior approval of the Federal Reserve Board. Any company that acquires such control becomes a “bank holding company” subject to registration, examination and regulation by the Federal Reserve Board.

Massachusetts Banking Law

Under Massachusetts banking laws, a company owning or controlling two or more banking institutions, including a savings bank, is regulated as a bank holding company. Each Massachusetts bank holding company: (i) must obtain the approval of the Massachusetts Board of Bank Incorporation before engaging in certain transactions, such as the acquisition of more than 5% of the voting stock of another banking institution; (ii) must register, and file reports, with the Massachusetts Division of Banks; and (iii) is subject to examination by the Massachusetts Division of Banks. Provident Bancorp would become a Massachusetts bank holding company if it acquires a second banking institution and holds and operates it separately from The Provident Bank. In addition, for a period of three years following completion of a conversion to stock form, no person may directly or indirectly offer to acquire or acquire beneficial ownership of more than 10% of any class of equity security of a converting mutual savings bank or mutual holding company without prior written approval of the Massachusetts Commissioner of Banks.

Benefit Plans

In addition to the provisions of Provident Bancorp’s articles of incorporation and bylaws described above, benefit plans of Provident Bancorp and The Provident Bank that may authorize the issuance of equity to its board of directors, officers and employees adopted in connection with or following the offering contain or may contain provisions which also may discourage hostile takeover attempts which the board of directors of The Provident Bank might conclude are not in the best interests of Provident Bancorp and The Provident Bank or Provident Bancorp’s stockholders.