Selenium vs. Puppeteer for Test Automation: Is a New Leader Emerging?

Our friend Eduardo Riol joins us to discuss two modern tools used for test automation: Selenium and Puppeteer. Selenium is a time-tested tool, but Puppeteer, a new contender, has some powerful features that might make you re-consider using it. Here are the advantages to each tool.

This is a guest post contributed by Eduardo Riol, also posted on his blog and translated from Spanish to English by Flood's own Antonio Jimenez.  

When we talk about functional test automation for browsers, probably the first tool that comes to mind is Selenium WebDriver.  

In the enterprise world, there are other solutions like UFT and Silk Test, both now owned by Microfocus. However, it is clear that in the past years Selenium has become the most widely adopted tool in the market, due to its open source model and increasing maturity compared to other free tools. Selenium has now positioned itself as the standard solution when dealing with functional automation because it’s low cost of acquisition and high level of integration with .Net, Python, Node.js, and of course, Java.

Since the release of Selenium,  Puppeteer has appeared in the scene with a new way of automating browser actions. Puppeteer is an open source Node.js library developed by Google with wide support for almost any action in Google’s Chrome browser. The basic idea is an API at the high level that allows us to automate actions in either of the Google browsers, Chrome and Chromium. For those non-Chrome stalwarts, Mozilla is also implementing Puppeteer in Firefox, and we hear they are about 85% done with that project.

What are the benefits of Puppeteer vs. Selenium ?

The first question that we might have, which are the functional advantages that the tool could provide us, beyond Google's sponsorship and support of the project.  Some of the main feature benefits we hear highlighted when comparing Puppeteer and Selenium are:

  • Puppeteer allows access to the measurement of loading and rendering times provided by the Chrome Performance Analysis tool.
  • Puppeteer affords more control over Chrome's browsers than Selenium WebDriver offers (likely due to Google’s support and sophisticated knowledge of Chrome).
  • Puppeteer removes the dependency on an external driver to run the tests, although this problem in Selenium could be mitigated by using Boni García's WebDriverManager dependency.
  • Puppeteer is set as default to run in headless mode, and it can also be changed to watch the execution live in non-headless mode.

While there are many benefits to Puppeteer, some of the main drawbacks we hear from customers include:

  • Puppeteer is limited to Chrome browser only for now, until Firefox support is completed
  • Puppeteer scripting only available in JavaScript for Node.js, and it is unclear if other languages will be supported in the future
  • Puppeteer has a smaller testing community using the tool currently, there is more test-specific support for Selenium

Hands-on: an example test with Puppeteer

If you aren’t yet convinced there are benefits to using Puppeteer over Selenium for your test needs, there is no better way to be sure than to create a script. It also helps to get familiar with the syntax, and you might find it helpful to review the code for this example in my Github Account.

Before the implementation of this example, and as it's explained in the README file, it's required to have Node.js and Chrome installed on your computer. Also, you might need to find more detailed instructions like these to complete your execution; this post is focused on explaining the Puppeteer code itself.

Like Selenium WebDriver for Java, where in order to define a test we need a framework like JUnit. Here, we are going to use Mocha, which is a framework for JavaScript projects, executed over Node.js and Chai.  This library allows us to use friendly asserts, similar to Hamcrest style in Java.

The scope of this example is to automate three simple tasks on the DuckDuckGo search engine:

  • We are going to validate the search page's title.
  • Perform a search and validating that the search throws some results.
  • Lastly, perform an invalid search that shouldn't throw results.

With the following code:

js
const puppeteer = require('puppeteer');
const { expect }  = require('chai');

describe('Duck Duck Go search using basic Puppeteer', () => {

   let browser;
   let page;

   beforeEach(async () => {
       browser = await puppeteer.launch();
       page = await browser.newPage();
       await page.goto('https://duckduckgo.com');
   });

   afterEach(async () => {
       await browser.close();
   });

   it('should have the correct page title', async () => {
       expect(await page.title()).to.eql('DuckDuckGo — Privacy, simplified.');
   });

   it('should show a list of results when searching actual word', async () => {
       await page.type('input[id=search_form_input_homepage]', 'puppeteer');
       await page.click('input[type="submit"]');
       await page.waitForSelector('h2 a');
       const links = await page.evaluate(() => {
           return Array.from(document.querySelectorAll('h2 a'));
       });
       expect(links.length).to.be.greaterThan(0);
   });

   it('should show a warning when searching fake word', async () => {
       await page.type('input[id=search_form_input_homepage]', 'pupuppeppeteerteer');
       await page.click('input[type="submit"]');
       await page.waitForSelector('div[class=msg__wrap]');
       const text = await page.evaluate(() => {
           return document.querySelector('div[class=msg__wrap]').textContent;
       });
       expect(text).to.contain('Not many results contain');
   });

});

As we can see, the API of Puppeteer that interacts with Chrome is quite intuitive, making use of functions such as:

  • puppeteer.launch(): to initialize the Chrome browser.
  • browser.newPage(): to create a new page in the context of the initialized browser.
  • page.goTo(URL): to navigate to a certain page.

Beyond these 3, there is a laundry list of functions that we can see in full detail at the Puppeteer API on GitHub.

Like we do with Java and JUnit, we will take advantage of Mocha's framework to define steps before and after the action, by using beforeEach and afterEach, respectively. In the example, you might also notice that I used the async/await instructions from JavaScript to bring this test closer to the synchronous paradigm of Java tests. But if you are proficient with JavaScript and would you like to make use of the Promises take a look into the following examples here.

The previous example shows a basic Puppeteer script, but Is this automation syntax scalable and maintainable?

Improving the example: Page Object Model

Anyone accustomed to automating tests knows that one of the critical factors is the use of programming patterns that allows test suite maintainability for the root definition of the dynamic objects. That’s the main reason why we need patterns like Page Object Model (POM), which allow us to structure the test code based on the architecture that we want to automate.  In this model, we represent each page and the objects it contains as a single artifact that exposes its API. This way, our tests can request specific actions to them easily one definition to update when changes inevitably occur.

However, is it possible to apply this paradigm in JavaScript? The simple answer is yes.  We can use JavaScript classes to enable a structure of pages with their own functions and make our tests maintainable and intelligible.

We can use POM to define a HomePage class that represents the home page of the DuckDuckGo search engine, in the following code:

js
class HomePage {

   constructor(page) {
       this.page = page;
   }

   async getTitle() {
       return this.page.title();
   }

   async searchFor(word) {
       await this.page.type('input[id=search_form_input_homepage]', word);
       await this.page.click('input[type="submit"]');
   }

}

module.exports = HomePage;

In addition, the results page can also be adapted to the POM pattern by defining a ResultsPage class, with the following code:

js
class ResultsPage {

   constructor(page) {
       this.page = page;
   }

   async getNumberOfLinks(page) {
       await this.page.waitForSelector('h2 a');
       const links = await this.page.evaluate(() => {
           return Array.from(document.querySelectorAll('h2 a'));
       });
       return links.length;
   }

   async checkIfResultsExist(page) {
       await this.page.waitForSelector('div[class=msg__wrap]');
       const text = await this.page.evaluate(() => {
           return document.querySelector('div[class=msg__wrap]').textContent;
       });
       return !text.includes('Not many results contain');
   }

}

module.exports = ResultsPage;

In this way, the definition of our tests is simplified and made more readable abstracting the code to a higher business oriented level:

js
const puppeteer = require('puppeteer');
const { expect }  = require('chai');
const HomePage = require('./pages/homePage');
const ResultsPage = require('./pages/resultsPage');

describe('Duck Duck Go search using Puppeteer with Page Object Model', () => {

   let browser;
   let page;

   beforeEach(async () => {
       browser = await puppeteer.launch();
       page = await browser.newPage();
       await page.goto('https://duckduckgo.com');
   });

   afterEach(async () => {
       await browser.close();
   });

   it('should have the correct page title', async () => {
       const homePage = new HomePage(page);
       expect(await homePage.getTitle()).to.eql('DuckDuckGo — Privacy, simplified.');
   });

   it('should show a list of results when searching actual word', async () => {
       const homePage = new HomePage(page);
       await homePage.searchFor('puppeteer');
       const resultsPage = new ResultsPage(page);
       expect(await resultsPage.getNumberOfLinks()).to.be.greaterThan(0);
   });

   it('should show a warning when searching fake word', async () => {
       const homePage = new HomePage(page);
       await homePage.searchFor('pupuppeppeteerteer');
       const resultsPage = new ResultsPage(page);
       expect(await resultsPage.checkIfResultsExist()).to.be.false;
   });

});

Conclusion

In my opinion, Puppeteer is a fascinating tool that can capture the attention of Selenium WebDriver users.  Especially when cross-browser testing is not needed, and teams that have experience working with Node.js, Puppeteer should be considered. Likewise, it is a mighty library if we want to read load values ​​and web performance as part of the metrics to be evaluated in a test.

However, the cross-browser testing limitation and the existence of Puppeteer in a single language is, in my opinion, what makes the Selenium still the de facto tool for web testing for the time being.  For us developers and testers, it will be interesting to have Puppeteer as one more tool to experiment in our toolbox. As the tool evolves, it will be interesting to see if it displaces Selenium as the leading web testing tool in the market.

Finally, as I read in a comment on Hacker News: "I know plenty of people who dislike working with Selenium, I have not yet met anyone who had enough experience with Puppeteer to hate it yet."


The license of this article is under Creative Commons, and we are respecting the terms agreed and described here.


Start load testing now

It only takes 30 seconds to create an account, and get access to our free-tier to begin load testing without any risk.

Keep reading: related stories
Return to the Flood Blog