How to use Puppeteer in a Chrome extension

Travis — Mon, 03 Feb 2020 10:36:42 +0000

At my current job I would sometimes input client details into our CRM platform and would subsequently input these same details while setting them up on another third party platform. As such, there is a lot of copying data around and manual labor.

Automating this process via a Chrome Extension seemed like a good idea because I could also share it with non-technical colleagues that weren't comfortable with running scripts. Also at this company, Chrome is the default browser for all of the machines which run on Windows. With all the stars aligned, I managed to develop a working solution, although a bit hacky but works for my needs. This post was inspired by this thread.

Some prerequisite knowledge

basic understanding of Chrome Extension structure
basic understanding of Puppeteer

I had no prior experience working with Chrome Extensions, so I highly recommend The Coding Train's Playlist which is an excellent primer to developing them. It gave me everything I needed to get up and running.

With Puppeteer, the docs is quite thorough.

Bringing it all together

The trick is to use the bundled version of puppeteer. This link points to an old commit as this is no longer within the scope of their project. S

Bundle the repository per above documentation
Place it in your chrome extension folder
Reference the newly bundled folder in your popup.html with something like

<script src="./puppeteer/utils/browser/puppeteer-web.js"></script>

This is the hacky part. You'll then need to take advantage of Chrome's remote debugging functionality as puppeteer-web can't start its own instance via puppeteer.launch() and can only use puppeteer.connect() to connect to an already existing chrome instance. Add --remote-debugging-port=9222 to the end of the target field of your chrome.exe short cut. To learn more about this, read here.

Once remote debugging is activated you'll be able to see the webSocketDebuggerUrl property by visiting http://localhost:9222/json/version on your browser. This is the browserWSEndpoint the connect method will invoke.

Example response when visiting http://localhost:9222/json/version

{
   "Browser": "Chrome/79.0.3945.130",
   "Protocol-Version": "1.3",
   "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.130 Safari/537.36",
   "V8-Version": "7.9.317.33",
   "WebKit-Version": "537.36 (@e22de67c28798d98833a7137c0e22876237fc40a)",
   "webSocketDebuggerUrl": "ws://127.0.0.1:9222/devtools/browser/343998a5-ce82-456f-944c-a214ebb59e83"
}

You will also need to add the port address to the permissions array in the Chrome Extension's manifest.json file otherwise ajax requests won't work and you'll suffer from cors errors.

Example manifest.json file.

{
  "name": "Chrome Extension",
  "version": "0.1",
  "content_scripts": [
    {
      "matches": [
        "https://url-that-you-want-to-scrape/*"
      ],
      "js": ["content.js"]
    }
  ],
  "background": {
    "scripts": ["background.js"],
    "persistent": true
  },
  "permissions": [ "tabs" , "identity","http://localhost:9222/*"],
  "browser_action": {
    "default_title": "Chrome Extension",
    "default_popup": "popup.html"
  }
   "content_security_policy": "script-src 'self' 'unsafe-eval' https://apis.google.com/; object-src 'self'",
   "manifest_version": 2
}

Example popup.html file

<!DOCTYPE html>
<html>
  <head>
    <title>Example popup</title>
    <link rel="stylesheet" type="text/css" href="style.css">
  </head>
  <body>
    <div>
      <button id='puppeteer-button'>Do Puppeteer Things</button>
      <script src="./puppeteer/utils/browser/puppeteer-web.js"></script>
      <script type="module" src="popup.js"></script>
    </div>
  </body>
</html>

Example popup.js file

let browserWSEndpoint = '';
const puppeteer = require("puppeteer");

async function initiatePuppeteer() {
  await fetch("http://localhost:9222/json/version")
    .then(response => response.json())
    .then(function(data) {
        browserWSEndpoint = data.webSocketDebuggerUrl;
      })
    .catch(error => console.log(error));
}

initiatePuppeteer();

// Assign button to puppeteer function

document
  .getElementById("puppeteer-button")
  .addEventListener("click", doPuppeteerThings);

async function doPuppeteerThings() {
  const browser = await puppeteer.connect({
    browserWSEndpoint: browserWSEndpoint
  });
  const page = await browser.newPage();

  // Your puppeteer code goes here

}

Hopefully this is of help to someone. Happy automating.

Edit: Upon reflection, I'm pretty sure this can be accomplished with just basic javascript and dom manipulation via chrome.tabs.create and chrome.tabs.executeScript API and editing .value of input fields and what not. In my use case, it involved logging into multiple platforms and clicking various buttons through various states which may have been messy with plain javascript and I've already committed to puppeteer. Although if you just had a form that needed to be inputted and submitted, that's a different story.

Forem: Travis

How to use Puppeteer in a Chrome extension

Some prerequisite knowledge

Bringing it all together