Firefox WebDriver Newsletter — 114 – Firefox Developer Experience

WebDriver is a remote control interface that enables introspection and control of user agents. As such it can help developers to verify that their websites are working and performing well with all major browsers. The protocol is standardized by the W3C and consists of two separate specifications: WebDriver classic (HTTP) and the new WebDriver BiDi (Bi-Directional).

This newsletter gives an overview of the work we’ve done as part of the Firefox 114 release cycle.

Contributions

With Firefox being an open source project, we are grateful to get contributions from people outside of Mozilla:

WebDriver code is written in JavaScript, Python, and Rust so any web developer can contribute! Read how to setup the work environment and check the list of mentored issues for Marionette, and geckodriver.

WebDriver BiDi

By enhancing the implementation of the WebDriver BiDi protocol we can offer more features to our users. The following new commands and events are available now:

Emulation of user input

With the input.performActions and input.releaseActions commands being available, clients can emulate user input for interacting with elements on the web page. In Firefox we offer support for all the available input sources of the WebDriver specification, which are key, pointer, and wheel. Only the sub-type pen of the pointer input source is not supported yet.

Similar to the WebDriver classic implementation in Marionette several input sources can also be combined which will then be performed in parallel. This can cover situations when a modifier key (e.g. Shift) needs to be pressed while performing a click onto an element.

To give an easy example the following payload of a WebSocket message simulates a move of the mouse pointer to a position that is 100px to the right and 50px to the bottom of the element’s in-view center point:

{
  "id": 1,
  "method": "input.performActions",
  "params": {
    "context": "<context-id>",
    "actions": [
      {
        "id": "pointer-0", 
        "type":"pointer", 
        "actions": [
          {
            "type": "pointerMove",
            "x": 100,
            "y": 50,
            "origin": {
              "type": "element",
              "element": {
                "sharedId": "<element-id>"
              }
            }
          }
        ],
        "parameters": {
          "pointerType": "mouse"
        }
      }
    ]
  }
}

Support custom browser to client messages

With the release of Firefox 112 we added APIs to add and to remove pre-load (bootstrap) scripts, which can be used to execute code when a new window gets attached and before any author-defined scripts have run. This was already helpful to perform setup logic, or to expose functions that could be called later on from script.evaluate or script.callFunction. However there was no way to properly handle more dynamic situations, such as monitoring DOM mutations and notify the client about them.

Now with the client messages support added the pre-load script can be installed with a channel argument. This channel argument is a function which can be called at any point to send data to the client, via a script.message event which will also include the id of the channel.

For example, one highly requested feature from the WebDriver community was the ability to handle DOM mutations. While this was not possible to support in WebDriver classic, some frameworks like Selenium implemented such a feature already by using the Chrome DevTools Protocol (CDP). But that’s not cross-browser compatible, and would only allow clients to run tests against Chrome. Now with WebDriver BiDi it finally becomes a reality and hopefully soon all major browsers that support the new protocol will include it as well. But how does it actually work? Let’s take a closer look.

First we should subscribe to the script.message event. In our example we will do that globally to automatically receive events for newly created tabs or frames as well. The payload of the command looks like:

{
  "id": 1,
  "method": "session.subscribe",
  "params": {
    "events": [
      "script.message"
    ]
  }
}

Now the pre-load script can be added which registers a listener that gets called when mutations are observed, and which handles the mutation data before sending messages for each mutation by invoking the channel() callback:

{
  "id": 2,
  "method": "script.addPreloadScript",
  "params": {
    "functionDeclaration": "
      (channel) => {
        function observe(mutations) {
          mutations.forEach(mutation => {
            const attrName = mutation.attributeName;
            const newValue = 
              mutation.target.getAttribute(attrName);

            channel({ attrName, newValue });
          });
        }

        const observer = new MutationObserver(observe);
        observer.observe(document, {
          attributes: true, subtree: true
        });
      }",
    "arguments": [
      {
        "type": "channel",
        "value": {
          "channel": "my.mutations.channel"
        }
      }
    ]
  }
}

Starting from now the client will receive a script.message event every time a Mutation takes place:

{
  "method": "script.message",
  "params": {
    "channel": "my.mutations.channel",
    "data": {
      "type": "object",
        "value": [
          ["attributeName", {
            "type": "string",
            "value": "class"
          }],
          ["newValue", {
            "type": "string",
            "value": "mutated" }
          ]]
    },
    "source": {
      "realm": "<realm-id>",
      "context": "<context-id>"
    }
  }
}

Support `serializationOptions` for RemoteValue serialization

When serializing JavaScript objects or DOM nodes to a RemoteValue, the existing algorithm was not that flexible and had some disadvantages like not being able to have a one-shot serialization of nested objects or including a <a href="https://developer.mozilla.org/en-US/docs/Web/API/DocumentFragment">DocumentFragment</a> of a <a href="https://developer.mozilla.org/en-US/docs/Web/API/ShadowRoot">ShadowRoot</a> node. As such the serializationOptions have been added to the Webdriver BiDi specification. With this feature added to Firefox it is now possible to define individual serialization depths for DOM nodes and generic JS objects, as well as to control the inclusion of the shadow tree.

Bug fixes

Besides adding new features we have also fixed a couple of known bugs:

Both script.evaluate and script.callFunction are no longer failing to include the stack trace and to properly build the exception details for a rejected Promise.
The browsingContext.documentLoaded and browsingContext.load events now use the target URL of the navigation as value of the url field. Before it was using the baseURI and this would significantly differ when a page defines a <base> meta tag.

Marionette (WebDriver classic)

No new features have been added for this release of Firefox but the following bugs have been fixed:

The command WebDriver:GetComputedRole properly returns the WAI-ARIA roles now.
Modifier keys are now correctly reset when they are used again within the same call to WebDriver:ElementSendKeys.

Firefox WebDriver Newsletter — 114