Is WebDriver Fading Out? Enter BiDi

WebDriver and CDP are two protocols often mentioned in web test automation; now BiDi has arrived — how do they relate?

Preface

In the field of web test automation, the most central thing is actually the method of driving the browser. The most popular today is webDriver, which has been standardized by the W3C, while Chrome, which holds a monopoly position among browsers, actually also has its own standard protocol, namely CDP (Chrome DevTools Protocol), which provides lower-level control over the browser than WebDriver. Google’s own test automation tool, puppeteer, is in fact an automation implementation built on top of CDP.

The Evolution of WebDriver

The webDriver protocol, which grew up alongside Selenium, is the de facto standard in the field of Web test automation today. And this evolution also developed step by step.

  • 2004 Selenium is born
  • 2006 WebDriver is born as a tool
  • 2009 At the Google Test Automation Conference, Selenium and WebDriver are merged into one project, releasing Selenium 2.0
  • 2016 Selenium 3 is released, completely abandoning Selenium RC from 1.0
  • 2018 The W3C standards organization adopts WebDriver as a standard protocol
  • 2021 Selenium 4.0 is released, dropping WebDriver’s original Json Wire protocol and fully adopting W3C webDriver

The Rise of JavaScript and Node.js, and a Flourishing of Test Automation

As front-end development of Web pages has come to rely more and more on JavaScript, a batch of new test automation tools has also gradually emerged, including new automation solutions such as WebdriverIO, Appium, Nightwatch, Protractor (deprecated), Testcafe, Cypress, Puppeteer, and Playwright.

The Two Major Schools of Automation Tools

Based on the level at which they operate, the automation tools above can be divided into the following two major schools

  • High-level control: Tools that execute JavaScript in the browser. For example, Cypress and TestCafe use Web APIs and Node.js to run tests directly in the browser.
  • Low-level control: Tools that execute remote commands outside the browser. When a tool needs stronger control (for example, opening multiple tabs or emulating device mode), it needs to execute remote commands to control the browser through a protocol. The two main automation protocols are WebDriver (W3C) and the Chrome DevTools Protocol (CDP), represented by Selenium and puppeteer respectively

A Comparison of WebDriver and CDP

How webDriver Works

Test automation tools based on the WebDriver protocol require a dedicated browser driver, through which a standard protocol across different browsers is implemented. The tool and the driver interact over the http protocol.

In other words, the operations we perform on a web page are actually converted by the automation tool into an http request conforming to the webDriver protocol, and then the specific driver drives the browser to complete the action.

For example, a page click operation actually issues the following request to the corresponding browser driver:

1
2
3
4
5
# WebDriver: Click on a coffee element

curl -X POST http://127.0.0.1:4444/session/:element_id/element/click
   -H 'Content-Type: application/json'
   -d '{}'

But for some low-level control, such as certain browser-native events like opening a new window, because the webDriver protocol is based on http requests, it is difficult to handle.

How CDP Works

CDP, on the other hand, was not originally designed specifically for test automation purposes; it was designed for the debugging features of the Chrome browser’s developer tools, and was only later used for test automation by puppeteer. Compared with WebDriver, it is based on WebSocket to communicate directly at a low level with Chromium-based browsers, allowing bidirectional transmission, and can provide faster and lower-level control, accomplishing some actions that the webDriver protocol cannot handle.

Low-level control:

  • Capture console messages
  • Intercept network requests
  • Emulate Device Mode
  • Emulate geolocation
  • …..

Page operations are in fact converted into a series of CDP commands:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
// CDP: Click on a coffee element

// Mouse pressed
{ 
  command: 'Input.dispatchMouseEvent', 
  parameters: {
    type: 'mousePressed', x: 10.34, y: 27.1, clickCount: 1 }
}

// Mouse released
{ 
  command: 'Input.dispatchMouseEvent', 
  parameters: {
    type: 'mouseReleased', x: 10.34, y: 27.1, clickCount: 1 }
}

However, although it provides low-level support, the biggest problem with the CDP protocol is that it only supports Chromium-based browsers and is not an open standard. In addition, the CDP protocol is also relatively complex by design and not very suitable for being called directly from the coding level.

What Is WebDriver BiDi?

Based on the respective strengths and weaknesses of the WebDriver protocol and the CDP protocol, the test automation field is currently discussing the launch of a brand-new standard, namely WebDriver BiDi.

The goal of this protocol is to combine the strengths of the WebDriver and CDP protocols, providing the following features (the vision):

  • Cross-browser support
  • Bidirectional information transfer
  • As a universal standard, incorporated into the W3C
  • Provide low-level control
  • Designed for test automation, with simpler and more convenient calls through APIs
  • Support for various tools and browsers

At present this protocol is still in the negotiation and development stage. But it has already brought in the participation of major mainstream browser vendors, testing tools, and service providers, and it can be foreseen that it will be the next major update in the field of Web test automation!

Participating vendors:

The latest progress so far is the draft version released on 2024-09-23.

Judging from the content of the draft, it is still at a relatively early stage and is unlikely to be officially released in the short term, so for quite a long time WebDriver will remain the de facto standard for web test automation tools, but I believe BiDi will be the main direction of subsequent development.

In fact, from the recent updated source code of Selenium 4, we can already see extension methods supporting the BiDi protocol

We will keep a long-term eye on the development trend of this standard and its impact on current test automation technology.

Built with Hugo
Theme Stack designed by Jimmy