[Feature]: Mouse.dragAndDrop API not supported for WebDriver BiDi

[timvandermeij]

Minimal, reproducible example

import puppeteer from "puppeteer";

const browser = await puppeteer.launch({
  product: "chrome",
  headless: false,
  protocol: "webDriverBiDi",
});
const pages = await browser.pages();
const page = pages[0];
await page.goto("https://github.com/puppeteer/puppeteer");
await page.mouse.dragAndDrop({ x: 0, y: 0 }, { x: 100, y: 100 });

Error string

UnsupportedOperation

Bug behavior

• Flaky
• PDF

Background

For Mozilla's PDF.js project we use the WebDriver BiDi protocol for Firefox at the moment (to test it and provide early feedback). I tried to replace our custom drag-and-drop code with the native Mouse.dragAndDrop API provided by Puppeteer, see https://pptr.dev/api/puppeteer.mouse.draganddrop, but while this works for CDP it doesn't work for WebDriver BiDi as an unsupported operation exception is raised.

(I have filed this as a bug because the APIs are not equal between CDP and WebDriver BiDi, and dragging-and-dropping is not a very rarely used operation I think, but could also understand if this is seen as a missing feature instead; in that case please feel free to move/amend the issue description.)

Expectation

Mouse.dragAndDrop works equally for CDP and WebDriver BiDi.

Reality

Mouse.dragAndDrop works for CDP, but doesn't seem to be supported in WebDriver BiDi.

Puppeteer configuration file (if used)

No response

Puppeteer version

22.6.5

Node version

21.7.2

Package manager

npm

Package manager version

10.5.2

Operating system

Linux

[github-actions]

This issue was not reproducible. Please check that your example runs locally and the following:

• Ensure the script does not rely on dependencies outside of puppeteer and puppeteer-core.
• Ensure the error string is just the error message.

  • Bad:

    Error: something went wrong
      at Object.<anonymous> (/Users/username/repository/script.js:2:1)
      at Module._compile (node:internal/modules/cjs/loader:1159:14)
      at Module._extensions..js (node:internal/modules/cjs/loader:1213:10)
      at Module.load (node:internal/modules/cjs/loader:1037:32)
      at Module._load (node:internal/modules/cjs/loader:878:12)
      at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:81:12)
      at node:internal/main/run_main_module:23:47

  • Good: Error: something went wrong.

• Ensure your configuration file (if applicable) is valid.
• If the issue is flaky (does not reproduce all the time), make sure 'Flaky' is checked.
• If the issue is not expected to error, make sure to write 'no error'.

Once the above checks are satisfied, please edit your issue with the changes and we will
try to reproduce the bug again.

---

Analyzer run

[timvandermeij]

I'm not sure why the analyzer has trouble reproducing this because UnsupportedOperation is the only error string that is logged when the reproducer is run locally, and given the Script failed; unknown error found analyzer log I presume this is an issue with the analyzer itself rather than the reproducer script?

[OrKoN]

Yeah, those are not supported since WebDriver does not define those events that CDP provides. Can you use the alternative API that causes drag event as the side effect of mouse events instead of dispatching drag events directly?

Example from tests:

  it('should drop', async () => {
    const {page, server} = await getTestState();

    await page.goto(server.PREFIX + '/input/drag-and-drop.html');

    using draggable = await page.$('#drag');
    assert(draggable);
    using dropzone = await page.$('#drop');
    assert(dropzone);

    await dropzone.drop(draggable);

    expect(await getDragState()).toBe(1234);
  });

Note that currently only Chrome synthesizes additional events for mouse events.

[OrKoN]

cc @whimboo

[OrKoN]

Alternatively, it should be possible to drag and drop using mouse events directly:

  it('should drop using mouse', async () => {
    const {page, server} = await getTestState();

    await page.goto(server.PREFIX + '/input/drag-and-drop.html');

    using draggable = await page.$('#drag');
    assert(draggable);
    using dropzone = await page.$('#drop');
    assert(dropzone);

    await draggable.hover();
    await page.mouse.down();
    await dropzone.hover();

    expect(await getDragState()).toBe(123);

    await page.mouse.up();
    expect(await getDragState()).toBe(1234);
  });

[timvandermeij]

Thank you for the feedback! We currently use https://github.com/mozilla/pdf.js/blob/master/test/integration/test_utils.mjs#L356-L359 and that works. It's mainly that I was reading the documentation and thought that using the Mouse.dragAndDrop API would perhaps be simpler/nicer so we can use the built-in Puppeteer functionality instead of simulating it ourselves. However, if that's currently the preferred way of doing (at least until/if the Mouse.dragAndDrop functionality is implemented in WebDriver BiDi) that's also fine for now.

(I mainly got curious to try it out and then ran into this and figured I should at least report the finding upstream, but fortunately there is no real need to replace the current code.)

[OrKoN]

Thanks, actually, the current way you use is the preferred way and we might deprecate dragAndDrop eventually since it injects drag events directly without emulating mouse/pointer input.