Improve targeting devtools panel iframe created by an extension

[antross]

Filing as the workaround listed in #3699 wasn't sufficient to target content inside the iframe created for a devtools extension panel. I was able to eventually get this working, but it required accessing private APIs (as did the workaround in #3699) so I'd like to see this scenario improved and officially supported.

Here's what it took:

---

✔ Good: Installing the extension and opening the devtools on a page of my choosing works well.

    const browser = await launch({
        args: [
            `--disable-extensions-except=${pathToExtension}`,
            `--load-extension=${pathToExtension}`
        ],
        defaultViewport: null,
        devtools: true,
        headless: false
    });

    const page = (await browser.pages())[0];

    await page.goto(url);

---

⚠ Issue 1: Getting a puppeteer Page pointing to the devtools requires some URL-matching and a hack as the Target type is other (this could be loaded in a separate browser tab as described in #3699, but that also requires accessing private data).

    const targets = await browser.targets();
    const devtoolsTarget = targets.filter((t) => {
        return t.type() === 'other' && t.url().startsWith('chrome-devtools://');
    })[0];

    // Hack to get a page pointing to the devtools
    devtoolsTarget._targetInfo.type = 'page';

    const devtoolsPage = await devtoolsTarget.page();

🙏 Proposal 1: Expose the devtools as a known target type that can return a Page.

    const targets = await browser.targets();
    const devtoolsTarget = targets.filter((t) => t.type() === 'devtools')[0];
    const devtoolsPage = await devtoolsTarget.page();

---

✔ Good: Activating the extension devtools tab so it's iframe gets created is awkward, but at least this uses public APIs. It takes advantage of keyboard shortcuts as suggested in #3699 - though I'm iterating just once in reverse as the extension is always the last tab.

    await devtoolsPage.keyboard.down('Control');
    await devtoolsPage.keyboard.press('[');
    await devtoolsPage.keyboard.up('Control');

---

⚠ Issue 2: At this point the example in #3699 shows how to get a reference to the tab element in the devtools, but that is outside of the extension panel's iframe and I want to debug the content inside the frame. Unfortunately devtoolsPage.frames() doesn't contain the extension panel frame, but I was able to work around this because the panel did appear in browser.targets().

    const targets = await browser.targets();
    const extensionPanelTarget = targets.filter((t) => {
        return t.type() === 'other' &&
            t.url().startsWith('chrome-extension://') &&
            t.url().endsWith('/panel.html');
    })[0];

    // Hack to get a page pointing to the devtools extension panel.
    extensionPanelTarget._targetInfo.type = 'page';

    // Most APIs on `Page` fail as `mainFrame()` is `undefined` (frame has a `parentId`).
    const extensionPanelPage = await extensionPanelTarget.page();

    // Getting the first frame and working with that instead provides something usable.
    const extensionPanelFrame = extensionPanelPage.frames()[0];

    // And now we can finally interact with our extension panel frame!
    extensionPanelFrame.click(selector);

🙏 Proposal 2: Fix returning the extension panel frame in devtoolsPage.frames():

    const extensionPanelFrame = devtoolsPage.frames().filter((frame) => {
        return frame.url().startsWith('chrome-extension://') &&
            frame.url().endsWith('/panel.html');
    })[0];

    extensionPanelFrame.click(selector);

---

Let me know if you'd prefer to have the proposals split into two separate issues. I wanted to present them together initially for context on what it took to interact with a devtools extension panel and to better help anyone else who may be trying to do the same.

[aslushnikov]

Proposal 1: Expose the devtools as a known target type that can return a Page.

@antross sure, this'd work. Would you mind sharing why do you want to automate DevTools with Puppeteer?

Proposal 2: Fix returning the extension panel frame in devtoolsPage.frames():

This is a dupe of #2548. We're experimenting with different approaches to get it working.

[antross]

@aslushnikov Thanks for the response!

Would you mind sharing why do you want to automate DevTools with Puppeteer?

What I'm most interested in at the moment is automating a panel created by the devtools extension we've been working on for webhint. I'm using Puppeteer to drive the extension panel as part of a basic end-to-end test to ensure the user flow is working.

At the moment anything that could activate the panel and get a reference would be good enough, but in the future I'd also like to validate integration points like linking to the Elements and Sources panels. That would likely involve a bit more direct automation of the devtools.

This is a dupe of #2548. We're experimenting with different approaches to get it working.

Understood. Thanks for the reference.

[aslushnikov]

@antross I actually like your subtle hack in "Proposal 1". It's not too bad, and given that you're probably the first ever client to test Chrome DevTools extension - let's keep it like this for now. Once there are some other clients, we'll expand our API surface.

"Proposal 2" will be covered with OOPIFs.

I'll close this for now - there's nothing to do at the moment to address this. Thanks for reaching though!

[scurker]

Just wanted to add a note that we're attempting to run E2E testing for the Axe extension as well. The methods we have to run through to get an isolated instance are probably extremely hacky and require heavy usage of the chrome devtools protocol. We couldn't find another way to capture screenshots of the extension without having full page screenshots.

[dgp1130]

+1

I also own a Chrome extension with a non-trivial DevTools panel which I would like to be able to test with Puppeteer. An explicitly supported means of getting access to the panel would be very useful for us.

[pistone]

+1

My use case is to test emulation pane in DevTools; the hack is how I can get to the "Toggle device mode" button (edited: or I can hack the url simply by making the DevTools dockable).