[Bug]: waitForNavigation does not complete navigation when used with click

[coder-hxl]

Bug expectation

After executing the navigation to the second page, I still get the content of the first page,

Bug behavior

• Flaky
• PDF

Minimal, reproducible example

import puppeteer from 'puppeteer'
;(async () => {
  const browser = await puppeteer.launch()
  const page = await browser.newPage()
  await page.setViewport({ width: 1280, height: 1024 })
  await page.goto('https://github.com/coder-hxl/x-crawl/releases')

  const navHandles = await page.$$('div[role="navigation"] [aria-label]')
  for (let i = 1; i <= navHandles.length; i++) {
    const tagContents = await page.$$eval(
      'section .Box-body .wb-break-word a',
      (aEls) => aEls.map((item) => item.innerHTML)
    )

    console.log(tagContents)

    if (i < navHandles.length) {
      await Promise.all([
        page.waitForNavigation(),
        page.click('div[role="navigation"] .next_page')
      ])
    }
  }

  await browser.close();
})()

Error string

no error

Puppeteer configuration

No response

Puppeteer version

v19.8.0

Node version

v19.0.0

Package manager

pnpm

Package manager version

v7.14.2

Operating system

Windows

[github-actions]

The issue has been labeled as confirmed by the automatic analyser.
Someone from the Puppeteer team will take a look soon!

---

Analyzer run

[coder-hxl]

Add await page.screenshot({ path: ./upload/${i}-page.jpg }) to complete the navigation.

import puppeteer from 'puppeteer'
;(async () => {
  const browser = await puppeteer.launch()
  const page = await browser.newPage()
  await page.setViewport({ width: 1280, height: 1024 })
  await page.goto('https://github.com/coder-hxl/x-crawl/releases')

  const navHandles = await page.$$('div[role="navigation"] [aria-label]')
  for (let i = 1; i <= navHandles.length; i++) {
    await page.screenshot({ path: `./upload/${i}-page.jpg` })

    const tagContents = await page.$$eval(
      'section .Box-body .wb-break-word a',
      (aEls) => aEls.map((item) => item.innerHTML)
    )

    console.log(tagContents)

    if (i < navHandles.length) {
      await Promise.all([
        page.waitForNavigation(),
        page.click('div[role="navigation"] .next_page')
      ])
    }
  }

  await browser.close();
})()

[OrKoN]

I am unable to reproduce. All navigations happen without problem for me (the code was missing the browser.close() to exit):

import puppeteer from 'puppeteer'
;(async () => {
  const browser = await puppeteer.launch({ headless: true, })
  const page = await browser.newPage()
  await page.setViewport({ width: 1280, height: 1024 })
  await page.goto('https://github.com/coder-hxl/x-crawl/releases')

  const navHandles = await page.$$('div[role="navigation"] [aria-label]')
  for (let i = 1; i <= navHandles.length; i++) {
    const tagContents = await page.$$eval(
      'section .Box-body .wb-break-word a',
      (aEls) => aEls.map((item) => item.innerHTML)
    )

    console.log(tagContents)

    if (i < navHandles.length) {
      await Promise.all([
        page.waitForNavigation(),
        page.click('div[role="navigation"] .next_page')
      ])
    }
  }
  await browser.close();
})()

[coder-hxl]

@OrKoN After executing the navigation to the second page, I still get the content of the first page

[OrKoN]

This might be due to the fact that GitHub does not always trigger a navigation and loads the page via SPA? You need to wait for the new page content instead of the navigation then. E.g.,

import puppeteer from 'puppeteer'
;(async () => {
  const browser = await puppeteer.launch({ headless: true, })
  const page = await browser.newPage()
  await page.setViewport({ width: 1280, height: 1024 })
  await page.goto('https://github.com/coder-hxl/x-crawl/releases')

  const navHandles = await page.$$('div[role="navigation"] [aria-label]')
  for (let i = 1; i <= navHandles.length; i++) {
    await page.waitForSelector(`[aria-current=page][aria-label="Page ${i}"]`);

    const tagContents = await page.$$eval(
      'section .Box-body .wb-break-word a',
      (aEls) => aEls.map((item) => item.innerHTML)
    )

    console.log(tagContents)

    if (i < navHandles.length) {
      console.log('here')
      await page.click('div[role="navigation"] .next_page');
    }
  }
  await browser.close();
})()

[coder-hxl]

@OrKoN Appreciate your reply.