The chromium binary is not available for arm64 #7740
Comments
|
I have arm64 Chromium binaries available at www.chromiumforlambda.com |
|
A workaround I'm using is to install chromium with the base image's package manager (e.g. |
|
Thanks @talisto, that's what I've been doing as a work around, but took two days to come up with that. I'm using puppeteer via Backstop JS and in my scenario Backstop uses a Docker image to do the screenshot comparisons (yields consistent results across OSes) It would be nice if |
This ^ |
hello, could you please share your |
|
Another workaround is to use a custom browser with something like: Obviously, choose a user agent you need and tweak the custom browser options. |
@talisto would you mind sharing your |
This is what I'm currently using: ..some of which I lifted from here: ..which is mostly based on the "Running Puppeteer in Docker" section of the Puppeteer troubleshooting docs: The only relevant bits to this issue are the |
|
I just noticed that the Puppeteer docs actually have an example for Docker using Alpine which also sets https://github.com/puppeteer/puppeteer/blob/main/docs/troubleshooting.md#running-on-alpine |
|
It's important to note the tight version requirements between Puppeteer and Chromium when using this approach. The version of Chromium currently available in alpine3.15 is 99.0.4844.84-r0, which mean the versions of Puppeteer you could/should use are 13.2.0-13.4.1. However, if you're using an alpine3.14 image, you'll get Chromium 93.0.4577.82-r0, which maps to Puppeteer 10.2.0-11.0.0. |
|
My image bases on debian, so I have to |
How do you get the mapping from Chromium 99.0.4844.84-r0 to puppeteer versions 13.2.0-13.4.1? Inspecting I am trying to solve this for debian bullseye, which would install https://packages.debian.org/de/bullseye/chromium version 99.0.4844.74-1~deb11u1. Seems it's the same version rquirements but I don't get the mapping from |
|
That's pretty annoying. Thanks for sharing @flaushi looks like I have to go to the same rabbit hole like you.
But the Lines 22 to 25 in 7b38b45 confusing |
I check the CHANGELOG, which is also published as GitHub release notes. Puppeteer releases that pull in a new Chromium version will include the version and snapshot revision number. For example:
That version applies until the next release that notes a Chromium change. |
|
Does anyone have a viable workaround for running Puppeteer 13 or 14 via Docker on an M1 mac using a Debian Node base image? e.g.
Happy for rosetta emulation at this point since a native |
|
@ReDrUm you can test my Dockerfile: https://github.com/it-novum/puppeteer-docker/blob/development/Dockerfile |
|
@ReDrUm the ideal fix would be an arm build of Chromium for Debian, but doesn't sound like that's on Puppeteer's radar. Here's the Dockerfile I used to work around my issues (unfortunately it's an old version of Chrome) |
|
I've spent a few hours trying to run one of our Docker containers on Linux/arm64, it was hard and mostly trial and error mode. My final setup is the whole PR is at elastic/apm-integration-testing#1493 |
|
Thanks! I now have a working copy by installing Fingers crossed we get some |
|
Based on @nook24 works, I created a docker with dumb-init and wkhtmltopdf which works in both arm64 and amd64 (tested on both on Linux and Mac) https://github.com/alaminopu/puppeteer-docker Docker hub: https://hub.docker.com/repository/docker/alaminopu/puppeteer-docker/general |
I feel compelled to send you a basket of mini muffins. 😄 ❤️ |
|
For anyone that is using Docker and having issues with |
|
All workarounds posted on this thread are unreliable because they are all based on installing a random version of Chrome via a package manager (no matter if it's The official Puppeteer docs clearly say that:
So while installing a stable Chromium version via a package manager works, it doesn't guarantee that it works well because there's no guarantee it was tested with the version of Puppeteer you use. |
|
Node.js version Npm version Environment : Kernel : Issue : |
|
As @wujekbogdan says, using a package-managed version is unworkable, but a solution, assuming you trust their builds is to use the arm64 chromium provided by Playwright, built/hosted Microsoft. It's a little fiddly but the point of their project is to automatically build the latest versions of browsers for all platforms. This means that you can look at the commit history of For example, Puppeteer v21.5.0 wants Chromium 119.0.6045.105 and that's in r1088, which can be downloaded from:
In addition, a Ideally, Puppeteer would work natively with arm64 though 😃 👍 |
|
Dockerfile for above solution (no md5 checksum): # chromium arm64 version solution: https://github.com/puppeteer/puppeteer/issues/7740#issuecomment-1833202428
# puppeteer version: ~21.5.2
# chromium version: 119.0.6045.105 (https://pptr.dev/chromium-support)
# playwright arm64 chromium build for chromium version 119.0.6045.105: r1088 (https://github.com/microsoft/playwright/commit/38115d121bd330b596a1fde2c81bbc2930783f86)
FROM node:20-slim
ENV NODE_ENV production
RUN apt-get update && apt-get install wget unzip -y
RUN wget -q -O - 'https://playwright.azureedge.net/builds/chromium/1088/chromium-linux-arm64.zip' && \
unzip chromium-linux-arm64.zip && \
rm -f ./chromium-linux-arm64.zip
ENV PUPPETEER_SKIP_CHROMIUM_DOWNLOAD true
ENV CHROME_PATH=/chrome-linux/chrome
ENV PUPPETEER_EXECUTABLE_PATH=/chrome-linux/chrome
# install the rest |
|
I've provided a solution here for mac os -- #6622 (comment) |
|
same issue here |
|
@20manas solution does work. however if you want to abstract the hardcoded url away and have an architecture-independent Dockerfile then you can just |
diff --git a/deployment/provision_vagrant_vm.sh b/deployment/provision_vagrant_vm.sh index 6c1c539f..5d478d2a 100755 --- a/deployment/provision_vagrant_vm.sh +++ b/deployment/provision_vagrant_vm.sh @@ -15,7 +15,7 @@ export DEBIAN_FRONTEND=noninteractive apt-get -q update # system utilities that docker containers don't have -apt-get -q install -y sudo wget git bash-completion +apt-get -q install -y sudo wget git bash-completion software-properties-common # docker weirdly needs this -- see https://stackoverflow.com/questions/46247032/how-to-solve-invoke-rc-d-policy-rc-d-denied-execution-of-start-when-building printf '#!/bin/sh\nexit 0' > /usr/sbin/policy-rc.d @@ -93,6 +93,27 @@ cp $REPO_FOLDER/deployment/manage_autocompletion.sh /etc/bash_completion.d/ # install chrome, see: puppeteer/puppeteer#7740 apt-get -q install -y chromium-browser +# install firefox and geckodriver +cat <<EOT >> /etc/apt/preferences.d/mozillateam +Package: * +Pin: release o=LP-PPA-mozillateam +Pin-Priority: 100 + +Package: firefox* +Pin: release o=LP-PPA-mozillateam +Pin-Priority: 1001 + +Package: firefox* +Pin: release o=Ubuntu +Pin-Priority: -1 +EOT +add-apt-repository -y ppa:mozillateam/ppa +apt-get -q install -y firefox + +wget https://github.com/mozilla/geckodriver/releases/download/v0.33.0/geckodriver-v0.33.0-linux64.tar.gz -O geckodriver.tar.gz +tar xzf geckodriver.tar.gz -C /usr/local/bin/ +chmod +x /usr/local/bin/geckodriver + # install libraries for puppeteer apt-get -q install -y libasound2 libgconf-2-4 libgbm1 libgtk-3-0 libnss3 libx11-xcb1 libxss1 libxshmfence-dev diff --git a/evap/evaluation/tests/test_live.py b/evap/evaluation/tests/test_live.py new file mode 100644 index 00000000..f664a73e --- /dev/null +++ b/evap/evaluation/tests/test_live.py @@ -0,0 +1,35 @@ +from django.core import mail +from django.urls import reverse +from selenium.webdriver.common.by import By +from selenium.webdriver.support import expected_conditions +from selenium.webdriver.support.wait import WebDriverWait + +from evap.evaluation.tests.tools import LiveServerTest + + +class ContactModalTests(LiveServerTest): + def test_contact_modal(self): + self._login() + self.selenium.get(self.live_server_url + reverse("evaluation:index")) + self.selenium.find_element(By.ID, "feedbackModalShowButton").click() + self._screenshot("feedback_modal_") + + WebDriverWait(self.selenium, 10).until( + expected_conditions.visibility_of_element_located((By.ID, "feedbackModalMessageText")) + ) + self._screenshot("feedback_modal_2") + self.selenium.find_element(By.ID, "feedbackModalMessageText").send_keys("Testmessage") + self._screenshot("feedback_modal_typed") + self.selenium.find_element(By.ID, "feedbackModalActionButton").click() + + WebDriverWait(self.selenium, 10).until( + expected_conditions.text_to_be_present_in_element( + (By.CSS_SELECTOR, "#successMessageModal_feedbackModal .modal-body"), + "Your message was successfully sent.", + ) + ) + self._screenshot("feedback_modal_success") + + self.assertEqual(len(mail.outbox), 1) + + self.assertEqual(mail.outbox[0].subject, f"[EvaP] Message from {self.test_user.email}") diff --git a/evap/evaluation/tests/tools.py b/evap/evaluation/tests/tools.py index c1a07d5a..169649fb 100644 --- a/evap/evaluation/tests/tools.py +++ b/evap/evaluation/tests/tools.py @@ -9,10 +9,14 @@ from django.conf import settings from django.contrib.auth.models import Group from django.db import DEFAULT_DB_ALIAS, connections from django.http.request import QueryDict +from django.test.selenium import SeleniumTestCase, SeleniumTestCaseBase from django.test.utils import CaptureQueriesContext from django.utils import timezone from django_webtest import WebTest from model_bakery import baker +from selenium.webdriver.common.by import By +from selenium.webdriver.support import expected_conditions +from selenium.webdriver.support.wait import WebDriverWait from evap.evaluation.models import ( CHOICES, @@ -254,3 +258,53 @@ def assert_no_database_modifications(*args, **kwargs): lower_sql = query["sql"].lower() if not any(lower_sql.startswith(prefix) for prefix in allowed_prefixes): raise AssertionError("Unexpected modifying query found: " + query["sql"]) + + +class CustomSeleniumTestCaseBase(SeleniumTestCaseBase): + external_host = os.environ.get("TEST_HOST", "") or None + browsers = ["firefox"] + selenium_hub = os.environ.get("TEST_SELENIUM_HUB", "") or None + headless = True + + def create_options(self): # pylint: disable=bad-mcs-method-argument + options = super().create_options() + + if self.browser == "chrome": + options.add_argument("--headless") + options.add_argument("--no-sandbox") + options.add_argument("--disable-dev-shm-usage") + options.add_argument("--disable-gpu") + elif self.browser == "firefox": + options.add_argument("--headless") + + return options + + +class LiveServerTest(SeleniumTestCase, metaclass=CustomSeleniumTestCaseBase): + def _screenshot(self, name): + self.selenium.save_screenshot(os.path.join(settings.BASE_DIR, f"{name}.png")) + + def _create_test_user(self): + self.test_user = baker.make( # pylint: disable=attribute-defined-outside-init + UserProfile, email="evap@institution.example.com", groups=[Group.objects.get(name="Manager")] + ) + self.test_user_password = "evap" # pylint: disable=attribute-defined-outside-init + self.test_user.set_password(self.test_user_password) + self.test_user.save() + return self.test_user + + def _login(self): + self._create_test_user() + self.selenium.get(self.live_server_url) + self.selenium.find_element(By.ID, "id_email").click() + self.selenium.find_element(By.ID, "id_email").send_keys(self.test_user.email) + self.selenium.find_element(By.ID, "id_email").click() + self.selenium.find_element(By.ID, "id_password").send_keys(self.test_user_password) + self.selenium.find_element(By.CSS_SELECTOR, ".login-button").click() + self.selenium.save_screenshot(os.path.join(settings.BASE_DIR, "login_success.png")) + + WebDriverWait(self.selenium, 10).until(expected_conditions.presence_of_element_located((By.ID, "logout-form"))) + + @classmethod + def tearDownClass(cls): + cls.selenium.quit() diff --git a/requirements-dev.txt b/requirements-dev.txt index dc456c81..c274373e 100644 --- a/requirements-dev.txt +++ b/requirements-dev.txt @@ -14,3 +14,4 @@ ruff==0.3.5 tblib~=3.0.0 xlrd~=2.0.1 typeguard~=4.2.1 +selenium~=4.15.2
diff --git a/.github/dependabot.yml b/.github/dependabot.yml
index 9b48bb90..3e6ade14 100644
--- a/.github/dependabot.yml
+++ b/.github/dependabot.yml
@@ -18,7 +18,5 @@ updates:
ignore:
- dependency-name: "*"
update-types: ["version-update:semver-patch"]
- - dependency-name: "*puppeteer*"
- update-types: ["version-update:semver-minor"]
labels:
- "[T] Dependencies"
diff --git a/deployment/provision_vagrant_vm.sh b/deployment/provision_vagrant_vm.sh
index c5ead652..3ca66fe6 100755
--- a/deployment/provision_vagrant_vm.sh
+++ b/deployment/provision_vagrant_vm.sh
@@ -90,9 +90,6 @@ sed -i -e "s/\${SECRET_KEY}/$(head /dev/urandom | tr -dc A-Za-z0-9 | head -c 32)
# setup vm auto-completion
cp $REPO_FOLDER/deployment/manage_autocompletion.sh /etc/bash_completion.d/
-# install chrome, see: puppeteer/puppeteer#7740
-apt-get -q install -y chromium-browser
-
# install firefox and geckodriver
sudo install -d -m 0755 /etc/apt/keyrings
wget -q https://packages.mozilla.org/apt/repo-signing-key.gpg -O- | sudo tee /etc/apt/keyrings/packages.mozilla.org.asc > /dev/null
@@ -109,9 +106,6 @@ wget https://github.com/mozilla/geckodriver/releases/download/v0.34.0/geckodrive
tar xzf geckodriver.tar.gz -C /usr/local/bin/
chmod +x /usr/local/bin/geckodriver
-# install libraries for puppeteer
-apt-get -q install -y libasound2 libgconf-2-4 libgbm1 libgtk-3-0 libnss3 libx11-xcb1 libxss1 libxshmfence-dev
-
# install nvm
wget https://raw.githubusercontent.com/nvm-sh/nvm/v0.38.0/install.sh --no-verbose --output-document - | sudo -H -u $USER bash
diff --git a/evap/static/ts/tests/utils/matchers.ts b/evap/static/ts/tests/utils/matchers.ts
deleted file mode 100644
index c82e190e..00000000
--- a/evap/static/ts/tests/utils/matchers.ts
+++ /dev/null
@@ -1,70 +0,0 @@
-import { ElementHandle } from "puppeteer";
-import MatcherUtils = jest.MatcherUtils;
-
-declare global {
- namespace jest {
- interface Matchers<R> {
- toBeChecked(): Promise<R>;
- toHaveClass(className: string): Promise<R>;
- }
- }
-}
-
-function createTagDescription(element: ElementHandle): Promise<string> {
- return element.evaluate(element => {
- let tagDescription = element.tagName.toLowerCase();
- if (element.id) {
- tagDescription += ` id="${element.id}"`;
- }
- if (element.className) {
- tagDescription += ` class="${element.className}"`;
- }
- return `<${tagDescription}>`;
- });
-}
-
-async function createElementMessage(
- this: MatcherUtils,
- matcherName: string,
- expectation: string,
- element: ElementHandle,
- value?: any,
-): Promise<() => string> {
- const tagDescription = await createTagDescription(element);
- return () => {
- const optionallyNot = this.isNot ? "not " : "";
- const receivedLine = value ? `\nReceived: ${this.utils.printReceived(value)}` : "";
- return (
- this.utils.matcherHint(matcherName, undefined, undefined, { isNot: this.isNot }) +
- "\n\n" +
- `Expected ${this.utils.RECEIVED_COLOR(tagDescription)} to ${optionallyNot}${expectation}` +
- receivedLine
- );
- };
-}
-
-expect.extend({
- async toBeChecked(received: ElementHandle): Promise<jest.CustomMatcherResult> {
- const pass = await received.evaluate(element => {
- return (element as HTMLInputElement).checked;
- });
- const message = await createElementMessage.call(this, "toBeChecked", "be checked", received);
- return { message, pass };
- },
-
- async toHaveClass(received: ElementHandle, className: string): Promise<jest.CustomMatcherResult> {
- const classList = await received.evaluate(element => {
- return [...element.classList];
- });
- const pass = classList.includes(className);
- const message = await createElementMessage.call(
- this,
- "toHaveClass",
- `have the class ${this.utils.printExpected(className)}`,
- received,
- classList,
- );
-
- return { message, pass };
- },
-});
diff --git a/evap/static/ts/tests/utils/page.ts b/evap/static/ts/tests/utils/page.ts
deleted file mode 100644
index 0cb49c20..00000000
--- a/evap/static/ts/tests/utils/page.ts
+++ /dev/null
@@ -1,84 +0,0 @@
-import * as fs from "fs";
-import * as path from "path";
-import { Browser, Page } from "puppeteer";
-import { Global } from "@jest/types/";
-import DoneFn = Global.DoneFn;
-
-const contentTypeByExtension: Map<string, string> = new Map([
- [".css", "text/css"],
- [".js", "application/javascript"],
- [".png", "image/png"],
- [".svg", "image/svg+xml"],
-]);
-
-async function createPage(browser: Browser): Promise<Page> {
- const staticPrefix = "/static/";
-
- const page = await browser.newPage();
- await page.setRequestInterception(true);
- page.on("request", request => {
- const extension = path.extname(request.url());
- const pathname = new URL(request.url()).pathname;
- if (extension === ".html") {
- // requests like /evap/evap/static/ts/rendered/results/student.html
- request.continue();
- } else if (pathname.startsWith(staticPrefix)) {
- // requests like /static/css/tom-select.bootstrap5.min.css
- const asset = pathname.substr(staticPrefix.length);
- const body = fs.readFileSync(path.join(__dirname, "..", "..", "..", asset));
- request.respond({
- contentType: contentTypeByExtension.get(extension),
- body,
- });
- } else if (pathname.endsWith("catalog.js")) {
- // request for /catalog.js
- // some pages will error out if translation functions are not available
- // rendered in RenderJsTranslationCatalog
- const absolute_fs_path = path.join(__dirname, "..", "..", "..", "ts", "rendered", "catalog.js");
- const body = fs.readFileSync(absolute_fs_path);
- request.respond({
- contentType: contentTypeByExtension.get(extension),
- body,
- });
- } else {
- request.abort();
- }
- });
- return page;
-}
-
-export function pageHandler(fileName: string, fn: (page: Page) => void): (done?: DoneFn) => void {
- return async done => {
- let finished = false;
- // This wrapper ensures that done() is only called once
- async function finish(reason?: Error) {
- if (!finished) {
- finished = true;
- await page.evaluate(() => {
- localStorage.clear();
- });
- await page.close();
- done!(reason);
- }
- }
-
- const context = await browser.defaultBrowserContext();
- await context.overridePermissions("file:", ["clipboard-read"]);
-
- const page = await createPage(browser);
- page.on("pageerror", async error => {
- await finish(new Error(error.message));
- });
-
- const filePath = path.join(__dirname, "..", "..", "rendered", fileName);
- await page.goto(`file:${filePath}`, { waitUntil: "networkidle0" });
-
- try {
- await fn(page);
- await finish();
- } catch (error) {
- if (error instanceof Error) await finish(error);
- else throw error;
- }
- };
-}
diff --git a/package.json b/package.json
index 63222602..5b0e5a80 100644
--- a/package.json
+++ b/package.json
@@ -2,16 +2,13 @@
"devDependencies": {
"@types/bootstrap": "^5.2.6",
"@types/jest": "^29.5.12",
- "@types/jest-environment-puppeteer": "^5.0.3",
"@types/jquery": "^3.5.16",
"@types/sortablejs": "^1.15.1",
"jest": "^29.7.0",
"jest-environment-jsdom": "^29.7.0",
- "jest-environment-puppeteer": "^10.0.0",
"jest-jasmine2": "^29.7.0",
"jest-ts-webcompat-resolver": "^1.0.0",
"prettier": "^3.2.2",
- "puppeteer": "^21.0.1",
"sass": "1.74.1",
"ts-jest": "^29.1.0",
"typescript": "^5.4.2"
@@ -30,9 +27,6 @@
"transform": {
"^.+\\.ts$": "ts-jest"
},
- "globalSetup": "jest-environment-puppeteer/setup",
- "globalTeardown": "jest-environment-puppeteer/teardown",
- "testEnvironment": "jest-environment-puppeteer",
"resolver": "jest-ts-webcompat-resolver"
}
}
diff --git a/deployment/provision_vagrant_vm.sh b/deployment/provision_vagrant_vm.sh index 6c1c539f..5d478d2a 100755 --- a/deployment/provision_vagrant_vm.sh +++ b/deployment/provision_vagrant_vm.sh @@ -15,7 +15,7 @@ export DEBIAN_FRONTEND=noninteractive apt-get -q update # system utilities that docker containers don't have -apt-get -q install -y sudo wget git bash-completion +apt-get -q install -y sudo wget git bash-completion software-properties-common # docker weirdly needs this -- see https://stackoverflow.com/questions/46247032/how-to-solve-invoke-rc-d-policy-rc-d-denied-execution-of-start-when-building printf '#!/bin/sh\nexit 0' > /usr/sbin/policy-rc.d @@ -93,6 +93,27 @@ cp $REPO_FOLDER/deployment/manage_autocompletion.sh /etc/bash_completion.d/ # install chrome, see: puppeteer/puppeteer#7740 apt-get -q install -y chromium-browser +# install firefox and geckodriver +cat <<EOT >> /etc/apt/preferences.d/mozillateam +Package: * +Pin: release o=LP-PPA-mozillateam +Pin-Priority: 100 + +Package: firefox* +Pin: release o=LP-PPA-mozillateam +Pin-Priority: 1001 + +Package: firefox* +Pin: release o=Ubuntu +Pin-Priority: -1 +EOT +add-apt-repository -y ppa:mozillateam/ppa +apt-get -q install -y firefox + +wget https://github.com/mozilla/geckodriver/releases/download/v0.33.0/geckodriver-v0.33.0-linux64.tar.gz -O geckodriver.tar.gz +tar xzf geckodriver.tar.gz -C /usr/local/bin/ +chmod +x /usr/local/bin/geckodriver + # install libraries for puppeteer apt-get -q install -y libasound2 libgconf-2-4 libgbm1 libgtk-3-0 libnss3 libx11-xcb1 libxss1 libxshmfence-dev diff --git a/evap/evaluation/tests/test_live.py b/evap/evaluation/tests/test_live.py new file mode 100644 index 00000000..f664a73e --- /dev/null +++ b/evap/evaluation/tests/test_live.py @@ -0,0 +1,35 @@ +from django.core import mail +from django.urls import reverse +from selenium.webdriver.common.by import By +from selenium.webdriver.support import expected_conditions +from selenium.webdriver.support.wait import WebDriverWait + +from evap.evaluation.tests.tools import LiveServerTest + + +class ContactModalTests(LiveServerTest): + def test_contact_modal(self): + self._login() + self.selenium.get(self.live_server_url + reverse("evaluation:index")) + self.selenium.find_element(By.ID, "feedbackModalShowButton").click() + self._screenshot("feedback_modal_") + + WebDriverWait(self.selenium, 10).until( + expected_conditions.visibility_of_element_located((By.ID, "feedbackModalMessageText")) + ) + self._screenshot("feedback_modal_2") + self.selenium.find_element(By.ID, "feedbackModalMessageText").send_keys("Testmessage") + self._screenshot("feedback_modal_typed") + self.selenium.find_element(By.ID, "feedbackModalActionButton").click() + + WebDriverWait(self.selenium, 10).until( + expected_conditions.text_to_be_present_in_element( + (By.CSS_SELECTOR, "#successMessageModal_feedbackModal .modal-body"), + "Your message was successfully sent.", + ) + ) + self._screenshot("feedback_modal_success") + + self.assertEqual(len(mail.outbox), 1) + + self.assertEqual(mail.outbox[0].subject, f"[EvaP] Message from {self.test_user.email}") diff --git a/evap/evaluation/tests/tools.py b/evap/evaluation/tests/tools.py index c1a07d5a..169649fb 100644 --- a/evap/evaluation/tests/tools.py +++ b/evap/evaluation/tests/tools.py @@ -9,10 +9,14 @@ from django.conf import settings from django.contrib.auth.models import Group from django.db import DEFAULT_DB_ALIAS, connections from django.http.request import QueryDict +from django.test.selenium import SeleniumTestCase, SeleniumTestCaseBase from django.test.utils import CaptureQueriesContext from django.utils import timezone from django_webtest import WebTest from model_bakery import baker +from selenium.webdriver.common.by import By +from selenium.webdriver.support import expected_conditions +from selenium.webdriver.support.wait import WebDriverWait from evap.evaluation.models import ( CHOICES, @@ -254,3 +258,53 @@ def assert_no_database_modifications(*args, **kwargs): lower_sql = query["sql"].lower() if not any(lower_sql.startswith(prefix) for prefix in allowed_prefixes): raise AssertionError("Unexpected modifying query found: " + query["sql"]) + + +class CustomSeleniumTestCaseBase(SeleniumTestCaseBase): + external_host = os.environ.get("TEST_HOST", "") or None + browsers = ["firefox"] + selenium_hub = os.environ.get("TEST_SELENIUM_HUB", "") or None + headless = True + + def create_options(self): # pylint: disable=bad-mcs-method-argument + options = super().create_options() + + if self.browser == "chrome": + options.add_argument("--headless") + options.add_argument("--no-sandbox") + options.add_argument("--disable-dev-shm-usage") + options.add_argument("--disable-gpu") + elif self.browser == "firefox": + options.add_argument("--headless") + + return options + + +class LiveServerTest(SeleniumTestCase, metaclass=CustomSeleniumTestCaseBase): + def _screenshot(self, name): + self.selenium.save_screenshot(os.path.join(settings.BASE_DIR, f"{name}.png")) + + def _create_test_user(self): + self.test_user = baker.make( # pylint: disable=attribute-defined-outside-init + UserProfile, email="evap@institution.example.com", groups=[Group.objects.get(name="Manager")] + ) + self.test_user_password = "evap" # pylint: disable=attribute-defined-outside-init + self.test_user.set_password(self.test_user_password) + self.test_user.save() + return self.test_user + + def _login(self): + self._create_test_user() + self.selenium.get(self.live_server_url) + self.selenium.find_element(By.ID, "id_email").click() + self.selenium.find_element(By.ID, "id_email").send_keys(self.test_user.email) + self.selenium.find_element(By.ID, "id_email").click() + self.selenium.find_element(By.ID, "id_password").send_keys(self.test_user_password) + self.selenium.find_element(By.CSS_SELECTOR, ".login-button").click() + self.selenium.save_screenshot(os.path.join(settings.BASE_DIR, "login_success.png")) + + WebDriverWait(self.selenium, 10).until(expected_conditions.presence_of_element_located((By.ID, "logout-form"))) + + @classmethod + def tearDownClass(cls): + cls.selenium.quit() diff --git a/requirements-dev.txt b/requirements-dev.txt index dc456c81..c274373e 100644 --- a/requirements-dev.txt +++ b/requirements-dev.txt @@ -14,3 +14,4 @@ ruff==0.3.5 tblib~=3.0.0 xlrd~=2.0.1 typeguard~=4.2.1 +selenium~=4.15.2
diff --git a/.github/dependabot.yml b/.github/dependabot.yml
index 9b48bb90..3e6ade14 100644
--- a/.github/dependabot.yml
+++ b/.github/dependabot.yml
@@ -18,7 +18,5 @@ updates:
ignore:
- dependency-name: "*"
update-types: ["version-update:semver-patch"]
- - dependency-name: "*puppeteer*"
- update-types: ["version-update:semver-minor"]
labels:
- "[T] Dependencies"
diff --git a/deployment/provision_vagrant_vm.sh b/deployment/provision_vagrant_vm.sh
index c5ead652..3ca66fe6 100755
--- a/deployment/provision_vagrant_vm.sh
+++ b/deployment/provision_vagrant_vm.sh
@@ -90,9 +90,6 @@ sed -i -e "s/\${SECRET_KEY}/$(head /dev/urandom | tr -dc A-Za-z0-9 | head -c 32)
# setup vm auto-completion
cp $REPO_FOLDER/deployment/manage_autocompletion.sh /etc/bash_completion.d/
-# install chrome, see: puppeteer/puppeteer#7740
-apt-get -q install -y chromium-browser
-
# install firefox and geckodriver
sudo install -d -m 0755 /etc/apt/keyrings
wget -q https://packages.mozilla.org/apt/repo-signing-key.gpg -O- | sudo tee /etc/apt/keyrings/packages.mozilla.org.asc > /dev/null
@@ -109,9 +106,6 @@ wget https://github.com/mozilla/geckodriver/releases/download/v0.34.0/geckodrive
tar xzf geckodriver.tar.gz -C /usr/local/bin/
chmod +x /usr/local/bin/geckodriver
-# install libraries for puppeteer
-apt-get -q install -y libasound2 libgconf-2-4 libgbm1 libgtk-3-0 libnss3 libx11-xcb1 libxss1 libxshmfence-dev
-
# install nvm
wget https://raw.githubusercontent.com/nvm-sh/nvm/v0.38.0/install.sh --no-verbose --output-document - | sudo -H -u $USER bash
diff --git a/evap/static/ts/tests/utils/matchers.ts b/evap/static/ts/tests/utils/matchers.ts
deleted file mode 100644
index c82e190e..00000000
--- a/evap/static/ts/tests/utils/matchers.ts
+++ /dev/null
@@ -1,70 +0,0 @@
-import { ElementHandle } from "puppeteer";
-import MatcherUtils = jest.MatcherUtils;
-
-declare global {
- namespace jest {
- interface Matchers<R> {
- toBeChecked(): Promise<R>;
- toHaveClass(className: string): Promise<R>;
- }
- }
-}
-
-function createTagDescription(element: ElementHandle): Promise<string> {
- return element.evaluate(element => {
- let tagDescription = element.tagName.toLowerCase();
- if (element.id) {
- tagDescription += ` id="${element.id}"`;
- }
- if (element.className) {
- tagDescription += ` class="${element.className}"`;
- }
- return `<${tagDescription}>`;
- });
-}
-
-async function createElementMessage(
- this: MatcherUtils,
- matcherName: string,
- expectation: string,
- element: ElementHandle,
- value?: any,
-): Promise<() => string> {
- const tagDescription = await createTagDescription(element);
- return () => {
- const optionallyNot = this.isNot ? "not " : "";
- const receivedLine = value ? `\nReceived: ${this.utils.printReceived(value)}` : "";
- return (
- this.utils.matcherHint(matcherName, undefined, undefined, { isNot: this.isNot }) +
- "\n\n" +
- `Expected ${this.utils.RECEIVED_COLOR(tagDescription)} to ${optionallyNot}${expectation}` +
- receivedLine
- );
- };
-}
-
-expect.extend({
- async toBeChecked(received: ElementHandle): Promise<jest.CustomMatcherResult> {
- const pass = await received.evaluate(element => {
- return (element as HTMLInputElement).checked;
- });
- const message = await createElementMessage.call(this, "toBeChecked", "be checked", received);
- return { message, pass };
- },
-
- async toHaveClass(received: ElementHandle, className: string): Promise<jest.CustomMatcherResult> {
- const classList = await received.evaluate(element => {
- return [...element.classList];
- });
- const pass = classList.includes(className);
- const message = await createElementMessage.call(
- this,
- "toHaveClass",
- `have the class ${this.utils.printExpected(className)}`,
- received,
- classList,
- );
-
- return { message, pass };
- },
-});
diff --git a/evap/static/ts/tests/utils/page.ts b/evap/static/ts/tests/utils/page.ts
deleted file mode 100644
index 0cb49c20..00000000
--- a/evap/static/ts/tests/utils/page.ts
+++ /dev/null
@@ -1,84 +0,0 @@
-import * as fs from "fs";
-import * as path from "path";
-import { Browser, Page } from "puppeteer";
-import { Global } from "@jest/types/";
-import DoneFn = Global.DoneFn;
-
-const contentTypeByExtension: Map<string, string> = new Map([
- [".css", "text/css"],
- [".js", "application/javascript"],
- [".png", "image/png"],
- [".svg", "image/svg+xml"],
-]);
-
-async function createPage(browser: Browser): Promise<Page> {
- const staticPrefix = "/static/";
-
- const page = await browser.newPage();
- await page.setRequestInterception(true);
- page.on("request", request => {
- const extension = path.extname(request.url());
- const pathname = new URL(request.url()).pathname;
- if (extension === ".html") {
- // requests like /evap/evap/static/ts/rendered/results/student.html
- request.continue();
- } else if (pathname.startsWith(staticPrefix)) {
- // requests like /static/css/tom-select.bootstrap5.min.css
- const asset = pathname.substr(staticPrefix.length);
- const body = fs.readFileSync(path.join(__dirname, "..", "..", "..", asset));
- request.respond({
- contentType: contentTypeByExtension.get(extension),
- body,
- });
- } else if (pathname.endsWith("catalog.js")) {
- // request for /catalog.js
- // some pages will error out if translation functions are not available
- // rendered in RenderJsTranslationCatalog
- const absolute_fs_path = path.join(__dirname, "..", "..", "..", "ts", "rendered", "catalog.js");
- const body = fs.readFileSync(absolute_fs_path);
- request.respond({
- contentType: contentTypeByExtension.get(extension),
- body,
- });
- } else {
- request.abort();
- }
- });
- return page;
-}
-
-export function pageHandler(fileName: string, fn: (page: Page) => void): (done?: DoneFn) => void {
- return async done => {
- let finished = false;
- // This wrapper ensures that done() is only called once
- async function finish(reason?: Error) {
- if (!finished) {
- finished = true;
- await page.evaluate(() => {
- localStorage.clear();
- });
- await page.close();
- done!(reason);
- }
- }
-
- const context = await browser.defaultBrowserContext();
- await context.overridePermissions("file:", ["clipboard-read"]);
-
- const page = await createPage(browser);
- page.on("pageerror", async error => {
- await finish(new Error(error.message));
- });
-
- const filePath = path.join(__dirname, "..", "..", "rendered", fileName);
- await page.goto(`file:${filePath}`, { waitUntil: "networkidle0" });
-
- try {
- await fn(page);
- await finish();
- } catch (error) {
- if (error instanceof Error) await finish(error);
- else throw error;
- }
- };
-}
diff --git a/package.json b/package.json
index 63222602..5b0e5a80 100644
--- a/package.json
+++ b/package.json
@@ -2,16 +2,13 @@
"devDependencies": {
"@types/bootstrap": "^5.2.6",
"@types/jest": "^29.5.12",
- "@types/jest-environment-puppeteer": "^5.0.3",
"@types/jquery": "^3.5.16",
"@types/sortablejs": "^1.15.1",
"jest": "^29.7.0",
"jest-environment-jsdom": "^29.7.0",
- "jest-environment-puppeteer": "^10.0.0",
"jest-jasmine2": "^29.7.0",
"jest-ts-webcompat-resolver": "^1.0.0",
"prettier": "^3.2.2",
- "puppeteer": "^21.0.1",
"sass": "1.74.1",
"ts-jest": "^29.1.0",
"typescript": "^5.4.2"
@@ -30,9 +27,6 @@
"transform": {
"^.+\\.ts$": "ts-jest"
},
- "globalSetup": "jest-environment-puppeteer/setup",
- "globalTeardown": "jest-environment-puppeteer/teardown",
- "testEnvironment": "jest-environment-puppeteer",
"resolver": "jest-ts-webcompat-resolver"
}
}
This was what worked for me. But you have to use the package manager available in your base image (alpine or debian) |
and others
Bug description
In an ARM based Docker container (or ARM based Linux):
npm install puppeteerError:
This is related to #6622 which wasn't really solved just worked around via Rosetta.
I'm trying to build an ARM based Docker Image that uses BackstopJS/puppeteer. Ref garris/BackstopJS#1300
Are there any plans to build an arm64 version of
chromium?Puppeteer version
10.0.0
Node.js version
16.3.0
npm version
7.15.1
What operating system are you seeing the problem on?
Linux
Relevant log output
No response
The text was updated successfully, but these errors were encountered: