Artillery v2.0.12

AWS Lambda

Artillery can now use Lambda Container Images to run tests on AWS Lambda. Pass the --container flag to run-lambda command to use a container instead of a zip file. No further configuration is required.

This change brings two main benefits:

1. Much faster startup times for tests
2. Ability to use certain features that were previously only available in tests running on AWS Fargate:

• Run test scripts that TypeScript hooks
• Use third-party dependencies
• Use the built-in publish-metrics plugin

Container-based Lambda functions will become the default option in a future release of Artillery.

• #2674 • #2724

Playwright

• Update Playwright to v1.44.0
• Fix tests failing on Fargate with ENOSPC error with Playwright tracing enabled #2717
• Improve reliability of trace collection:
  • Reduce trace collection cooldown period to 1-5m
  • Increase the number of concurrent trace recordings to 3 up from 1
• Track the number of collected/discarded recordings with browser.traces_collected and browser.traces_discarded metrics

CLI & Core

• Make path to the config directory available to test scripts via the special $dirname variable #2614 docs
• The --scenario-name flag will now default to using an exact string match, and fall back to using a regex #2709
• Clarify the meaning of --tags flag #2699
• Remove an unnecessary dependency #2703

Slack integration

• Artillery now includes an official Slack integration, which can be used to post test results to a Slack channel automatically #2688 docs

---

With contributions from: @bernardobridge @InesNi @hassy @43081j @MIJOTHY-V2

Artillery v2.0.11

Playwright

• Display errors from the Playwright engine in Artillery reports and the console (#2677) - docs
• Report errors from failed Playwright assertions (#2677) - docs

Fargate

• Fix flag allowing custom IAM role in Fargate tasks (--task-role-name ) (#2669) - docs

CLI & Core

• Log an error message when using an unsupported expectation with the expect plugin (#2667)

---

This release brought to you by @InesNi

Artillery v2.0.10

CLI & Core

• Add parallel action for making parallel requests in HTTP tests

Fargate

• Add ability to suppress metrics output with --quiet flag (#2639) - docs
• Add eu-north-1 to supported regions (#2646)

Lambda

• Add ability to suppress metrics output with --quiet flag (#2639) - docs

Playwright

• Fix issue preventing stack traces for errors from being displayed (#2643)
• Update Playwright version to 1.43.1 (#2644) - docs

Publish Metrics

• Make reporters using OpenTelemetry compatible with parallel requests (#2628)

---

This release brought to you by @hassy @InesNi @bernardobridge

Artillery v2.0.9

Publish Metrics

• Sending traces to AWS X-Ray is now available through the CloudWatch reporter (#2555) - Docs
• Allow replacing of all tracing span names with replaceSpanNameRegex config option (#2607) - Docs
• Fix bug in Datadog reporter causing last batches of traces to occasionally drop (#2610)
• Fix errors happening when sampleRate tracing option set (#2601)
• Fix errors in tracing happening when using HTTP streaming (#2601)
• Display number of spans exported in the Artillery report (plugins.publish-metrics.spans.exported ) when tracing is enabled. (#2597)

CLI & Core

• Fix incorrectly declared dependencies in Artillery CLI (#2549)

---

This release brought to you by @InesNi @95th

Artillery v2.0.8

Artillery Cloud

• Fix issue preventing ARTILLERY_CLOUD_API_KEY environment variable from being picked up by tests on Fargate when using --dotenv (#2585)

Metrics by Endpoint

• Report request URLs for requests that resulted in a network error such as ETIMEOUT or ECONNRESET (#2574) - docs

CLI and Core

• Fix issue causing some intermediate reports to not be reported (#2565)
• Fix issue with incorrect formatting of time units in “phase started” messages (#2581)

---

This release brought to you by @hassy @bernardobridge

Artillery v2.0.7

CLI and Core

• Add support for loading custom code from ES modules (#2522) - docs
• Add support for using async functions as hooks (#2522) - docs

Playwright

• Update Playwright version to 1.42.1
• Fix bug that prevented traces from Playwright tests from being sent to Datadog and Honeycomb (#2533)

AWS Fargate

• Tests running in Fargate will now use AWS Distro for OpenTelemetry instead of Datadog Agent for sending traces to Datadog (#2481) - docs
• Fix bug that prevented setting CPU and memory allocation with the --launch-config flag (#2526)

---

This release brought to you by @hassy @bernardobridge @InesNi

Artillery v2.0.6

Playwright

• Make expect from @playwright/test work with Typescript support (#2488) - Documentation
• Playwright’s baseURL is now set from target (#2493) - Documentation
• Update Playwright version to 1.41.2 (#2492) - Documentation

Artillery Cloud

• Load phases are now shown in Artillery Dashboard for Fargate tests (#2485)

CLI & Core

• Improve Typescript support to work with more packages, by exposing config.bundling.external to set packages that can’t be bundled as external (#2488) - Documentation
• Make test run id available to test runs via {{ $testId }} in scenarios/config and vuContext.vars.$testId in custom code (#2499)
• Routine dependency updates (#2500)

Publish Metrics

• Add test_id attribute to metrics and traces sent via the OpenTelemetry reporter (#2505) - Documentation

Fargate

• Improve error message and handling when setting CloudWatch retention policy (#2489)

---

This release brought to you by @bernardobridge, @InesNi, @hassy

Artillery v2.0.5

New release cadence

A new stable version of the Artillery CLI will now be released twice a month - on the first and third Monday of the month. Canary releases will continue being released from the main branch.

AWS Fargate

• Add support for running tests with Fargate Spot (#2464) - Documentation
• Set default bucket lifecycle policy for the S3 bucket containing artifacts from test runs (#2434) - Documentation
• Fix issue with traces sent via the OpenTelemetry reporter causing spans to go missing or be formatted incorrectly (#2439)
• Set default retention policy on CloudWatch logs (#2446) - Documentation
• Force headless mode for Playwright tests running on Fargate (#2455)
• Fix bug causing Fargate tests triggered on Windows to crash when running with custom JS code (#2431)
• Fix bug that would cause Fargate workers to crash on startup (#2428, #2459)

Playwright

• Track and report the INP Core Web Vitals metrics (#2461) - Documentation
• Add support for configuring Playwright’s testIdAttribute setting (#2441) - Documentation
• Update Playwright version used by Artillery to v1.41.0 (#2425) - Documentation

publish-metrics plugin

• Add tracing support for the Datadog, New Relic and Dynatrace reporters using OpenTelemetry (#2418, #2466)
• Improve traces sent to Honeycomb to include full metadata, exceptions, and events (#2418, #2466)

AWS Lambda

• Disable artillery-plugin-publish-metrics on Lambda (#2457). This is a temporary workaround to get tests to run on Lambda reliably again
• Add default bucket lifecycle policy for S3 bucket containing artifacts from test runs (#2434) - Documentation

CLI & Core

• Fix $randomString(length) utility to return correct number of characters and allow length larger than 10 (#2342)

Artillery Cloud

• Fix bug that prevented CloudWatch logs for all workers from being shown on test reports (#2421)

---

This release brought to you by @InesNi, @hassy, @bernardobridge, @Georgegriff

Artillery v2.0.4

Fargate

• Enable support for 12 more AWS regions (eu-west-2, eu-west-3, us-east-2, us-west-2, ca-central-1, ap-east-1, ap-northeast-2, ap-southeast-1, ap-southeast-2, ap-northeast-1, me-south-1, sa-east-1 ) (#2358, 29a4861)
• Add support for sending Playwright and HTTP traces to Datadog APM (#2381 and #2393) - Documentation
• Add support for --variables flag in run-fargate command (#2391) - Documentation
• Show supported regions when using the --region flag (960b5ae)

CLI & Core

• Add TypeScript support for custom code for tests running locally or on AWS Fargate. This is an experimental feature. (#2360) - Documentation

Playwright

• VUs will now run in isolated browsers contexts rather than new browser instances by default for better performance. (#2382) - Documentation
• HTTP status codes are now tracked and reported through the browser.page.codes.<code> metric (#2383) - Documentation
• Fix bug in before/ after hooks that prevented the Playwright engine from working (#2396) - Documentation
• Add experimental TypeScript support. See the docs for current limitations. (#2382) - Documentation

OpenTelemetry

• Fix issue with virtual users not failing when using test.step() with OpenTelemetry Playwright tracing enabled (#2368)
• OpenTelemetry tracing can now be used to send HTTP and Playwright engine traces to Datadog in Fargate tests (#2381 and #2393) - Documentation
• Add support for smart sampling when using HTTP tracing with the OpenTelemetry reporter (#2343)

---

This release brought to you by @InesNi @hassy @bernardobridge

Artillery v2.0.3

Fixes & Improvements

• Fix a bug that prevented tests from running on AWS Lambda (#2355)
• Fix a bug that prevented tests from running on AWS Fargate (#2354)