Some lively.morphic/text
tests regularly fail in CI
#1524
Labels
🔍 cause unknown
A problem which needs more investigation/is not yet reproducible
🐞 bug
Something isn't working
⏱️ postponed
A problem/idea that is known/documented but is not urgent/has previous pathdependencies.
Ever since merging #1481 our CI pipelines failed more or less reliably.
We often had trouble reproducing this locally. Although it was not impossible to reproduce it locally, it did work more often than not and inside of a running lively environment the tests never failed.
We found out that disabling the caching inside of the actions, thus enforcing a clean reinstallation of lively reliably seems to solve the issue.
We then tried to move the lively.morphic tests, which are the failing tests, to the start of the pipeline for easier testing. This lead to the tests failing again.
We briefly experimented with setting
headless: 'shell'
instead ofheadless: 'new'
. However, this would need to be investigated further.Overall the situation is highly unsatisfactory and we have no real indicator what the exact cause for the problems is. For now, we just hope that the problem does not persist, but we should investigate how we can harden the pipeline. Enabling the caching again would also be quite nice as the runtime of the pipelines skyrockets drastically, which is especially annoying for the PR checks.
In the worst case we could try to force the installation only for the daily checks and for
lively.morpic
.The text was updated successfully, but these errors were encountered: