Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: enable flakybot for test failure detection #7195

Open
noahdietz opened this issue Apr 2, 2024 · 3 comments
Open

ci: enable flakybot for test failure detection #7195

noahdietz opened this issue Apr 2, 2024 · 3 comments
Assignees
Labels
type: process A process-related concern. May include testing, release, or the like.

Comments

@noahdietz
Copy link
Contributor

We should install the flakybot app to notify us of failing tests in our continuous job.

@vishwarajanand has identified a set of already failing tests. Once we do a bit of clean up, we can start the process of enabling the app on the repo. Those tests that aren't addressed initially should result in new issues being created.

@noahdietz noahdietz added the type: process A process-related concern. May include testing, release, or the like. label Apr 2, 2024
@noahdietz noahdietz transferred this issue from googleapis/gapic-generator-php Apr 2, 2024
@noahdietz
Copy link
Contributor Author

Based on the docs, since this repo is in googleapis, the app is already installed. We need to make sure the other setup is aligned.

@noahdietz
Copy link
Contributor Author

It looks like only a few steps are necessary to enable it at this point.
image

here is where we have sponge_log files being created:

mkdir -p ${SHORT_JOB_NAME}/unit
mkdir -p ${SHORT_JOB_NAME}/snippets
mkdir -p ${SHORT_JOB_NAME}/system
UNIT_LOG_FILENAME=${SHORT_JOB_NAME}/unit/sponge_log.xml
SNIPPETS_LOG_FILENAME=${SHORT_JOB_NAME}/snippets/sponge_log.xml
SYSTEM_LOG_FILENAME=${SHORT_JOB_NAME}/system/sponge_log.xml

but then they are populated with junit format?

echo "Running Unit Test Suite"
vendor/bin/phpunit -c phpunit${PHPUNIT_SUFFIX}.xml.dist --log-junit ${UNIT_LOG_FILENAME}
echo "Running Snippet Test Suite"
vendor/bin/phpunit -c phpunit-snippets.xml.dist --verbose --log-junit \
${SNIPPETS_LOG_FILENAME}
echo "Running System Test Suite"
vendor/bin/phpunit -d memory_limit=512M -c phpunit${PHPUNIT_SUFFIX}-system.xml.dist \
--verbose --log-junit ${SYSTEM_LOG_FILENAME}

not sure if that needs changing.

@vishwarajanand do you mind handling this? It's already installed on the repo per the documentation

@vishwarajanand
Copy link
Contributor

I've fixed all storage tests (few PRs pending reviews though).
I am looking for Spanner tests to be fixed (most of them have similar failures).
Once thats done, I will look into this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: process A process-related concern. May include testing, release, or the like.
Projects
None yet
Development

No branches or pull requests

2 participants