Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set default test size to "small" #22355

Open
matts1 opened this issue May 13, 2024 · 0 comments
Open

Set default test size to "small" #22355

matts1 opened this issue May 13, 2024 · 0 comments
Labels
team-Bazel General Bazel product/strategy issues type: feature request untriaged

Comments

@matts1
Copy link

matts1 commented May 13, 2024

Description of the feature request:

If unspecified, a test's size currently defaults to "medium". I think small is a far more reasonable default.

aspect_bazel_lib says:

The [test-encyclopedia](https://bazel.build/reference/test-encyclopedia) says:
* Tests may return arbitrarily fast regardless of timeout.
* A test is not penalized for an overgenerous timeout, although a warning may be issued:
* You should generally set your timeout as tight as you can without incurring any flakiness.

However Bazel's default for timeout is medium, which is dumb given this guidance.

I'm inclined to agree with this. The vast majority of tests should be marked as small, so it would be more convenient, and given the guidance, it would make sense to set the timeout as tight as is allowed (small) by default, and then bazel will timeout and tell you to increase the timeout.

To achieve this, I request creating a flag --incompatible_default_test_size and default it to medium. Then in a future release, we can change it to small, before eventually deleting the flag.

Which category does this issue belong to?

Java Rules

What underlying problem are you trying to solve with this feature?

Our pre-upload checks add the flag --test_size_filters=small. I just recently ran into an issue where the size of a test was not specified, and so the test was skipped. I think defaulting it to small is far more useful, and also means that users don't have to remember to set the test size - instead, they will know to set the test size because the test fails from timing out.

Which operating system are you running Bazel on?

Linux

What is the output of bazel info release?

7.1.1

If bazel info release returns development version or (@non-git), tell us how you built Bazel.

No response

What's the output of git remote get-url origin; git rev-parse HEAD ?

No response

Have you found anything relevant by searching the web?

No response

Any other information, logs, or outputs that you want to share?

No response

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
team-Bazel General Bazel product/strategy issues type: feature request untriaged
Projects
None yet
Development

No branches or pull requests

5 participants