Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Test config changes on main can cause problems for cherry picks to beta and stable #51481

Closed
nshahan opened this issue Feb 21, 2023 · 1 comment
Closed

Test config changes on main can cause problems for cherry picks to beta and stable #51481

nshahan opened this issue Feb 21, 2023 · 1 comment
Labels
area-infrastructure Use area-infrastructure for SDK infrastructure issues, like continuous integration bot changes. web-dart2js web-dev-compiler

Comments

@nshahan
Copy link
Contributor

nshahan commented Feb 21, 2023

This is a tracking issue for potential problems from the test/builder configuration changes we are making for dart2js and ddc.

  • Test and builder configurations are defined in the SDK repo:
    https://github.com/dart-lang/sdk/blob/main/tools/bots/test_matrix.json
  • The infra that runs the builders is configured outside of the SDK and is independent of the SDK main/beta/stable branches.
  • When we make a change that requires changes on both sides they are unlikely to be backwards compatible.

Cherry picks for any other reason that happen after the configuration change lands in the SDK and before it gets merged into the cherry pick destination branch will not be able to run tests with the ddc or dart2js builders.

In that event, we should also cherry pick the configuration changes that landed in the SDK.

cc @athomas
@nshahan nshahan added web-dart2js area-infrastructure Use area-infrastructure for SDK infrastructure issues, like continuous integration bot changes. web-dev-compiler labels Feb 21, 2023
This was referenced Feb 22, 2023
Closed
Closed
copybara-service bot pushed a commit that referenced this issue Feb 28, 2023
@nshahan @athomas
[stable][ddc] Cleanup names of modular tests
da32dab 
Fix names in test matrix to match what is actually running on
each bot.

Updated file names:
- Runs with sound null safety
  modular_suite_nnbd.dart -> modular_suite.dart

- Runs without sound null safety
  modular_suite.dart -> modular_suite_unsound.dart

Updated configs to record status:
- Sound
  dartdevk-strong-linux-release-chrome
- Unsound
  dartdevk-checked-linux-release-chrome
  dartdevk-checked-mac-release-chrome
  dartdevk-checked-win-release-chrome
  dartdevc-canary-weak-linux-release-chrome-x64

Issue: #51481
Cherry-pick: https://dart-review.googlesource.com/c/sdk/+/279471
Change-Id: I4048034676bd87674cf1f545243882a13f3c4c66
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/285920
Reviewed-by: Alexander Thomas <athom@google.com>
copybara-service bot pushed a commit that referenced this issue Feb 28, 2023
@nshahan @athomas
[stable][ddc] Cleanup sourcemap and stacktrace tests
739ad03 
- Remove all refrences to "ddk" vs "ddc".
- Update throw_in_instance_method to reflect the current sourcemappings
  and start running the test again.

Issue: #51481
Cherry-pick: https://dart-review.googlesource.com/c/sdk/+/279473
Change-Id: I7bb30149e83938498d6d95cf52f14c66441d798f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/285921
Reviewed-by: Alexander Thomas <athom@google.com>
copybara-service bot pushed a commit that referenced this issue Feb 28, 2023
@nshahan @athomas
[stable][ddc] Refactor modular test suites
360c9b7 
- Move DDC modular pipeline to helper library.
- Add null safety flag to `SourceToSummaryDillStep`.
- Add null safety and canary mode flags to `DDCStep`.
- Add new suite that runs in canary mode without sound null safety.

Issue: #51481
Cherry-pick: https://dart-review.googlesource.com/c/sdk/+/279478
Change-Id: Ic62269cf19735675a7104c5239117a06c84af09a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/285922
Reviewed-by: Alexander Thomas <athom@google.com>
copybara-service bot pushed a commit that referenced this issue Feb 28, 2023
@nshahan @athomas
[stable][ddc] Add ddc-options to compiler configuration
0ec8740 
Use explicit ddc-options to pass `--canary` flag to the compiler instead
of sniffing for a builder tag.

This just feels more expected and makes more sense if you are trying
to understand how the canary configurations compile tests differently
from the stable configurations.

Issue: #51481
Cherry-pick: https://dart-review.googlesource.com/c/sdk/+/280783
Change-Id: I6cc7c6a0a573b6f397b8b183dff13758f816fd33
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/284542
Reviewed-by: Alexander Thomas <athom@google.com>
copybara-service bot pushed a commit that referenced this issue Feb 28, 2023
@nshahan @athomas
[stable][ddc] Add new name for ddc compiler
9e95988 
* The new compiler name "ddc" will be used for all configurations
  in an upcoming change.
* Aliases "dartdevc" and "dartdevk" will be removed in the future.

Issue: #51481
Cherry-pick: https://dart-review.googlesource.com/c/sdk/+/280862
Change-Id: Icd81c63bff57911816a4e187a40e4a4012eead85
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/284543
Reviewed-by: Alexander Thomas <athom@google.com>
copybara-service bot pushed a commit that referenced this issue Feb 28, 2023
@nshahan @athomas
[stable][ddc] Add new test and builder configs
cead04d 
Issue: #51481
Cherry-pick: https://dart-review.googlesource.com/c/sdk/+/279750
Change-Id: I84dbbe238f34ddf945ebab709995a021802d9cf3
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/284544
Reviewed-by: Alexander Thomas <athom@google.com>
This was referenced Mar 9, 2023
Closed
Closed
copybara-service bot pushed a commit that referenced this issue Mar 13, 2023
@nshahan
[beta][dart2js] Update tests matrix entries
f7575a0 
- Update options for existing configurations.
- Add one new configuration for running _2 test suites.
- Rename builders and update to run new configurations.

Tested: Tested with led runs.
Issue: #51481
Cherry-pick: https://dart-review.googlesource.com/c/sdk/+/281348
Change-Id: I8c71757ab605edf0bf91bbae28e78d38b2dbabe9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/287905
Reviewed-by: Alexander Thomas <athom@google.com>
copybara-service bot pushed a commit that referenced this issue Mar 15, 2023
@nshahan
[stable][dart2js] Only test matrix changes
2c89851 
Issue: #51481
Cherry-pick: https://dart-review.googlesource.com/c/sdk/+/270860
Change-Id: I4e1ba3ac1eb36993232950ed917fc75b7fd8f5b4
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/287900
Reviewed-by: Alexander Thomas <athom@google.com>
copybara-service bot pushed a commit that referenced this issue Mar 15, 2023
@nshahan
[stable][dart2js] Remove all unused test configs
b5dbbfe 
Reduce the test matrix configurations to only those used in actual
build bots.

Issue: #51481
Cherry-pick: https://dart-review.googlesource.com/c/sdk/+/281346
Change-Id: I07c7313c10aa415cd0ec884fb27499cfeae9a0ab
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/287901
Reviewed-by: Alexander Thomas <athom@google.com>
copybara-service bot pushed a commit that referenced this issue Mar 15, 2023
@nshahan
[stable][dart2js] Add new test configurations
e1243ca 
Issue: #51481
Cherry-pick: https://dart-review.googlesource.com/c/sdk/+/281347
Change-Id: Ieebd213844d5da33f0b8001722c4c027348adadf
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/287902
Reviewed-by: Alexander Thomas <athom@google.com>
copybara-service bot pushed a commit that referenced this issue Mar 15, 2023
@nshahan
[stable][dart2js] Update tests matrix entries
70de72a 
- Update options for existing configurations.
- Add one new configuration for running _2 test suites.
- Rename builders and update to run new configurations.

Tested: Tested with led runs.
Issue: #51481
Cherry-pick: https://dart-review.googlesource.com/c/sdk/+/281348
Change-Id: I695525b0bfa936ad6c5c7f2ecac212299de969c2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/287903
Reviewed-by: Alexander Thomas <athom@google.com>
@nshahan
Copy link
Contributor Author

nshahan commented Apr 7, 2023

I've landed all the changes and cherry picks. Closing this tracking issue.

@nshahan nshahan closed this as completed Apr 7, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
area-infrastructure Use area-infrastructure for SDK infrastructure issues, like continuous integration bot changes. web-dart2js web-dev-compiler
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@nshahan