Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deduplicate incompatible target_compatible_with labels #16274

Conversation

philsc
Copy link
Contributor

@philsc philsc commented Sep 14, 2022

The latest refactor unintentionally made it so you can list the same
incompatible constraint multiple times in the target_compatible_with
attribute.

It was unintentional, but actually greatly simplifies a discussion
point on bazelbuild/bazel-skylib#381. That skylib patch aims to make
it easier to write non-trivial target_compatible_with expressions.
For example, to express that something is compatible with everything
except for Windows, one can use the following:

foo_binary(
    name = "bin",
    target_compatible_with = select({
        "@platforms//os:windows": ["@platforms//:incomaptible"],
        "//conditions:default: [],
    }),
)

The skylib patch aims to reduce that to:

foo_binary(
    name = "bin",
    target_compatible_with = compatibility.none_of("@platforms//os:windows"),
)

This works fine on its own, but a problem arises when these
expressions are composed. For example, a macro author might write the
following:

def foo_wrapped_binary(name, target_compatible_with = [], **kwargs):
    foo_binary(
        name = name,
        target_compatible_with = target_compatible_with + select({
            "@platforms//os:none": ["@platforms//:incompatible"],
            "//conditions:default": [],
        }),
    )

A macro author should be able to express their own incompatibility
while also honouring user-defined incompatibility.

It turns out that my latest refactor (#14096) unintentionally made
this work. This happened because we now check for incompatibility
before we perform a lot of sanity checks. That's also one of the
reasons that visibility restrictions are not honoured for incomaptible
targets at the moment (#16044).

Without the deduplicating behaviour, macro authors are stuck with
either not allowing composition, or having to create unique
incompatible constraints for every piece in a composed
target_compatible_with expression.

This patch adds a test to validate the deduplicating behaviour to
cement it as a feature.

Unfortunately, this means that target_compatible_with behaves
differently from other label list attributes. For other label list
attributes, bazel complains when labels are duplicated. For example,
the following BUILD file:

py_library(
    name = "lib",
)

py_binary(
    name = "bin",
    srcs = ["bin.py"],
    deps = [
    |   ":lib",
    |   ":lib",
    ],
)

will result in the following error:

$ bazel build :bin
INFO: Invocation ID: 3d8345a4-2e76-493e-8343-c96a36185b52
ERROR: /home/jenkins/repos/test_repo/BUILD:41:10: Label '//:lib' is duplicated in the 'deps' attribute of rule 'bin'
ERROR: error loading package '': Package '' contains errors
INFO: Elapsed time: 2.514s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (1 packages loaded)

The latest refactor unintentionally made it so you can list the same
constraint multiple times in the `target_compatible_with` attribute.

It was unintentional, but actually greatly simplifies a discussion
point on bazelbuild/bazel-skylib#381. That skylib patch aims to make
it easier to write non-trivial `target_compatible_with` expressions.
For example, to express that something is compatible with everything
except for Windows, one can use the following:

    foo_binary(
        name = "bin",
        target_compatible_with = select({
            "@platforms//os:windows": ["@platforms//:incomaptible"],
            "//conditions:default: [],
        }),
    )

The skylib patch aims to reduce that to:

    foo_binary(
        name = "bin",
        target_compatible_with = compatibility.none_of("@platforms//os:windows"),
    )

This works fine on its own, but a problem arises when these
expressions are composed. For example, a macro author might write the
following:

    def foo_wrapped_binary(name, target_compatible_with = [], **kwargs):
        foo_binary(
            name = name,
            target_compatible_with = target_compatible_with + select({
                "@platforms//os:none": ["@platforms//:incompatible"],
                "//conditions:default": [],
            }),
        )

A macro author should be able to express their own incompatibility
while also honouring user-defined incompatibility.

It turns out that my latest refactor (bazelbuild#14096) unintentionally made
this work. This happened because we now check for incompatibility
before we perform a lot of sanity checks. That's also one of the
reasons that visibility restrictions are not honoured for incomaptible
targets at the moment (bazelbuild#16044).

Without the deduplicating behaviour, macro authors are stuck with
either not allowing composition, or having to create unique
incompatible constraints for every piece in a composed
`target_compatible_with` expression.

This patch adds a test to validate the deduplicating behaviour to
cement it as a feature.

Unfortunately, this means that `target_compatible_with` behaves
differently from other label list attributes. For other label list
attributes, bazel complains when labels are duplicated. For example,
the following BUILD file:

    py_library(
        name = "lib",
    )

    py_binary(
        name = "bin",
        srcs = ["bin.py"],
        deps = [
        |   ":lib",
        |   ":lib",
        ],
    )

will result in the following error:

    $ bazel build :bin
    INFO: Invocation ID: 3d8345a4-2e76-493e-8343-c96a36185b52
    ERROR: /home/jenkins/repos/test_repo/BUILD:41:10: Label '//:lib' is duplicated in the 'deps' attribute of rule 'bin'
    ERROR: error loading package '': Package '' contains errors
    INFO: Elapsed time: 2.514s
    INFO: 0 processes.
    FAILED: Build did NOT complete successfully (1 packages loaded)
@philsc
Copy link
Contributor Author

philsc commented Sep 14, 2022

@gregestren , @katre , at the risk of disappointing you with my poor decision making, what do you think of this patch?

Copy link
Member

@katre katre left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this makes a lot of sense, and there are probably more places in Bazel where we should deduplicate instead of erroring.

@gregestren
Copy link
Contributor

2570fe1 is a recent change recognizing some subtlety with select(). It's not the same purpose as yours, but a reminder that dedupe logic can't be exactly the same everwyhere.

This all reads reasonable to me but my main concern is API consistency. Surely this kind of theme has come up before w.r.t. macros? Let me run this by @comius and/or @brandjon , both of who I'll chat with tomorrow.

@philsc philsc marked this pull request as ready for review September 15, 2022 05:38
@philsc philsc changed the title Deduplicate target_compatible_with labels Deduplicate incompatible target_compatible_with labels Sep 15, 2022
@katre
Copy link
Member

katre commented Sep 15, 2022

Looks good to me, waiting for @gregestren, @comius, and @brandjon to weigh in.

@comius comius added the awaiting-PR-merge PR has been approved by a reviewer and is ready to be merge internally label Sep 15, 2022
Copy link
Contributor

@gregestren gregestren left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds like no concerns.

@sgowroji sgowroji added team-Configurability Issues for Configurability team and removed awaiting-PR-merge PR has been approved by a reviewer and is ready to be merge internally labels Sep 16, 2022
aiuto pushed a commit to aiuto/bazel that referenced this pull request Oct 12, 2022
The latest refactor unintentionally made it so you can list the same
incompatible constraint multiple times in the `target_compatible_with`
attribute.

It was unintentional, but actually greatly simplifies a discussion
point on bazelbuild/bazel-skylib#381. That skylib patch aims to make
it easier to write non-trivial `target_compatible_with` expressions.
For example, to express that something is compatible with everything
except for Windows, one can use the following:

    foo_binary(
        name = "bin",
        target_compatible_with = select({
            "@platforms//os:windows": ["@platforms//:incomaptible"],
            "//conditions:default: [],
        }),
    )

The skylib patch aims to reduce that to:

    foo_binary(
        name = "bin",
        target_compatible_with = compatibility.none_of("@platforms//os:windows"),
    )

This works fine on its own, but a problem arises when these
expressions are composed. For example, a macro author might write the
following:

    def foo_wrapped_binary(name, target_compatible_with = [], **kwargs):
        foo_binary(
            name = name,
            target_compatible_with = target_compatible_with + select({
                "@platforms//os:none": ["@platforms//:incompatible"],
                "//conditions:default": [],
            }),
        )

A macro author should be able to express their own incompatibility
while also honouring user-defined incompatibility.

It turns out that my latest refactor (bazelbuild#14096) unintentionally made
this work. This happened because we now check for incompatibility
before we perform a lot of sanity checks. That's also one of the
reasons that visibility restrictions are not honoured for incomaptible
targets at the moment (bazelbuild#16044).

Without the deduplicating behaviour, macro authors are stuck with
either not allowing composition, or having to create unique
incompatible constraints for every piece in a composed
`target_compatible_with` expression.

This patch adds a test to validate the deduplicating behaviour to
cement it as a feature.

Unfortunately, this means that `target_compatible_with` behaves
differently from other label list attributes. For other label list
attributes, bazel complains when labels are duplicated. For example,
the following BUILD file:

    py_library(
        name = "lib",
    )

    py_binary(
        name = "bin",
        srcs = ["bin.py"],
        deps = [
        |   ":lib",
        |   ":lib",
        ],
    )

will result in the following error:

    $ bazel build :bin
    INFO: Invocation ID: 3d8345a4-2e76-493e-8343-c96a36185b52
    ERROR: /home/jenkins/repos/test_repo/BUILD:41:10: Label '//:lib' is duplicated in the 'deps' attribute of rule 'bin'
    ERROR: error loading package '': Package '' contains errors
    INFO: Elapsed time: 2.514s
    INFO: 0 processes.
    FAILED: Build did NOT complete successfully (1 packages loaded)

Closes bazelbuild#16274.

PiperOrigin-RevId: 474747867
Change-Id: Iacc4de12ce90176d803e0bc9e695d4ec14603449
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
team-Configurability Issues for Configurability team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants