Design review for "Integrated bzl_library"

[brandjon]

This is a proposal to create a native rule similar to Skylib's bzl_library. The main differences are:

• It would do proper strict-deps enforcement: If you load foo.bzl, then foo.bzl must appear in srcs of the current target or one of its direct deps.

• Native rules would be able to access the underlying .bzl module object represented by the target. This would be useful to the new starlark_doc_extract rule, which implements the new version of Stardoc that dumps docstrings directly from Bazel rather than inferring them based on mock evaluation of .bzl files.

• Only source .bzl files would be allowed, not generated .bzl files (which can't be loaded anyway).

The migration path from bzl_library is the part that's most uncertain at this point. (See the Compatibility section.)

[sluongng]

This might be a tangent to the proposal, but I think this is a big, one-way turn solution to a relatively small(?) problem.

---

First, let's talk about why I think this is a small problem:

If I read this correctly, this is mainly to improve how we are currently generating documentation for starlark rules.
If that's the case, I would go out to say that most starlark rules today, in my experience, are not that useful.
This could be because inside an enterprise setting, checked-in rules are often lagged behind vs opensource release.
So sites that display doc for starlark rules often have to keep several version of the same doc but often would only index the latest version in their search engine.

This resulted in me, as well as some folks I know, preferring to read the implementation instead of reading the generated doc for more accurate references.
Improving starlark doc in my view, yield very little impact vs a solution that helps me navigate Starlark source code better.
For example, the recent Buck2's starlark LSP provides really "good find references" and "go to definition" for Buck2's native rules.
The optional typing annotation support on Buck2's starlark also helps provide autocompletion while authoring/maintaining starlark rules for Buck2.

It would be interesting if we could ask some questions related to this topic (starlark rules authoring pain points) in the next user's survey so we could see how relevant documentation vs source navigation.

---

Second, I think the current proposal is a big one-way street.

While we are trying to migrate away from native rules over to starlark rules, this would add one more native rule into the mix.
Double down on bzl_library alternative approach seems to be a much more practical solution that is easier to implement.

Keep in mind: as a user, bazel is a black box binary, but bzl_library is a starlark file that could be opened and read through.
So troubleshooting a starlark rule would be much easier than a native rule for downstream users.

[brandjon]

Thanks for the insightful reply.

Yes, a big part of the motivation comes from Stardoc. Your point about source navigation is an argument for prioritizing other features over Stardoc. But I would argue that documentation generation is still necessary, even if it doesn't free you from reading the source, and we still want to migrate away from legacy Stardoc's mocking-based approach in order to reduce our tech debt in Bazel. Given that Stardoc's what we're working on right now, this is the time to consider what to do about bzl_library.

So the next question is whether we need bzl_library for Stardoc, and whether we need/want it for other reasons.

For Stardoc, it's used to support the use case where doc pages are written as BUILD targets and generated by a bazel build invocation. If we only support a use case where docs are generated by a separate non-build invocation (bazel dump-docs //pkg:foo.bzl?), then we don't need to do deps tracking on .bzls in the target graph, so Stardoc and bzl_library are unrelated concerns. We chose to support the target use case because that's (generally) backwards compatible with how people use legacy Stardoc, and wanted to minimize migration-related pain for this rewrite. (We may still need the "dump-docs" form anyway, e.g. to inspect documentation across the whole workspace regardless of visibility.)

It's also possible to support the Stardoc target use case without doing anything to bzl_library -- the "Double down on bzl_library" alternative you cited. The cost is that it keeps Stardoc's rule somewhat weakly typed (it accepts any rule type, e.g. a filegroup, to avoid having Bazel depend on Skylib for StarlarkInfoProvider).

The non-Stardoc reasons for wanting a native bzl_library are mainly strict-deps checking that's guaranteed to have the same semantics as Bazel. We'd still want external tooling that does its own load parsing and deps analysis to automatically write and update starlark_library targets though.

Having a native rule also means people aren't loading Skylib in almost every repo just for the sake of this one rule. If we wanted to commit to a Starlark implementation, we could put it in @bazel_tools. If we did that then we could also solve the weak typing issue with Stardoc's rule, since it could refer to the provider directly. The Stardoc rule would still have to do its own transitive deps validation (flattening depsets), since it's relying on that information at analysis time whereas a non-native bzl_library could at best do its own validation at execution time.

Regarding implementation cost, I suspect compatibility with and migration from bzl_library is far more significant than the actual coding and maintenance of the new rule in Bazel. Note that adding deps checking to bzl_library is still an incompatible change. The main thing I worry/wonder about is the effect of disallowing generated .bzls. I think gathering more information on this point blocks any decision to move forward.

In summary, my thinking is that the Stardoc side of the puzzle is a relatively fixed constraint (supporting the build rule, possibly alongside a separate command), and the choice is whether we want a native rule to swipe at all the other problems I cited above at once, or whether we'd rather live with a Starlark rule and accept the drawbacks.

[brandjon]

Update: This design has been rejected. See the new Decision section for an explanation of the two possible future designs -- one is a new incarnation of this proposal, and the other is a completely different approach that would involve more magic but more benefits. Unfortunately, we can't really prioritize either one right now, so Stardoc will continue to use its current hack of doing its own direct dependency validation.

Thanks to everyone for the very useful comments!