bazel rules & IDEs

[abrams27]

hi, with my team we are working on bazel bsp server and one of our goals is to support multiple languages / bazel rules. Unfortunately it's a bit tricky for 'custom' (not builtin) rules like rust rules and the fact that the old providers are deprecated.

It means that in order to obtain all the data we need to add a load statement for a provider. (example for rust). It means that this code wont work in workspaces without rust rules. Probably we will encounter similar problems with other languages in the future : (

We can try to solve it somehow on our side by creating aspects on the fly based on the installed rules or defined languages by user, but we were wondering is there a better way?

possible options:

• 'ide' providers - but we'd have to load them as well per rule since we need language-specific info (the rest, like srcs, we can obtain directly from the target)
• another way (conditional?) of loading providers
• another idea?

[meteorcloudy]

Can you elaborate a bit on when and how you run those aspects? Do you need to run all of them no matter which languages the project is configured for? I guess it's running within the same workspace as the project?

/cc @comius @lberki @Wyverald

[abrams27]

ohh yea sorry, #18292 (comment) explains the issue

[lberki]

How do you expect people to depend on aspects.bzl in the example you linked?

That file does depend on Rust things so it's natural to require that the Rust rules are in the workspace. If you use bzlmod, that will naturally happen if you add the Rust rules to your MODULE.bazel file. If you want to support running without bzlmod, too, the usual answer is to either document that you require the Rust rules also or to provide a little deps.bzl file that one can load from their WORKSPACE file, which then creates the repositories where your aspects.bzl lives and the rules_rust one.

[comius]

I'm familiar with the problem. It has 2 main ingredients:

• opening any Bazel repository in an IDE
• an IDE aspect "injected" into that repository

The desire is that IDE aspect can support all possible rulesets (for example multiple flavours of Rust rules).

The problem is, that IDE aspect then needs to depend on all of the rulesets it supports, because it needs to load provider key.

If the repository is using particular ruleset, that ok, because there's already a repo dependency.

If the repository is not using particular ruleset, IDE aspect still needs to pull it in. The more rulesets you support, the more of unneeded things get pulled in.

[lberki]

How is the IDE aspect injected in this case?

[comius]

I think something similar to --override_repository=ide_aspect=/path/to/aspect/repo. Don't know all the details.

[abrams27]

#18292 (comment) sums the issue quite well

overview of the flow:

1. when the server is installed we copy aspects into the project directory (under .bazelbsp)
2. then we call bazel with --override_repository=bazelbsp_aspect=<path to .bazelbsp>
3. we collect things from the workspace
4. we map obtained info to bsp and then send it to the client (intellij in this case)

[Wyverald]

Just to be clear, the command you run for "3" is probably something like bazel build --aspects=@bazelbsp_aspect//:something //..., right?

[abrams27]

yup bazel build --bes_backend=grpc://localhost:63398 --override_repository=bazelbsp_aspect=/Users/Marcin.Abramowicz/dev/bazel-bsp/.bazelbsp --aspects=@bazelbsp_aspect//:aspects.bzl%bsp_target_info_aspect --output_groups=bsp-target-info-transitive-deps,bsp-ide-resolve-transitive-deps --keep_going --color=yes --build_manual_tests -- //... in my case for the bazel-bsp project

[meteorcloudy]

I can see this might be solvable with Bzlmod by using --override_module=bazelbsp_aspect=<path to .bazelbsp>, then bazelbsp_aspect can specify the minimal versions of rules sets it needs in its MODULE.bazel file. The only blocker here is that using --override_module currently doesn't add a direct dependency from root module to bazelbsp_aspect, therefore you'll still need a bazel_dep in the MODULE.bazel file of the root module, but we can change that from Bazel side.

But there are still going to be some side-effects of this approach, because the transitive dependencies you introduced via the rulesets may change the resolved external dependencies of the root module.

We can try to solve it somehow on our side by creating aspects on the fly based on the installed rules or defined languages by user

I actually think this is the best approach because this runs only what needs to be run and doesn't require much Bazel magic.

[abrams27]

i see, then we will try to implement the workaround on our side;

one more question - is there a nice way to know what rules / languages are included in the workspace? it'd be useful in the mechanizm, to determine what providers can be loaded

[meteorcloudy]

With WORKSPACE, I've heard people using the native.existing_rules API to dump the repo rules declared. With Bzlmod, we are planning to implement bazel mod query to help examining the external deps graph, but that's still WIP, aim to ship in 6.3. /cc @Wyverald

[abrams27]

awesome, thanks!

[sluongng]

Over in buck2, there is an ability to create a starlark script(BXL) and execute it using buck2 bxl against Bazel's build graph.
https://buck2.build/docs/developers/bxl_getting_started/

Given how convoluted the current approach of "sideloading" an aspect into a repo, perhaps Bazel should consider adopting something similar.

bazel build \
  --override_module=bazelbsp_aspect=<path to .bazelbsp> \
  --aspects=@bazelbsp_aspect//:aspects.bzl%bsp_target_info_aspect \
  --output_groups=bsp-target-info-transitive-deps,bsp-ide-resolve-transitive-deps \
  --keep_going \
  -- //...

could be replaced with

bazel bxl //myscript.bxl:your_function_name -- --arg1=foo --arg2=bar //...