bazel doesn't generate runfiles directory

[xiedeacc]

Description of the bug:

I write a rule to genrate a source file, then this source file used by other source file. maybe a little complicate is that this rule need run another cc_binary, this cc_binary take some proto and a template file as input, then write that final source file as output.

briefly:
cc_test: loader_impl_test depend cc_proto_plugin: pb_code_generator_plugin, and pb_code_generator_plugin denpend cc_binary: pb_code_generator, pb_code_generator take proto and template file as input, then generate file source file

notice loader_impl.h was outs of by pb_code_generator_plugin not pb_code_generator, thoug it's writen by pb_code_generator.

but I find template source file was not contained in any runfiles directory.

bazle-5.3.1 build //... will failed, log:

DEBUG: /root/src/cpp/demo/bazel/pb_code_gen_plugin.bzl:59:10: [<source file src/data/template/loader_impl.h>]
DEBUG: /root/src/cpp/demo/bazel/pb_code_gen_plugin.bzl:97:10: ["bazel-out/k8-opt/bin/external/com_google_protobuf/protoc --proto_path=src/proto --proto_path=. --plugin=protoc-gen-PLUGIN=bazel-out/k8-opt/bin/src/data/pb_code_generator --PLUGIN_opt=src/data/template --PLUGIN_out=bazel-out/k8-opt/bin/src/data src/proto/config.proto"]
DEBUG: /root/src/cpp/demo/bazel/pb_code_gen_plugin.bzl:98:10: [<source file src/data/template/loader_impl.h>, <source file src/proto/config.proto>]
INFO: Analyzed 14 targets (89 packages loaded, 8513 targets configured).
INFO: Found 14 targets...
ERROR: /root/src/cpp/demo/src/data/BUILD:20:16: Action src/data/loader_impl.h failed: (Exit 1): bash failed: error executing command /bin/bash -c ... (remaining 1 argument skipped)

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20240108 00:50:35.120276     3 pb_code_generator.cc:80] runfiles_dir: /root/.cache/bazel/_bazel_root/f3e78c6699384341665c660140b5c5b3/execroot/demo/bazel-out/k8-opt/bin/src/data/pb_code_generator.runfiles
E20240108 00:50:35.120370     3 pb_code_generator.cc:84] fail to list directory: /root/.cache/bazel/_bazel_root/f3e78c6699384341665c660140b5c5b3/execroot/demo/bazel-out/k8-opt/bin/src/data/pb_code_generator.runfiles
--PLUGIN_out: protoc-gen-PLUGIN: Plugin failed with status code 255.
INFO: Elapsed time: 30.238s, Critical Path: 18.59s
INFO: 890 processes: 127 internal, 762 linux-sandbox, 1 local.
FAILED: Build did NOT complete successfully

below is BUILD file and rule's bzl file.
cat src/data/BUILD

load("//tools:cpplint.bzl", "cpplint")
load("//bazel:pb_code_gen_plugin.bzl", "cc_proto_plugin")

package(default_visibility = ["//visibility:public"])

cc_binary(
    name = "pb_code_generator",
    srcs = [
        "pb_code_generator.cc",
        "pb_code_generator.h",
    ],
    linkstatic = True,
    deps = [
        "//src/util",
        "@com_google_absl//absl/strings",
        "@com_google_protobuf//:protoc_lib",
    ],
)

cc_proto_plugin(
    name = "pb_code_generator_plugin",
    src = "//src/proto:config.proto",
    outs = [
        "loader_impl.h",
    ],
    data = [
        "//src/data/template:loader_impl.h",
    ],
    linkstatic = True,
    plugin = ":pb_code_generator",
    proto_deps = ["//src/proto:config.proto"],
    proto_path = "src/proto",
    protoc = "@com_google_protobuf//:protoc",
    deps = [
        "//src/util",
    ],
)

cc_test(
    name = "loader_impl_test",
    srcs = ["loader_impl_test.cc"],
    deps = [
        ":pb_code_generator_plugin",
        "//src/test_util:test_main",
        "@com_github_glog_glog//:glog",
    ],
)

cpplint()

cat bazel/pb_code_gen_plugin.bzl

def _get_external_root(ctx):
    gendir = ctx.var["GENDIR"] + "/"
    external_roots = []
    path = ctx.attr.src.files.to_list()[0].path
    if path.startswith(gendir):
        path = path[len(gendir):]
    path = path.split("/")
    if path[0] == "external":
        external_roots += ["/".join(path[0:2])]
    roots = depset(external_roots)
    n = len(roots.to_list())
    if n > 1:
        fail("""
           You are attempting simultaneous compilation of protobuf source files that span multiple workspaces (%s).
           Decompose your library rules into smaller units having filesets that belong to only a single workspace at a time.
           Note that it is OK to *import* across multiple workspaces, but not compile them as file inputs to protoc.
           """ % roots)
    elif n == 1:
        return external_roots[0]
    else:
        return "."

def _check_if_protos_are_generated(ctx):
    generated_path = ctx.var["GENDIR"]
    for plfile in ctx.attr.src.files.to_list():
        if not plfile.path.startswith(generated_path):
            return False
        if not plfile.is_source:
            return False
    return True

def _get_offset_path(root, path):
    """Adjust path relative to offset"""

    if path.startswith("/"):
        fail("path argument must not be absolute: %s" % path)

    if not root:
        return path

    if root == ".":
        return path

    # "external/foobar/file.proto" --> "file.proto"  if path.startswith(root):
    if path.startswith(root):
        start = len(root)
        if not root.endswith("/"):
            start += 1
            return path[start:]

    depth = root.count("/") + 1
    return "../" * depth + path

def _proto_generate_impl(ctx):
    execdir = _get_external_root(ctx)
    if _check_if_protos_are_generated(ctx):
        external = "" if execdir == "." else "/" + execdir
        execdir = ctx.var["GENDIR"] + external
    print(ctx.attr.data[0].files.to_list())
    ctx.runfiles(ctx.attr.data[0].files.to_list())
    protoc = _get_offset_path(execdir, ctx.executable.protoc.path)
    plugin = _get_offset_path(execdir, ctx.executable.plugin.path)
    dir_out = _get_offset_path(execdir, ctx.executable.plugin.dirname)
    plugin_directory = ctx.attr.data[0].files.to_list()[0].dirname

    out_files = [ctx.actions.declare_file(out) for out in ctx.attr.outs]
    path = _get_offset_path(execdir, ctx.files.src[0].path)

    #print(path)

    all_files = ctx.attr.src.files.to_list()
    protoc_cmd = [protoc]
    protoc_cmd += ["--proto_path=" + ctx.attr.proto_path]
    protoc_cmd += ["--proto_path=" + "."]
    all_inputs = []
    all_inputs += ctx.attr.data[0].files.to_list()

    for dep in ctx.attr.proto_deps:
        for pfile in dep.files.to_list():
            all_inputs = all_inputs + [pfile]
            ppath = _get_offset_path(execdir, pfile.path)
            rpath = "/".join(ppath.split("/")[4:])
            if rpath == "":
                continue
            protoc_cmd += ["-I" + rpath + "=" + ppath]

    protoc_cmd += ["--plugin=protoc-gen-PLUGIN=" + plugin]
    protoc_cmd += ["--PLUGIN_opt=" + plugin_directory]
    protoc_cmd += ["--PLUGIN_out=" + dir_out]
    protoc_cmd += [path]

    cmds = []
    if execdir != ".":
        cmds += ["cd %s" % execdir]
    cmds += [" ".join(protoc_cmd)]

    print(cmds)
    print(all_inputs)
    ctx.actions.run_shell(
        inputs = all_inputs + [ctx.executable.plugin] + [ctx.executable.protoc],
        outputs = out_files,
        command = " && ".join(cmds),
    )
    return struct(files = depset(out_files))

_proto_generate = rule(
    attrs = {
        "src": attr.label(
            mandatory = True,
            allow_single_file = True,
        ),
        "proto_deps": attr.label_list(
            allow_files = True,
        ),
        "proto_path": attr.string(
            mandatory = True,
        ),
        "protoc": attr.label(
            default = Label("@com_google_protobuf//:protoc"),
            executable = True,
            cfg = "target",
        ),
        "plugin": attr.label(
            mandatory = True,
            executable = True,
            providers = ["files_to_run"],
            cfg = "target",
            allow_files = True,
        ),
        "outs": attr.string_list(),
        "data": attr.label_list(allow_files = True),
    },
    output_to_genfiles = True,
    implementation = _proto_generate_impl,
)

def proto_generate(name, src, proto_deps, proto_path, protoc, plugin, data, outs = []):
    args = {}
    args.update({
        "name": name,
        "src": src,
        "proto_deps": proto_deps,
        "proto_path": proto_path,
        "protoc": protoc,
        "plugin": plugin,
        "data": data,
        "outs": outs,
    })
    return _proto_generate(**args)

def cc_proto_plugin(
        name,
        src,
        proto_deps,
        proto_path,
        protoc,
        plugin,
        data = [],
        outs = [],
        deps = [],
        **kwargs):
    proto_name = name + "_proto"
    proto_generate(proto_name, src, proto_deps, proto_path, protoc, plugin, data, outs)
    native.cc_library(
        name = name,
        srcs = [proto_name],
        deps = depset(deps + ["@com_google_protobuf//:protobuf"]).to_list(),
        **kwargs
    )

some struggles

if add

    data = [
        "//src/data/template:loader_impl.h",
    ],

to target pb_code_generator, some thing changes
run bazel-5.3.1 build //src/data:loader_impl_test at first, will fail for same reason, but if run bazel-5.3.1 build //src/data:loader_impl_test after bazel-5.3.1 build //... will success.

my goal is run bazel-5.3.1 build //src/data:loader_impl_test at first can also build success, how to fix it?

here is a small demo, https://github.com/xiedeacc/demo.git
on master branch

• run bazel-5.3.1 build //src/data:loader_impl_test at first, build will fail
• run bazel-5.3.1 build //... at first, build success
  on error branch
• run bazel-5.3.1 build //src/data:loader_impl_test at first, build will fail
• run bazel-5.3.1 build //... at first, build will fail

Which category does this issue belong to?

Local Execution

What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

https://github.com/xiedeacc/demo.git

Which operating system are you running Bazel on?

ubuntu 22.04

What is the output of bazel info release?

5.3.1

If bazel info release returns development version or (@non-git), tell us how you built Bazel.

release 5.3.1- (@non-git)

What's the output of git remote get-url origin; git rev-parse master; git rev-parse HEAD ?

No response

Is this a regression? If yes, please try to identify the Bazel commit where the bug was introduced.

No response

Have you found anything relevant by searching the web?

cannot find

Any other information, logs, or outputs that you want to share?

No response

[pcjanzen]

You need to add the tools that require their own runfiles directories to the tools of the ctx.actions.run_shell call.

Additionally, you shouldn't be dereferencing a symbolic link pointed to by argv[0] to find the runfiles directory; that allows you to escape the sandbox and won't work if --nobuild_runfile_links is in effect. The c++ runfiles library can handle this for you.

[xiedeacc]

You need to add the tools that require their own runfiles directories to the tools of the ctx.actions.run_shell call.

Additionally, you shouldn't be dereferencing a symbolic link pointed to by argv[0] to find the runfiles directory; that allows you to escape the sandbox and won't work if --nobuild_runfile_links is in effect. The c++ runfiles library can handle this for you.

thanks very much, I'll try later