Replace go-bindata-assetfs build dependency with native go:embed

[catsby]

This PR replaces Vault's use of elazarl/go-bindata-assetfs in building the UI with Go's native Embed package.

[kalafut]

Awesome! The bootstrap process can probably be simplified too (https://github.com/hashicorp/vault/blob/master/tools/tools.go#L25-L29)

[catsby]

Awesome! The bootstrap process can probably be simplified too (https://github.com/hashicorp/vault/blob/master/tools/tools.go#L25-L29)

Indeed, I did that in cff9f7a shortly after you mentioned it. Thanks!

[briankassouf]

I think there are a few more references in the Makefile to remove:

vault/Makefile

Lines 11 to 12 in 1b96d2c

	github.com/elazarl/go-bindata-assetfs/... \
	github.com/hashicorp/go-bindata/... \

[briankassouf]

Looks good! I tested it locally, and it works well! We may want to try building with package spec too!

[catsby]

Hey @ncabatoff -

My only question/concern is: why move from pkg to http folders? Probably not an issue, but it feels like an unnecessary change that could conceivably bite us.

Some background: the previous bindata-assetFS approach converted the assets into []bytes slices and wrote them to a file, which is how the ui files became included in the binary. The embed approach includes the files as-is, and so they need to be in a folder that the go file invoking them can reach them, and are referenced from the location //go:embed is referenced. The documentation mentions forbidding the use of . and .. in paths:

Patterns may not contain ‘.’ or ‘..’ or empty path elements

(from https://golang.org/pkg/embed/#hdr-Directives)

I found that I couldn't reference a directory above the directory I was in, so I couldn't do this:

//go:embed ../pkg/web_ui/*
var content embed.FS

As a result, I had to move things into the http/ folder. I used the web_ui subfolder so that the generated files wouldn't need to be tracked in git.

We could alternatively start to track them if we want, since now they'll be included in the binaries with the UI instead of converted by bindata-assetFS. What do you think?

[kalafut]

cc: @chelshaw @Monkeychip for awareness

[briankassouf]

Just one comment, but looks good otherwise!

[heppu]

This actually breaks vendoring the vault as a go module since those ui files are not part of the repo.

$ go get -d github.com/hashicorp/vault@main
go: module github.com/golang/protobuf is deprecated: Use the "google.golang.org/protobuf" module instead.
$ go mod vendor
go mod vendor: pattern web_ui/*: no matching files found

[pmmukh]

@heppu We don’t really support vendoring Vault as a go module, as that can often run into conflicts when importing both Vault, along with the api and sdk modules. We understand you may have your reasons to do this, but we don’t offer much in the way of guarantees that this will work.

[heppu]

Yes I have encountered those issues previously but then again those are issues that user can always resolve with replace rules and some detective work on github. This how ever is something that goes straight against the Go's tooling and hence there is no nice way around. I guess what I'm asking here is that is there something preventing us from adding those web ui files in the repository?

[pmmukh]

@heppu if you'd like to raise a PR fixing this issue, please feel free to do so, and we'll be happy to review it. I'm not sure if there's something preventing us from including the web_ui files, I think that was a decision we'd be ok revisiting if there's a PR making a case for that to be changed.

[heppu]

@pmmukh Thanks for quick responses, I will do that!

[heppu]

@pmmukh I have now created the PR #12523 if you or some one else has time to take a look at it.