Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remaining perf work for wolfictl text #250

Open
jonjohnsonjr opened this issue Jun 6, 2023 · 0 comments
Open

Remaining perf work for wolfictl text #250

jonjohnsonjr opened this issue Jun 6, 2023 · 0 comments
Labels
enhancement New feature or request needs-triage applied to all new customer/user issues. Removed after triage occurs.

Comments

@jonjohnsonjr
Copy link
Contributor

Description

I'm going to stop digging into this, but I wanted to write down what I've found before dropping all context:

Once all the PRs I've sent are merged and those dependencies get bumped, wolfictl text ~400ms instead of ~250s (on my machine, at least).

What's left looks like this:

image

The biggest chunk in the middle is calling BuildFlavor, from here:
https://github.com/chainguard-dev/melange/blob/a3b7a002e874b75c318e5d2a2c3c7af7142f456c/pkg/build/pipeline.go#L97-L98

We end up Stating (I think) an empty directory a lot, so fixing that would shave off ~50ms.

There's also a ton of redundant reading of files and parsing of yamls and detecting of commits. I added some logging to see what's going on:

   7 loadUse.ReadFile("pipelines/cmake/install.yaml")
   8 loadUse.ReadFile("pipelines/cmake/build.yaml")
   8 loadUse.ReadFile("pipelines/cmake/configure.yaml")
  15 loadUse.ReadFile("pipelines/go/install.yaml")
  17 loadUse.ReadFile("pipelines/meson/configure.yaml")
  19 loadUse.ReadFile("pipelines/meson/compile.yaml")
  20 loadUse.ReadFile("pipelines/meson/install.yaml")
  31 loadUse.ReadFile("pipelines/ruby/clean.yaml")
  32 loadUse.ReadFile("pipelines/ruby/build.yaml")
  32 loadUse.ReadFile("pipelines/ruby/install.yaml")
  45 loadUse.ReadFile("pipelines/go/build.yaml")
 113 loadUse.ReadFile("pipelines/patch.yaml")
 166 loadUse.ReadFile("pipelines/autoconf/configure.yaml")
 185 loadUse.ReadFile("pipelines/git-checkout.yaml")
 253 loadUse.ReadFile("pipelines/autoconf/make-install.yaml")
 262 loadUse.ReadFile("pipelines/autoconf/make.yaml")
 495 loadUse.ReadFile("pipelines/strip.yaml")
 526 loadUse.ReadFile("pipelines/fetch.yaml")
 687 detectCommit(".")
4576 Stat("lib/libc.so.6")
4576 Stat("lib64/libc.so.6")

I'm a little hesitant to start refactoring things too much, but it seems like it would be straightforward to cache this stuff.

There are also a bunch of little things:

We spend a ton of time in https://github.com/chainguard-dev/go-apk/blob/2829525a71369b8c570b98332bb73639b1e59802/pkg/apk/version.go#L371 doing regex stuff. If it's possible to do this in a non-regexy way, we could save some time.

All of the string manipulation in here could also be a lot faster: https://gitlab.alpinelinux.org/alpine/go/-/blob/master/repository/repository.go

E.g. IndexUri(), NewRepositoryFromComponents, and Url should all be using path.Join instead of fmt.Sprintf.

The Packages() method should initialize pkgs with Count() capacity.

I suspect we could get this down to sub-100ms if we fix everything.

@jonjohnsonjr jonjohnsonjr added enhancement New feature or request needs-triage applied to all new customer/user issues. Removed after triage occurs. labels Jun 6, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request needs-triage applied to all new customer/user issues. Removed after triage occurs.
Projects
None yet
Development

No branches or pull requests

1 participant