Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: detect invalid binaries and show warning #1124

Merged
merged 4 commits into from Jul 4, 2022
Merged

Conversation

danez
Copy link
Contributor

@danez danez commented Jun 24, 2022

Summary

This now detects binaries not built for Linux/amd64 and prints a warning in this case.

Fixes netlify/pillar-runtime/issues#231

baboon copy

@danez danez added the type: feature code contributing to the implementation of a feature and/or user facing functionality label Jun 24, 2022
@danez danez requested a review from a team June 24, 2022 08:26
@github-actions
Copy link
Contributor

github-actions bot commented Jun 24, 2022

⏱ Benchmark results

Comparing with 02f248f

largeDepsEsbuild: 6.5s

⬆️ 1.55% increase vs. 02f248f

^   8.3s                                                                    8.4s                          
│   ┌──┐                                                                    ┌──┐                          
│   |  |                                                                    |  |                          
│   |  |                                     7s                             |  |                          
│   |  |    6.5s                            ┌──┐                    6.9s    |  |                          
│ ──┼──┼────┌──┐────────────────────────────┼──┼────────────────────┌──┐────┼──┼────────────6.5s────6.5s──
│   |  |    |  |            5.7s     6s     |  |    5.7s    5.9s    |  |    |  |            ┌──┐    ┌──┐  
│   |  |    |  |            ┌──┐    ┌──┐    |  |    ┌──┐    ┌──┐    |  |    |  |    5.6s    |  |    |▒▒|  
│   |  |    |  |            |  |    |  |    |  |    |  |    |  |    |  |    |  |    ┌──┐    |  |    |▒▒|  
│   |  |    |  |            |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |            |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |            |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |            |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |            |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |            |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |            |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |            |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |            |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |            |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |            |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |            |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    n/a     |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
└───┴──┴────┴──┴────────────┴──┴────┴──┴────┴──┴────┴──┴────┴──┴────┴──┴────┴──┴────┴──┴────┴──┴────┴──┴──>
    T-12    T-11    T-10    T-9     T-8     T-7     T-6     T-5     T-4     T-3     T-2     T-1      T    
Legend

largeDepsNft: 29.9s

⬇️ 4.19% decrease vs. 02f248f

^  42.9s                                                                   43.2s                          
│   ┌──┐           40.6s                                                    ┌──┐                          
│   |  |            ┌──┐                                                    |  |                          
│   |  |            |  |                                                    |  |                          
│   |  |            |  |                                                    |  |                          
│   |  |            |  |                   33.4s                   32.7s    |  |                          
│ ──┼──┼───31.3s────┼──┼───────────29.9s────┌──┐────30s─────────────┌──┐────┼──┼───────────29.6s───29.9s──
│   |  |    ┌──┐    |  |   27.1s    ┌──┐    |  |    ┌──┐   27.5s    |  |    |  |    27s     ┌──┐    ┌──┐  
│   |  |    |  |    |  |    ┌──┐    |  |    |  |    |  |    ┌──┐    |  |    |  |    ┌──┐    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
└───┴──┴────┴──┴────┴──┴────┴──┴────┴──┴────┴──┴────┴──┴────┴──┴────┴──┴────┴──┴────┴──┴────┴──┴────┴──┴──>
    T-12    T-11    T-10    T-9     T-8     T-7     T-6     T-5     T-4     T-3     T-2     T-1      T    
Legend

largeDepsZisi: 44.8s

⬇️ 3.90% decrease vs. 02f248f

^  58.1s           57.8s                                                                                  
│   ┌──┐            ┌──┐                                                                                  
│   |  |            |  |                                                                                  
│   |  |            |  |                   50.6s                   50.3s                                  
│   |  |   46.4s    |  |                    ┌──┐                    ┌──┐                    46s           
│ ──┼──┼────┌──┐────┼──┼────────────────────┼──┼───43.7s───43.3s────┼──┼────────────────────┌──┐───44.8s──
│   |  |    |  |    |  |   41.9s            |  |    ┌──┐    ┌──┐    |  |           41.7s    |  |    ┌──┐  
│   |  |    |  |    |  |    ┌──┐            |  |    |  |    |  |    |  |            ┌──┐    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |            |  |    |  |    |  |    |  |            |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |            |  |    |  |    |  |    |  |            |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |            |  |    |  |    |  |    |  |            |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |            |  |    |  |    |  |    |  |            |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |            |  |    |  |    |  |    |  |            |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |            |  |    |  |    |  |    |  |            |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |            |  |    |  |    |  |    |  |            |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |            |  |    |  |    |  |    |  |            |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |            |  |    |  |    |  |    |  |            |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |            |  |    |  |    |  |    |  |            |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |            |  |    |  |    |  |    |  |            |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |            |  |    |  |    |  |    |  |            |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |            |  |    |  |    |  |    |  |            |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    n/a     |  |    |  |    |  |    |  |    n/a     |  |    |  |    |▒▒|  
└───┴──┴────┴──┴────┴──┴────┴──┴────────────┴──┴────┴──┴────┴──┴────┴──┴────────────┴──┴────┴──┴────┴──┴──>
    T-12    T-11    T-10    T-9     T-8     T-7     T-6     T-5     T-4     T-3     T-2     T-1      T    
Legend

const binaryInfo = detect(buffer as Buffer)

if (!isValidFunctionBinary(binaryInfo)) {
console.warn(`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

zip-it-and-ship-it can be invoked as a binary, which prints a JSON string to stdout. For this reason, we don't typically print any additional messages because that'll technically make the output incorrect. To get around this, we've been adding properties to the output and then reading them from Netlify Build and/or CLI and show warning/error messages accordingly.

This is something we should revisit, because I don't think we're actually using the binary version anywhere, but I still wanted to mention it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, can you point me to something? I cannot find any other case where we supply warnings/errors in the json response.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

@danez danez Jun 30, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem here is that I cannot do it this way because the function is never actually returned by zisi. It is simply ignored. Now I could change that and mark it as invalid/incompatible and handle the ignoring outside of zisi, but that is a pretty hefty breaking change for just a simple warning.
I would rather try to remove the bin of zisi altogether and keep the printing as is :)

Or not do the warning at all.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I see. Agreed, let's not overcomplicate things! I think it's fine to ship this as is, and then look into removing the binary separately.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I changed the logic slightly to simply not print the warning in cli mode.

// CLI entry point
const runCli = async function () {
// @ts-expect-error TODO: `destFolder` and `srcFolder` are not being passed
// back from `parseArgs()`.
const { destFolder, srcFolder, ...options } = parseArgs()

try {
global.ZISI_CLI = true
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does this need to be a global?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not necessarily, but the alternative would be to pass the flag through the complete function hierarchy. And as the binary of zisi seems it might be deprecated and removed at some point, this was by far the more straightforward solution.

@danez danez added the automerge label Jul 4, 2022
@kodiakhq kodiakhq bot merged commit 77c835e into main Jul 4, 2022
@kodiakhq kodiakhq bot deleted the binary-warning branch July 4, 2022 14:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
automerge type: feature code contributing to the implementation of a feature and/or user facing functionality
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants