Add AWS CLI to `system install` tools #993

VariableExp0rt · 2023-10-24T14:09:33Z

Description

This PR attempts to add support for downloading and installing the AWS CLI for supported platforms. This turned out to be a bit more difficult with the special requirements for each platform supported (Linux, MacOS, Windows).

Motivation and Context

Fixes a currently open issue - #941

How Has This Been Tested?

Changes tested with:

gofmt -w -s ./pkg
gofmt -w -s ./cmd

go build

make test
make e2e

If updating or adding a new CLI to arkade get, run:

go build && ./hack/test-tool.sh TOOL_NAME

Types of changes

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to change)

Documentation

I have updated the list of tools in README.md if (required) with ./arkade get --format markdown
I have updated the list of apps in README.md if (required) with ./arkade install --help

Checklist:

My code follows the code style of this project.
My change requires a change to the documentation.
I have updated the documentation accordingly.
I've read the CONTRIBUTION guide
I have signed-off my commits with git commit -s

I have tested this on arm, or have added code to prevent deployment

(I've added a boolean flag --run-installer which defaults to false)

alexellis · 2023-10-24T16:53:30Z

@Jasstkn would you be able to take a look at the approach?

Jasstkn · 2023-10-25T14:09:28Z

@Jasstkn would you be able to take a look at the approach?

sure, will take a look. In the meantime, @VariableExp0rt can I ask you to add a confirmation how you tested the installation via arkade?

VariableExp0rt · 2023-10-25T15:55:48Z

@Jasstkn I tested via arkade system install aws-cli (I haven't tested on the other platforms, as I'm primarily operating on a Mac and don't have access to a Windows box). If the approach is a bit wrong, I can re-do it, the installer stuff AWS provide is a bit messy

VariableExp0rt · 2023-11-01T13:50:09Z

Hey @Jasstkn, let me know if there is anything else you need from me :)

Shikachuu

Please see my comments above and do a rebase!

Overall really nice work, I really like the tool unit test for this too, it is a cool idea!

cmd/system/aws_cli.go

derek · 2024-01-03T14:47:34Z

Thank you for your contribution. It seems that one or more of your commits have a "Signed-off-by" statement with an anonymous email address. The Developer Certificate of Origin (DCO) requires all commits to be signed off by genuine, contactable individuals. Please see our contributing guide.

### :bulb: Shall we fix this?

This will only take a few moments.

First, clone your fork and checkout this branch using the git CLI.

Next, set up your real name and email address:

git config --global user.name "Your Full Name"
git config --global user.email "you@domain.com"

Finally, run one of these commands to add the "Signed-off-by" line to your commits.

If you only have one commit so far then run: git commit --amend --signoff and then git push --force.
If you have multiple commits, watch this video.

Check that the message has been added properly by running "git log".

Shikachuu

Looks promising.

Please don't forget to update the README.md and squash the commits in to single one, we like to keep the git history clean for single tools like this.

Thanks!

cmd/system/aws_cli.go

VariableExp0rt · 2024-01-03T18:52:15Z

Looks promising.

Please don't forget to update the README.md and squash the commits in to single one, we like to keep the git history clean for single tools like this.

Thanks!

Have updated the README.md here 👍 .

VariableExp0rt · 2024-01-03T20:17:07Z

@Shikachuu I'm having quite a bit of trouble squashing into a single commit right now, is there a correct way I should be doing that other than git rebase -i HEAD~7 and squash-ing my commits? I think the previous rebase is making things awkward

Signed-off-by: liam.baker <liam.baker@sage.com>

VariableExp0rt · 2024-01-04T10:36:44Z

@Shikachuu I think I've managed to fix it, sorry about that! Let me know if there are any more comments to address 👍

VariableExp0rt · 2024-01-05T16:53:30Z

For posterity:

./hack/test-tool.sh awscli
+ ./arkade get awscli --arch arm64 --os darwin --quiet
+ file /Users/liam.baker/.arkade/bin/awscli
/Users/liam.baker/.arkade/bin/awscli: xar archive compressed TOC: 4792, SHA-1 checksum
+ rm /Users/liam.baker/.arkade/bin/awscli
+ echo

+ ./arkade get awscli --arch x86_64 --os darwin --quiet
+ file /Users/liam.baker/.arkade/bin/awscli
/Users/liam.baker/.arkade/bin/awscli: xar archive compressed TOC: 4792, SHA-1 checksum
+ rm /Users/liam.baker/.arkade/bin/awscli
+ echo

+ ./arkade get awscli --arch x86_64 --os linux --quiet
+ file /Users/liam.baker/.arkade/bin/awscli
/Users/liam.baker/.arkade/bin/awscli: Zip archive data, at least v2.0 to extract, compression method=store
+ rm /Users/liam.baker/.arkade/bin/awscli
+ echo

+ ./arkade get awscli --arch aarch64 --os linux --quiet
+ file /Users/liam.baker/.arkade/bin/awscli
/Users/liam.baker/.arkade/bin/awscli: Zip archive data, at least v2.0 to extract, compression method=store
+ rm /Users/liam.baker/.arkade/bin/awscli
+ echo

+ ./arkade get awscli --arch x86_64 --os mingw --quiet
+ file /Users/liam.baker/.arkade/bin/awscli.exe
/Users/liam.baker/.arkade/bin/awscli.exe: Composite Document File V2 Document, Little Endian, Os: Windows, Version 6.2, MSI Installer, Code page: 1252, Title: Installation Database, Subject: AWS Command Line Interface v2, Author: Amazon Web Services, Keywords: Installer, Comments: This installer database contains the logic and data required to install AWS Command Line Interface v2., Template: x64;1033, Revision Number: {568A8D9E-6206-47B9-8AF8-EA26143A52B8}, Create Time/Date: Fri Oct 20 19:03:24 2023, Last Saved Time/Date: Fri Oct 20 19:03:24 2023, Number of Pages: 200, Number of Words: 2, Name of Creating Application: Windows Installer XML Toolset (3.11.1.2318), Security: 2
+ rm /Users/liam.baker/.arkade/bin/awscli.exe
+ echo

VariableExp0rt · 2024-02-15T09:09:53Z

@Shikachuu is there anything else I should do here?

alexellis · 2024-02-16T13:14:33Z

cmd/system/aws_cli.go

+}
+
+func getAWSCLIVersion(owner, repo string) (string, error) {
+	url := fmt.Sprintf("https://api.github.com/repos/%s/%s/git/refs/tags", owner, repo)


We cannot use the API because it's rate-limited.

I'm curious, why did you add this when we already have code in the project (everywhere) that already determines the version?

The reason for this is that tags exist that are for all releases, but there are not releases to accompany them. I might be wrong, but I couldn't see specifically anywhere that only looks at the refs/tags (which is all that AWS provide). More info here #993 (comment)

alexellis · 2024-02-16T13:14:48Z

cmd/system/aws_cli.go

+	return latest.String(), nil
+}
+
+func runBundledInstaller(osVer string, workDir string, filename string, installPath string) error {


I don't understand why this option is being included?

What format is the AWS CLI in?

The CLI comes in different formats for different platforms, for instance OSx is a .pkg, Linux is a .zip, and similar for Windows. There isn't just a binary.

alexellis · 2024-02-16T13:15:14Z

cmd/system/aws_cli.go

+}
+
+func installationInstructions(osVer string, workDir string, filename string, installPath string) ([]byte, error) {
+	t := template.New("Installation Instructions")


Is there no binary package for the aws cli?

There is not, just zip, pkg, or the msi installer for Windows.

derek bot added the new-contributor label Oct 24, 2023

Shikachuu suggested changes Jan 2, 2024

View reviewed changes

derek bot added the no-dco label Jan 3, 2024

VariableExp0rt force-pushed the fork/add-aws-cli branch from 4b85a9f to 26a1b6e Compare January 3, 2024 14:48

derek bot removed the no-dco label Jan 3, 2024

VariableExp0rt requested a review from Shikachuu January 3, 2024 15:02

Shikachuu suggested changes Jan 3, 2024

View reviewed changes

cmd/system/aws_cli.go Show resolved Hide resolved

cmd/system/aws_cli.go Outdated Show resolved Hide resolved

VariableExp0rt requested a review from Shikachuu January 3, 2024 20:17

VariableExp0rt force-pushed the fork/add-aws-cli branch from 402274a to c1862d9 Compare January 4, 2024 09:48

Adds AWS CLI to system apps

35fd063

Signed-off-by: liam.baker <liam.baker@sage.com>

VariableExp0rt force-pushed the fork/add-aws-cli branch from 3670984 to 35fd063 Compare January 4, 2024 10:35

Shikachuu approved these changes Jan 4, 2024

View reviewed changes

alexellis reviewed Feb 16, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add AWS CLI to `system install` tools #993

Add AWS CLI to `system install` tools #993

VariableExp0rt commented Oct 24, 2023

alexellis commented Oct 24, 2023

Jasstkn commented Oct 25, 2023

VariableExp0rt commented Oct 25, 2023 •

edited

VariableExp0rt commented Nov 1, 2023

Shikachuu left a comment

derek bot commented Jan 3, 2024

Shikachuu left a comment

VariableExp0rt commented Jan 3, 2024

VariableExp0rt commented Jan 3, 2024 •

edited

VariableExp0rt commented Jan 4, 2024

VariableExp0rt commented Jan 5, 2024

VariableExp0rt commented Feb 15, 2024

alexellis Feb 16, 2024

VariableExp0rt Feb 21, 2024 •

edited

alexellis Feb 16, 2024

VariableExp0rt Feb 21, 2024

alexellis Feb 16, 2024

VariableExp0rt Feb 21, 2024

Add AWS CLI to system install tools #993

Are you sure you want to change the base?

Add AWS CLI to system install tools #993

Conversation

VariableExp0rt commented Oct 24, 2023

Description

Motivation and Context

How Has This Been Tested?

Types of changes

Documentation

Checklist:

alexellis commented Oct 24, 2023

Jasstkn commented Oct 25, 2023

VariableExp0rt commented Oct 25, 2023 • edited

VariableExp0rt commented Nov 1, 2023

Shikachuu left a comment

Choose a reason for hiding this comment

derek bot commented Jan 3, 2024

Shikachuu left a comment

Choose a reason for hiding this comment

VariableExp0rt commented Jan 3, 2024

VariableExp0rt commented Jan 3, 2024 • edited

VariableExp0rt commented Jan 4, 2024

VariableExp0rt commented Jan 5, 2024

VariableExp0rt commented Feb 15, 2024

alexellis Feb 16, 2024

Choose a reason for hiding this comment

VariableExp0rt Feb 21, 2024 • edited

Choose a reason for hiding this comment

alexellis Feb 16, 2024

Choose a reason for hiding this comment

VariableExp0rt Feb 21, 2024

Choose a reason for hiding this comment

alexellis Feb 16, 2024

Choose a reason for hiding this comment

VariableExp0rt Feb 21, 2024

Choose a reason for hiding this comment

Add AWS CLI to `system install` tools #993

Add AWS CLI to `system install` tools #993

VariableExp0rt commented Oct 25, 2023 •

edited

VariableExp0rt commented Jan 3, 2024 •

edited

VariableExp0rt Feb 21, 2024 •

edited