Add SSM Commands stdout/err to audit log #41478
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
marcoandredinis
added
the
no-changelog
hugoShaka
approved these changes
May 13, 2024
justinas
approved these changes
May 13, 2024
marcoandredinis
force-pushed
the
marco/ec2-ssm-describeinstancesinfo
branch
from
11a7910
to
37e2efb
Compare
marcoandredinis
force-pushed
the
marco/ec2-ssm-describeinstancesinfo
branch
2 times, most recently
from
0517d72
to
6b59009
Compare
This PR adds two new fields to the SSMRun audit events: -stdout -stderr This will help diagnose the failures of teleport installations in EC2 instances using SSM (EC2 Auto Discover).
marcoandredinis
force-pushed
the
marco/ssmrun-stdout-stderr
branch
from
e7b7c43
to
a845f78
Compare
|
@marcoandredinis See the table below for backport results.
|
marcoandredinis
added a commit
that referenced
this pull request
May 16, 2024
This PR adds two new fields to the SSMRun audit events: -stdout -stderr This will help diagnose the failures of teleport installations in EC2 instances using SSM (EC2 Auto Discover).
marcoandredinis
added a commit
that referenced
this pull request
May 17, 2024
This PR adds two new fields to the SSMRun audit events: -stdout -stderr This will help diagnose the failures of teleport installations in EC2 instances using SSM (EC2 Auto Discover).
marcoandredinis
added a commit
that referenced
this pull request
May 20, 2024
This PR adds two new fields to the SSMRun audit events: -stdout -stderr This will help diagnose the failures of teleport installations in EC2 instances using SSM (EC2 Auto Discover).
marcoandredinis
added a commit
that referenced
this pull request
May 20, 2024
This PR adds two new fields to the SSMRun audit events: -stdout -stderr This will help diagnose the failures of teleport installations in EC2 instances using SSM (EC2 Auto Discover).
justinas
pushed a commit
that referenced
this pull request
May 20, 2024
This PR adds two new fields to the SSMRun audit events: -stdout -stderr This will help diagnose the failures of teleport installations in EC2 instances using SSM (EC2 Auto Discover).
marcoandredinis
added a commit
that referenced
this pull request
May 21, 2024
This PR adds two new fields to the SSMRun audit events: -stdout -stderr This will help diagnose the failures of teleport installations in EC2 instances using SSM (EC2 Auto Discover).
marcoandredinis
added a commit
that referenced
this pull request
May 21, 2024
This PR adds two new fields to the SSMRun audit events: -stdout -stderr This will help diagnose the failures of teleport installations in EC2 instances using SSM (EC2 Auto Discover).
marcoandredinis
added a commit
that referenced
this pull request
May 22, 2024
This PR adds two new fields to the SSMRun audit events: -stdout -stderr This will help diagnose the failures of teleport installations in EC2 instances using SSM (EC2 Auto Discover).
github-merge-queue bot
pushed a commit
that referenced
this pull request
May 22, 2024
…dit log (#41664) * Improve messages when EC2 Auto Discover with SSM fails (#41465) * Improve unavailable messages when EC2 Auto Discover with SSM fails EC2 Auto Discover calls ssm:SendCommand to install teleport in a set of EC2 Instances. This requires that the SSM Agent to be running and reporting back to the AWS SSM Service. This PR adds a new API call which is used to query the current status of the SSM agent in the target EC2 instance. If the agent did not register, is not currently online or the EC2 instance is running an unsupported operating system, an error is reported. The specific error is returned and the user can see this in the Audit Log. As an example, let's say we have 3 instances: - i-A: missing IAM permissions to connect to SSM - i-B: SSM ran but is now unhealthy - i-C: instance is running Windows Previously we had the following observable output after running the Discovery Service: i-A (missing iam permissions) Log message with stack trace indicating that "instance is not valid for account" with link for further troubleshoot. No audit log was emitted i-B (SSM is unhealthy) No app log, but audit log with status:failed and exit_code:-1 i-C (windows instance) No app log, but audit log with status:success and exit_code:0 After this PR, the following is reported: i-A (missing iam permissions) No app log Audit log with a clear status message (see code/tests) i-B (SSM is unhealthy) No app log Audit log with a clear status message (see code/tests) i-C (windows instance) No app log Audit log with a clear status message (see code/tests) If any other error happens, it will still be reported in the generic handler for the SendCommand API call. Given this is a new API call, if the Role does not allow it, a log warning is emitted and the behavior is the same as before. * best effort on emitting events * improve maxresults param * Add SSM Commands stdout/err to audit log (#41478) This PR adds two new fields to the SSMRun audit events: -stdout -stderr This will help diagnose the failures of teleport installations in EC2 instances using SSM (EC2 Auto Discover). * SSMRun Audit Event: add invocation url (#41663) This PR adds a new field in the SSMRun audit event: invocation url. EC2 Auto Discover uses SSM to install teleport in the target instance. An invocation is the execution of a Command in an Instance. This URL points to that invocation and users can more easily debug what went wrong and how they can fix in case of a failure. * EC2 Auto Discover with SSM: add script stdout and stderr to audit log (#41479) This PRs fills in the stdout and stderr fields of the SSMRun audit event. The script to install teleport in ec2 instances has two steps: download and run shell script. This will help diagnose what failed during the auto discover of ec2 instances. * Fix EC2 Auto Discover SSM failure when sending an extra param (#41532) For agentless installations we would send an extra param to the ssm:SendCommand API. Customers can create and use custom SSM Documents, however, when using the default one, that parameter does not exist. The ssm:SendCommand API returns an error if an extra param is sent. This PR does a best-effort to accomodate for that: if a known error is returned and the known extra param was sent, remove it and try again. * EC2 Auto Discover with SSM: add invocation url to audit log (#41689) This PR adds the invocation URL into the audit log when running the teleport installer script during EC2 Auto Discover. * EC2 Auto Discover SSM: add support for debugging custom SSM Docs (#41706) This PR uses a new AWS API that list the steps of the current invocation. After listing them, it will ask for the output of each one. Previously, we were using a static list of steps: those defined in the default SSM Document. However, for custom documts with different list of steps that would fail. If the client does not have access to this new API, we will fallback to the list of steps that exist in the default SSM Document. If we ask for a status of one of those steps, and we receive a known error indicating that the step does not exist, instead of failing we will emit the overall invocation result (which doesnt include stdout/stderr, but better than nothing)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds two new fields to the SSMRun audit events:
-stdout
-stderr
This will help diagnose the failures of teleport installations in EC2 instances using SSM (EC2 Auto Discover).
Related #37620