Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Document exit code handling for composite actions #31974

Closed
wants to merge 5 commits into from
Closed
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
190 changes: 187 additions & 3 deletions content/actions/creating-actions/setting-exit-codes-for-actions.md
Expand Up @@ -17,10 +17,10 @@ type: how_to

{% data variables.product.prodname_dotcom %} uses the exit code to set the action's check run status, which can be `success` or `failure`.

Exit status | Check run status | Description
------------|------------------|------------
Exit status | Step conclusion | Description
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Exit status | Step conclusion | Description
Exit status | Check run status | Description

Just updating this to match the language in the prior sentence and other language used throughout GitHub.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is wrong.

GitHub Actions Runtime maps GitHub Jobs to Check Runs. Beneath jobs are a number of other layers.

We aren't directly in a check run (GitHub Job), we're in an action inside another thing which might or might not surface that outcome. It's a fundamental problem with this documentation, it's absolutely ignorant of the fact that composite actions can nest (and have been able to for a very long time). I believe you can in fact have 10 levels of nesting in composite actions.

In some ways, check-spelling is boring, it doesn't generally nest composite actions by more than two layers, although it definitely envisions people calling it in their own composite action (and thus adding at least one more layer). Most of the actions that check-spelling calls don't have many steps in them (if they're composite actions), but that's an implementation detail of the individual actions and not a constraint by the actions subsystem. The actions we're starting to use internally will probably have multiple steps in second layer nested actions. And it's quite likely that they'll include continue-on-error.

A very incomplete diagram of check-spelling
graph TB
    Workflow{ Spelling Workflow } -- Job --> JobCheckSpelling(( check-spelling ))
    Workflow -- Job --> JobComment(( comment))
    JobCheckSpelling -- job step1 --> CheckSpellingAction[ uses: check-spelling/check-spelling ]
    CheckSpellingAction -- action step 1 --> CheckSpellingActionStep1( act-broken - run bash )
    CheckSpellingActionStep1 -- action step 2 --> CheckSpellingActionStep2( act-windows-shim )
    CheckSpellingActionStep2 -- action step 3 --> CheckSpellingActionStep3( parse-alternate-engine )
    CheckSpellingActionStep3 -- action step 4 --> CheckSpellingActionStep4[ alternate-engine uses: actions/checkout ]
    CheckSpellingActionStep4 -- potential steps in nested action --> ActionsCheckoutStep1(( implementation details... ))
    ActionsCheckoutStep1 --> CheckSpellingActionStep5
    CheckSpellingActionStep4 -- action step 5 --> CheckSpellingActionStep5( shim-path )
    CheckSpellingActionStep5 -- action step 6 --> CheckSpellingActionStep6( check-actions )
    CheckSpellingActionStep6 -- action step 7 --> CheckSpellingActionStep7[ checkout-replacement uses: check-spelling/actions-checkout ]
    CheckSpellingActionStep7  -- potential steps in nested action --> CheckSpellingActionsCheckoutStep11(( implementation details... ))
    CheckSpellingActionsCheckoutStep11 --> CheckSpellingActionStep8
    CheckSpellingActionStep7 -- action step 8 --> CheckSpellingActionStep8[ checkout uses: check-actions/checkout ]
    CheckSpellingActionStep8 -- action step 9 --> CheckSpellingActionStep9( report if checkout failed )
    CheckSpellingActionStep9 -- action step 10 --> CheckSpellingActionStep10[ checkout-merge uses: check-spelling/checkout-merge ]
    CheckSpellingActionStep10 -- action step 11 --> CheckSpellingActionStep11( raise-merge-failure )
    CheckSpellingActionStep11 -- action step 12 --> CheckSpellingActionStep12( save sha )
    CheckSpellingActionStep12 -- action step 13 --> CheckSpellingActionStep13( prepare )
    CheckSpellingActionStep13 -- action step 14 --> CheckSpellingActionStep14[ retrieve-comment-matrix-replacement uses: check-spelling/actions-download-artifact ]
    CheckSpellingActionStep14 -- action step 15 --> CheckSpellingActionStep15[ retrieve-comment-replacement uses: check-spelling/actions-download-artifact ]
    CheckSpellingActionStep14 -- potential steps in nested action --> CheckSpellingActionsDownloadArtifact(( implementation details... ))
    CheckSpellingActionStep15 -- action step 16 --> CheckSpellingActionStep16( retrieve-comment )
    CheckSpellingActionStep15 -- potential steps in nested action --> CheckSpellingActionsDownloadArtifact(( implementation details... ))
    CheckSpellingActionsDownloadArtifact --> CheckSpellingActionStep15
    CheckSpellingActionStep16 -- action step 17 --> CheckSpellingActionStep17( hash-dictionaries )
    CheckSpellingActionStep17 -- action step 18 --> CheckSpellingActionStep18[ retrieve-dictionaries uses: actions/cache/restore ]
    CheckSpellingActionStep18 -- action step 19 --> CheckSpellingActionStep19( perl-config )
    CheckSpellingActionStep19 -- action step 20 --> CheckSpellingActionStep20[ retrieve-perl-libraries-arch uses: actions/cache/restore ]
    CheckSpellingActionStep20 -- potential steps in nested action --> ActionsCacheRestore(( implementation details... ))
    ActionsCacheRestore --> CheckSpellingActionStep21[ retrieve-perl-libraries-lib uses: actions/cache/restore ]
    CheckSpellingActionStep20 -- action step 21 --> CheckSpellingActionStep21
    CheckSpellingActionStep21 -- potential steps in nested action --> ActionsCacheRestore
    CheckSpellingActionStep21 --> CheckSpellingActionStep22( install perl modules )
    CheckSpellingActionStep22 -- action step 23 --> CheckSpellingActionStep23( spelling)
    CheckSpellingActionStep23 -- action step 24 --> CheckSpellingActionStep24[ ‎save-dictionaries‎ uses: actions/cache/save ]
    CheckSpellingActionStep24 -- action step 25 --> CheckSpellingActionStep25[ ‎save-perl-libraries-lib uses: actions/cache/save ]
    CheckSpellingActionStep24 -- potential steps in nested action --> ActionsCacheSave(( implementation details... ))
    CheckSpellingActionStep25 -- potential steps in nested action --> ActionsCacheSave
    ActionsCacheSave --> CheckSpellingActionStep26
    CheckSpellingActionStep25 -- action step 26 --> CheckSpellingActionStep26[ store comment replacement uses: check-spelling/actions-upload-artifact ]
    CheckSpellingActionStep26 -- potential steps in nested action --> CheckSpellingUploadArtifact(( implementation details... ))
    JobComment --> CheckSpellingAction
    Workflow -- Job --> JobUpdate(( update))
    JobUpdate --> CheckSpellingAction

    CheckSpellingActionStep10 -- checkout-merge step 1 --> CheckoutMergeStep1( merge )
    CheckoutMergeStep1 --> CheckoutMergeStep2( there could be more steps here... )
    CheckoutMergeStep2 --> CheckSpellingActionStep11

    Key{ Key } --> KeyWorkflow{ Workflow }
    Key --> Job(( Job ))
    Key --> RunStep( run step )
    Key --> UsesAction[ uses action ]
    UsesAction --> ImplementationDetails(( implementation details ))
    KeyWorkflow --> Job
    Job --> RunStep
    Job --> UsesAction
    Job -- up to 3 levels of nesting --> Job
    Job -- Set check run status --> CheckRunStatus{ ... }
    UsesAction --> RunStep
    UsesAction -- up to 9 levels of nesting --> UsesAction

------------|-----------------|------------
`0` | `success` | The action completed successfully and other tasks that depend on it can begin.
Nonzero value (any integer but 0)| `failure` | Any other exit code indicates the action failed. When an action fails, all concurrent actions are canceled and future actions are skipped. The check run and check suite both get a `failure` status.
Nonzero value (any integer but 0)| `failure` | Any other exit code indicates the action failed. When an action fails unless the step has `continue-on-error: true` the job will be set to `failed` and future steps in the job will be skipped unless they use `if:` with either `always()` or `failed()`. Concurrent jobs will be canceled unless `jobs.<job_id>.strategy.fail-fast` is `false`.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Nonzero value (any integer but 0)| `failure` | Any other exit code indicates the action failed. When an action fails unless the step has `continue-on-error: true` the job will be set to `failed` and future steps in the job will be skipped unless they use `if:` with either `always()` or `failed()`. Concurrent jobs will be canceled unless `jobs.<job_id>.strategy.fail-fast` is `false`.
Nonzero value (any integer but 0)| `failure` | Any other exit code indicates the action failed. When an action fails, all concurrent actions are canceled and future actions are skipped. The check run and check suite both get a `failure` status.

Since this article is aimed at people who are creating actions, I think the original text here makes more sense. If someone is writing a workflow and they don't want dependent jobs to be skipped because of a failed action, they can find that information in the Workflow syntax article.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nope, you might think that, but that's because the action syntax documentation is out of date. See #31356 (comment).

I just spent a nontrivial amount of effort over the last two days w/ a coworker on a private action that uses continue-on-error: true. Yes, it isn't mentioned in https://docs.github.com/en/actions/creating-actions/metadata-syntax-for-github-actions but I can assure you that it does work and we relied on it heavily.

check-spelling has been using it publicly for a while too:
https://github.com/check-spelling/check-spelling/blob/e0d8a1df4f07a2afade0882ac903ebfaba4f0bc8/action.yml#L398

The article is aimed at people like me, and I'm trying to fix it based on my experience as an action author.

I don't use continue-on-error in workflows, but I do use it in my actions.

Beyond that, the current text is totally wrong.

The only things concurrent to an action are Jobs, not actions. Within a job you can have actions or run statements, and a run statement in another job will be killed just as much as another action in another job would be killed -- but details about which is killed would be a layering violation -- at most the death of an action leads to a cascade -- its parent action dies until eventually its ancestor action kills its job, and then the death of the job results in the death of the job's siblings, which in turn would result in the death of job steps which may include other actions or run-shell steps. -- But it isn't that the death of the current action triggers any of those things directly, they're at most casualties of other things.


## Setting a failure exit code in a JavaScript action

Expand Down Expand Up @@ -48,3 +48,187 @@ fi
```

For more information, see "[AUTOTITLE](/actions/creating-actions/creating-a-docker-container-action)."

## Handling exit codes in composite actions
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would you be able to shorten this whole section to a relevant example that explains how a non-zero exit in the middle of a composite action will cause the remaining composite action to behave? So that it is structured more similarly to the previous two sections about Docker and JavaScript actions?

The audience for this article is people who are creating actions. Because of that, any content that is directed towards people who are writing workflows should be removed from this article. If you think the documentation in the Using workflows section of the docs is missing any of this information, you can definitely open an issue to propose adding it there. 👍

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't have the energy to do that at this time. Between dealing w/ a dozen other problems (including runners repeatably dying with my workflows), this was as good as I could manage while still providing something where someone could actually see an outcome.


### Handling failing steps in composite actions

If a step exits with a Nonzero value (any integer but 0), its `outcome` will be `failure`.

Any step in a composite action that doesn't have `continue-on-error: true` and has a `failure` outcome will cause the composite action `outcome` to be `failure`.

To perform other logic based on a step that uses an action that can fail for which you don't want to automatically have the consuming action fail, you can use `continue-on-error: true` on the step and then use `steps.<step_id>.outcome` for that possibly failing step to control the logic of the remaining steps in the composite action. For more information, see "[AUTOTITLE](/actions/learn-github-actions/contexts#steps-context)."

Consumers of an action in a matrix with `jobs.<job_id>.strategy.fail-fast` set to `true` will be canceled when an action step fails without `continue-on-error: true`.

### Setting a success outcome in a composite action

As long as all steps without `continue-on-error: true` yield an exit code of `0` whether explicit or implicit (e.g. a shell run step where commands don't fail or the shell is set not to treat failed steps as failures), the action's outcome will be success.

### Setting a failure in a composite action

Set the exit code of a step to a nonzero value and ensure that the step does not have `continue-on-error: true` to cause the composite action `outcome` to be `failure`.

### Examples of success and failure in composite actions

`maybe-fail/action.yaml`:

```yaml
{% raw %}name: 'Maybe fail'
description: 'Conditionally fail based on inputs'
inputs:
return-code:
description: 'Value to use as exit code for main step'
required: false
default: '0'
continue-on-error:
description: 'Whether to run the main step with `continue-on-error`'
required: false
default: 'false'
outputs:
result:
description: "Result for posterity"
value: ${{ steps.report-outcome.outputs.result }}
runs:
using: "composite"
steps:
- name: Run step if continue-on-error = false
id: continue-on-error-false
if: ${{ inputs.continue-on-error == 'false' }}
continue-on-error: false
run: exit ${{ inputs.return-code }}
shell: bash
- name: Run step if continue-on-error != false
id: continue-on-error-true
if: ${{ inputs.continue-on-error != 'false' }}
continue-on-error: true
run: exit ${{ inputs.return-code }}
shell: bash
- id: report-outcome
if: success() || failure()
run: echo "result=$outcome" | tee -a "$GITHUB_OUTPUT"
env:
outcome: ${{ steps.continue-on-error-false.outcome != 'skipped' && steps.continue-on-error-false.outcome || steps.continue-on-error-true.outcome }}
shell: bash{% endraw %}
```

`use-maybe-fail/action.yaml`:

```yaml
{% raw %}name: 'Use maybe fail'
description: 'Call maybe fail to conditionally fail based on inputs'
inputs:
return-code:
description: 'Value to use as exit code for main step'
required: false
default: '0'
continue-on-error:
description: 'Whether to run the main step with `continue-on-error`'
required: false
default: 'false'
outputs:
result:
description: "Result for posterity"
value: ${{ steps.maybe-fail.outputs.result }}
runs:
using: "composite"
steps:
- name: Call maybe fail
id: maybe-fail
uses: ./maybe-fail
with:
return-code: ${{ inputs.return-code }}
continue-on-error: ${{ inputs.continue-on-error }}
- id: report-outcome
if: (success() || failure()) && steps.maybe-fail.outcome == 'failure'
run: |
echo 'Something must be done because maybe-fail failed!' | tee -a "$GITHUB_STEP_SUMMARY"
shell: bash{% endraw %}
```

`.github/workflows/maybe-fail.yaml`:

```yaml
{% raw %}name: Maybe-fail

on:
push:
pull_request:
workflow_dispatch:
jobs:
use-action-without-fail-fast:
runs-on: ubuntu-latest
strategy:
matrix:
return-code:
- 0
- 1
continue-on-error:
- true
- false
fail-fast: false
name: Maybe fail
steps:
- uses: {% data reusables.actions.action-checkout %}
- name: Use maybe-fail action
id: maybe-fail
uses: ./use-maybe-fail
with:
return-code: ${{ matrix.return-code }}
continue-on-error: ${{ matrix.continue-on-error }}
- name: After Action
if: success() || failure()
env:
conclusion: ${{ steps.maybe-fail.conclusion }}
outcome: ${{ steps.maybe-fail.outcome}}
result: ${{ steps.maybe-fail.outputs.result }}
run: |
(
echo "outcome: $outcome"
echo "conclusion: $conclusion"
echo "result: $result"
echo
) | tee -a "$GITHUB_STEP_SUMMARY"
- name: Next Step
run:
echo Next step | tee -a "$GITHUB_STEP_SUMMARY"
use-action-with-fail-fast:
runs-on: ubuntu-latest
strategy:
matrix:
return-code:
- 0
- 1
continue-on-error:
- false
fail-fast: true
name: Maybe fail fast
steps:
- uses: {% data reusables.actions.action-checkout %}
- name: sleep
if: ${{ matrix.return-code == 0 }}
run:
sleep 10
- name: Use maybe-fail action
id: maybe-fail
uses: ./maybe-fail
with:
return-code: ${{ matrix.return-code }}
continue-on-error: ${{ matrix.continue-on-error }}
- name: After Action
if: success() || failure()
env:
conclusion: ${{ steps.maybe-fail.conclusion }}
outcome: ${{ steps.maybe-fail.outcome}}
result: ${{ steps.maybe-fail.outputs.result }}
run: |
(
echo "outcome: $outcome"
echo "conclusion: $conclusion"
echo "result: $result"
echo
) | tee -a "$GITHUB_STEP_SUMMARY"
- name: Next Step
run:
echo Next step | tee -a "$GITHUB_STEP_SUMMARY"{% endraw %}
```