Improve error message for missing configuration

[TimonVS]

Via #362

Currently we show the following when you're missing the Release Drafter config file:

11:13:07.934Z INFO probot: caicaishmily/actions_for_angular: No valid config found

We should update the message to be more actionable or link to the docs so that people can figure out how to solve the issue.

[jetersen]

With joi we can add custom error messages to each and we can further parse the errors before logging.

[jsoref]

I was going to file this, but apparently it's filed here, so...

#398 reported this problem but it doesn't appear that anyone improved the situation

https://github.com/jsoref/jenkins/runs/501901399?check_suite_focus=true

Action run log

Skip to content
Search or jump to…

Pull requests
Issues
Marketplace
Explore
 
@jsoref 
jsoref
/
jenkins
forked from jenkinsci/jenkins
1
06.1k
 Code Pull requests 0 Actions Projects 0 Security Insights Settings
Merge pull request #4561 from daniel-beck/SECURITY-1754
master @timja  01baa2f
 
Changelog Drafter
on: push
 
Annotations: 1 failure
Changelog Drafter / update_draft_release
failed 11 hours ago in 1m 30s
Search logs
  Set up job2s
  Build release-drafter/release-drafter@v5.6.046s
  Build jenkinsci/jenkins-core-changelog-generator@master39s
  Generate GitHub Release Draft2s
##[error]Invalid config file
/usr/bin/docker run --name e87b52ab3426808d504150b634e20916f0c743_dcaa37 --label e87b52 --workdir /github/workspace --rm -e GITHUB_TOKEN -e HOME -e GITHUB_REF -e GITHUB_SHA -e GITHUB_REPOSITORY -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_ACTOR -e GITHUB_WORKFLOW -e GITHUB_HEAD_REF -e GITHUB_BASE_REF -e GITHUB_EVENT_NAME -e GITHUB_WORKSPACE -e GITHUB_ACTION -e GITHUB_EVENT_PATH -e RUNNER_OS -e RUNNER_TOOL_CACHE -e RUNNER_TEMP -e RUNNER_WORKSPACE -e ACTIONS_RUNTIME_URL -e ACTIONS_RUNTIME_TOKEN -e ACTIONS_CACHE_URL -e GITHUB_ACTIONS=true -v "/var/run/docker.sock":"/var/run/docker.sock" -v "/home/runner/work/_temp/_github_home":"/github/home" -v "/home/runner/work/_temp/_github_workflow":"/github/workflow" -v "/home/runner/work/jenkins/jenkins":"/github/workspace" e87b52:ab3426808d504150b634e20916f0c743
Deprecation: [@octokit/request-error] `error.code` is deprecated, use `error.status`.
    at RequestError.get (/app/node_modules/@octokit/request/node_modules/@octokit/request-error/dist-node/index.js:29:17)
    at loadYaml (/app/node_modules/probot-config/lib/index.js:51:11) {
  name: 'Deprecation'
}
22:43:17.024Z  WARN probot: jsoref/jenkins: Invalid config file
  ValidationError: child "template" fails because ["template" is required]
      at Object.exports.process (/app/node_modules/@hapi/joi/lib/errors.js:202:19)
      at internals.Object._validateWithOptions (/app/node_modules/@hapi/joi/lib/types/any/index.js:763:31)
      at internals.Object.validate (/app/node_modules/@hapi/joi/lib/types/any/index.js:797:21)
      at validateSchema (/app/lib/schema.js:88:52)
      at getConfig (/app/lib/config.js:23:20)
      at async /app/index.js:11:20
22:43:17.034Z  INFO probot:
  jsoref/jenkins: Config validation errors, please fix the following issues in release-drafter.yml:
  ┌──────────┬────────────────────────┐
  │ Property │ Error                  │
  ├──────────┼────────────────────────┤
  │ template │ "template" is required │
  └──────────┴────────────────────────┘
##[error]Invalid config file
 Generate YAML changelog draft0s
 Upload Changelog YAML0s
  Complete job0s

Popup output

Annotations: 1 failure
 .github:1

Invalid config file

Steps

1. Set default branch to a spell-checking action instead of jenkins/master (this branch isn't related to jenkins/master)
2. Pushed upstream's master to the repository
3. Looked at an action run for master

Expected results

1. error messages should be helpful and understandable. If the problem is that a project's default branch needs configuration, then the error should say so (it does not; it is not remotely obvious that this is necessary)
2. the annotation message should be more helpful than ".github:1" which is really bogus.
3. personally, for a project like this, i'd expect an action to warn "this repository's default branch, (insert branch name here), does not have a configuration for this action -- see (insert url for documentation for action here)" and then quit. I think I'd actually rather a green check over a red x in such a case, but ideally something that is neither (a gray or yellow aborted/skipped icon would be better).

Technical improvements

 22:43:17.024Z  WARN probot: jsoref/jenkins: Invalid config file
-  ValidationError: child "template" fails because ["template" is required]
+  ValidationError: child "template" failed because ["template" is required]

-- while it's possible to think about things in the present continuous tense or future tense for testing, reporting to humans should be in the past tense. -- this appears to come from using an outdated version of @hapi/joi as the string was removed by hapijs/joi@6182a9b before v16.0.0 (current appears to be v17.1.0).

-  jsoref/jenkins: Config validation errors, please fix the following issues in release-drafter.yml:
-  ┌──────────┬────────────────────────┐
-  │ Property │ Error                  │
-  ├──────────┼────────────────────────┤
-  │ template │ "template" is required │
-  └──────────┴────────────────────────┘
+  jsoref/jenkins: Config validation errors, please fix the following issues in https://github.com/jsoref/jenkins/blob/spell-checking/.github/release-drafter.yml:
+  configuration not found
+  see ... for more information

[jetersen]

In your default branch there needs to be a release-drafter.yml file looking at https://github.com/jsoref/jenkins you only have the workflow file.

[jsoref]

@jetersen: I'm not asking how to fix the problem. I'm asking the action to fix its error handling. Which is the issue behind this issue.

My branch is for my use. I don't expect and shouldn't have to expect that my branch should be impinged by some other random branch. If some other random branch is missing a requirement, it should politely explain that to me instead of failing w/ a random incomprehensible error message.

[jetersen]

Remove action from the workflow? I am not sure I follow what exactly your issue is.

[jetersen]

Oh I see, you changed the default branch and because of how context.config works inside probot it cannot find the file when the actions on the master branch is run.

I suggest in your default branch add that single file to your default branch to make the action on the master branch happy. Sorry but this is how the core of both GitHub actions and probot works.

See https://github.com/probot/probot/blob/22d8215a631735f42c08b75ecc2657b7dcec2a54/src/context.ts#L189

Or revert the default branch to master.

[jsoref]

My action doesn't suffer from this. It might be how probot works, but it isn't how actions work in general. I've played w/ probot for a long time and have recently been working on my own action. There's nothing in the general GitHub action documentation that covers this.

And again, it isn't a good usability feature to have this error behavior. The error reporting should at the very least be actionable by a human. They shouldn't have to search for bug reports to try to figure out the problem/fix. Nor should they have to search for the source code in a dependent library.

Also, I can't change my default branch because I need to run a schedule which only runs for the default branch. And again, I shouldn't be forced to poison my default branch just to make some random action in some other branch happy. If the preconditions for an action in a non default branch aren't met, it should fail gracefully, w/ a pretty error, not in the ugly manner in which this action fails today.

[jetersen]

This was fixed in #973

[jetersen]

Could be improved even more by using: https://github.com/probot/octokit-plugin-config/blob/a613c0efd2873fb64a3a87f79365ec1951fd29ad/src/index.ts#L13 directly, as we would be able to specify a branch to get the config from. Actually this function is even better as it will attempt to use .github repo without using _extends