Ignore fails on base branch

[LironEr]

• when a PR fails the bundle checks, then the check is marked as failed for the PR (which is obviously the correct behavior)
• if this is an optional check then, as a reviewer, even though it has failed I can approve and merge the PR
• BUT now, the check fails on the master/main branch, which makes it look like the build is failing

Originally posted by @markwhitfeld in #115

Hey,

BUT now, the check fails on the master/main branch, which makes it look like the build is failing

If the target branch doesn't exist (meaning we are not in PR), BundleMon will compare your current commit to a previous commit in the same branch you currently are. This was implemented to show the correct message at the end of the run and in the commit status. But I think you are right and there is no need to fail the run if we are comparing commits in the same branch.

What do you think about adding a new flag to ignore fails in the same branch?

ignoreFailuresInSameBranch: true / false (default true)

The flag will only change the result to success

Originally posted by @LironEr in #115 (comment)

Super! Thank you so much for the response. Regarding the master branch, I don't know what is possible with checks, but potentially they could be represented as warnings and not failures? And for the setting name, I think that the terminology could potentially be "base" branch, because some might interpret commits within the PR branch as the "same" branch. Potentially the setting could be called: warnForFailuresOnBaseBranch: true/false Potentially making the default true might be an unexpected behaviour change (currently it would behave as if it was set to false). But it is up to you if you see the current failure on the base branch as a bug or a feature, so it is up to you 👍

Just jotting down the progression of my thoughts here.....

An alternative setting name could be: baseBranchFailureHandling: 'error'/'warn'/'ignore' or baseBranchFailureStrategy: 'error'/'warn'/'ignore' or baseBranchCommitValidation: 'error'/'warn'/'ignore'

or even ignoreFailuresOnBranches: [ 'master', 'main', 'release' ] or warnOnlyForFailureOnBranches: [ 'master', 'main', 'release' ] or softFailureOnBranches: [ 'master', 'main', 'release' ]

or even this could be in the github reporter options like this

"reportOutput": [
  [
    "github",
    {
      "checkRun": false,
      "commitStatus": true,
      "prComment": true,
      "branchCheckHandling": {
          "master": { "reportFailureAs": "warning" }, 
          "dev": { "reportFailureAs": "error" }, 
          "release": { "reportFailureAs": "information" }, 
      }
    }
  ]
]

I think I like the configurable aspect of the last one. Obviously whether or not we can do warnings would affect the design.

Haha, really over-engineering now, Imagine if we could even have "reportFailureAs": [ "warning", "email" ], which would email the code owners. Hahaha, I think there is some serious scope creep here 🤣 .

Originally posted by @markwhitfeld in #115 (comment)

[markwhitfeld]

Thank you for splitting this out. Happy to chat about API design if it helps. I added so many thoughts above ☝️

[LironEr]

Regarding the warning option:

The Commit status API allows external services to mark commits with an error, failure, pending, or success state
https://docs.github.com/en/rest/commits/statuses#about-the-commit-statuses-api

---

Potentially making the default true might be an unexpected behaviour change (currently it would behave as if it was set to false). But it is up to you if you see the current failure on the base branch as a bug or a feature, so it is up to you 👍

I believe the default option should be to fail the CLI only when running in a PR, but I might miss something..
Soon I will release BundleMon v2, and it will have breaking changes anyway, we can add another change (:

---

I'm trying to keep it simple without manual intervention of writing your base branch names.

so maybe these names can work, without the warn option

An alternative setting name could be: baseBranchFailureHandling: 'error'/'warn'/'ignore' or baseBranchFailureStrategy: 'error'/'warn'/'ignore' or baseBranchCommitValidation: 'error'/'warn'/'ignore'

More options:

1. Following ability to post GitHub output only on failures #123 new functionality, what about adding a new option to post github output only in a PR?
   Something like that:

   "reportOutput": [
     [
       "github",
       {
         "checkRun": false,
         "commitStatus": "pr-only",
         "prComment": "on-failure"
       }
     ]
   ]

   But that won't cover the CLI fail exit code

2. More name options for a flag:
   failOnlyInPR / runChecksOnlyInPR / ignoreErrorsInBaseBranch

   I find these names also a bit confusing... I hate naming stuff 😅

And maybe implement both options?

Would love to hear more ideas (:

[LironEr]

The change is available in bundlemon@2.0.0-rc.1

[markwhitfeld]

Oh man, I missed notifications about this.
If you would like to discuss any ideas (relevant to my requests or not), I will always be willing to be a sounding board. My github notifications are overrun, so I often miss them, but a DM on twitter to @markwhitfeld would be a good way to ping me if you would like my thoughts.

[LironEr]

Awesome, thank you!

Currently, I just changed the default behavior in the next major version without the option to change it, if someone will want it I will do that, but I'm not sure if anybody will want to change this behavior...