You can't add annotations when the PR comes from a fork?

Unfortunately no. GitHub only gives forks a read only token to make api requests including status checks.

Bummer. Makes it less useful for Open Source projects.

Agreed. Actions are still in the early stages so at some point there might be a way to do this.

Another bummer is that any secret managed by github is not provided to actions when run from a fork. This is to prevent malicious actors from using the secret (since the fork can modify the action steps that get invoked).

If you write workflow commands to stdout instead of making web API calls which depend on the token, the authentication issue goes away and the performance may even be better.

Example from docs page (https://docs.github.com/en/actions/reference/workflow-commands-for-github-actions#setting-a-warning-message):

echo "::warning file=app.js,line=1,col=5::Missing semicolon"

Both warning and error annotations can be created this way. A GitHub action using this approach: https://github.com/cschleiden/jest-github-actions-reporter

@jnm2 this is a great idea. I'll see about making this change soon.