Omit private packages from lists of changed and unchanged packages offered by the CLI when running yarn changeset

[mtlewis]

Affected Packages

• @changesets/cli

Problem

The list of packages offered by yarn changeset includes private packages (i.e. packages with private: true in package.json). In projects with broad contribution, contributors may not be aware that only packages that get published should be included in changesets, and include those private packages. This creates noise for maintainers, who have to explain the issue and ask the contributor to fix the changeset.

Proposed solution

If a package is marked as private, it shouldn't be included in the lists of changed and unchanged packages offered by the CLI. This behavior could be behind an optional flag if there are workflows where including private packages in this list makes sense.

[NMinhNguyen]

If I'm not mistaken, this issue is related to #436.

[Andarist]

@mtlewis do you mean private: true packages, publishConfig.access: 'restricted', or both?

[mtlewis]

The former. It seems to me that whether access to a package is restricted or not shouldn't have bearing on whether a package appears in changesets, but private packages (i.e. packages with private: true in package.json) can't ever be published, so including them in changesets doesn't make sense at all.

I've edited the issue description a bit to try and make this more explicit :)

[Andarist]

There have been some feature requests in the past to allow for creating changesets for private packages but those use cases have not been explored by me. I agree that we shouldn't list those packages by default either way.

The change for this wouldn't be hard to implement - the logic is contained in the createChangeset but probably its internal helper getPackagesToRelease should be adjusted.

We should take care of edge cases though - when there are no publishable packages for some reason (if everything is private)

[bhovhannes]

@Andarist I have implement changes locally, however, I think that will be a major change because people will no longer see private packages in the list displayed by CLI which can be a pain for them if they are creating changesets for private packages as well.
To avoid breaking changes I can work on introducing a new config option - skipPrivatePackages which will be false by default, fully preserving current behavior of the CLI. When skipPrivatePackages will be set to true private packages will be totally omitted by CLI.
There are multiple ways new config option can be added, so I'll be grateful to hear maintainers opinion on this.

We don't use changesets now, but we have a use case for generating changelog for private packages. We are moving to micro-frontends and each micro-frontend is a npm package, which never gets published. Instead we just deploy content of dist folder to S3 and update import-map in html. We even want to prohibit users from publishing micro-frontends to npm, so we have private: true set in package.json.

BTW, thank you for nice code in the repo. During implementing changes I realized that I do not feel like I am working in a "foreign" repo, which was awesome!

[Andarist]

I think that will be a major change because people will no longer see private packages in the list displayed by CLI which can be a pain for them if they are creating changesets for private packages as well.

This is a valid concern but keeping the CLI such as this without any potentially breaking changes is not feasible - because almost every change would require us to major bump. I think this is a reasonable request (to skip private packages there).

To avoid breaking changes I can work on introducing a new config option - skipPrivatePackages which will be false by default, fully preserving current behavior of the CLI. When skipPrivatePackages will be set to true private packages will be totally omitted by CLI.

That is an option - although we'd have to establish what this would actually skip. By making this a config option (and not let's say a command option) it's not obvious that this would only skip listing of those packages. As a user, I would expect this to influence more than that - unless we'd nest this in some top-level key that would indicate that this is only for this listing.

Truth to be told - I'mn a little bit fuzzy when it comes to how private packages are exactly handled by Changesets. I know though that there are some use cases not satisfied with the current state of things (some issues were raised about them in the past, for instance we have this PR open). So kinda before going further with this - we'd have to kinda establish (and hopefully document) differences between those package "types".

I've taken a quick look in the codebase and I think those might be the only differences - private packages are not published (duh...) and as a byproduct of that git tags are not created for them and github releases are not created for them (if we assume that Changeset Action is used for publishing). Everything else works the same way (versioning, dependency management etc).

cc @mitchellhamilton

We don't use changesets now, but we have a use case for generating changelog for private packages. We are moving to micro-frontends and each micro-frontend is a npm package, which never gets published. Instead we just deploy content of dist folder to S3 and update import-map in html. We even want to prohibit users from publishing micro-frontends to npm, so we have private: true set in package.json.

Have you already looked into the publishing part in combination with Changesets (for those private packages)? How do you plan to check if a private package has been bumped?

BTW, thank you for nice code in the repo. During implementing changes I realized that I do not feel like I am working in a "foreign" repo, which was awesome!

Thanks! :)

[bhovhannes]

Thank you for quick response!

That is an option - although we'd have to establish what this would actually skip. By making this a config option (and not let's say a command option) it's not obvious that this would only skip listing of those packages. As a user, I would expect this to influence more than that - unless we'd nest this in some top-level key that would indicate that this is only for this listing.

I totally forgot that commands may have options. I also think that introducing new config option will affect other commands as well, and the topic of how everything should behave deserves a separate discussion and, perhaps, more complex changes in the code. I believe we can focus on CLI here, as it is easy to reason about what "private" package mean for add command - we're just omitting private packages from the list. That will fix both this issue and #436.

So, if we're fine with breaking changes, I propose adding new --include-private option for changeset add command. By default, running yarn changeset will not show private packages in UI. Running yarn changeset --include-private will make UI to display private packages as well (that is the current behavior).
I think that --include-private option for add command will not contradict with possible future changes of how Changesets behave in regards to private packages and will be useful in any case.
What do you think?

We don't use changesets now, but we have a use case for generating changelog for private packages. We are moving to micro-frontends and each micro-frontend is a npm package, which never gets published. Instead we just deploy content of dist folder to S3 and update import-map in html. We even want to prohibit users from publishing micro-frontends to npm, so we have private: true set in package.json.

Have you already looked into the publishing part in combination with Changesets (for those private packages)? How do you plan to check if a private package has been bumped?

No. Currently, we don't use Changesets. We even don't change package versions, as we use git hash as a version. But things may change when we will start thinking of adopting Changesets. Currently, we don't generate changelogs for micro-frontends, maybe Changesets can help with that.
I brought this up because I thought that in that case we still need git tagging, version bump, changelog generation, github release generation, but want to skip the step of publishing to artifactory.

[Andarist]

q: we are thinking about adjusting the behavior to omit all the packages without a version field (or just private packages without a version field). This would probably match your expectations, right? Since they would not be displayed on the list then and it could work "out of the box". People wanting to version their private packages could still do that if they include package.json#version in them.

This is just a rough idea - so we are very much eager to hear your thoughts on that.

[bhovhannes]

q: we are thinking about adjusting the behavior to omit all the packages without a version field (or just private packages without a version field). This would probably match your expectations, right? Since they would not be displayed on the list then and it could work "out of the box". People wanting to version their private packages could still do that if they include package.json#version in them.

This is just a rough idea - so we are very much eager to hear your thoughts on that.

That will work. And that way new command line option won't be necessary, which is good. Also, NPM docs say that "If you don't plan to publish your package, the name and version fields are optional.", which means we are not going against official information.

My only concern is that people may still want to have a version field for private package to be able to reference them inside a monorepo. When using yarn, workspace:* protocol helps, but I am not sure it works with NPM, which means people need to specify some version to reference package they need. But I struggle to find any practical use case when 2 private packages inside a monorepo refer to each other.

2 questions:

1. Do I understand correctly, that adjusting the behavior to omit all the packages without a version field applies not only to changeset add command but to other commands as well?

2. Does Changesets even work when version field is absent? If it does not, skipping packages without version absolutely makes sense and my concern above can be ignored.

[Andarist]

My only concern is that people may still want to have a version field for private package to be able to reference them inside a monorepo. When using yarn, workspace:* protocol helps, but I am not sure it works with NPM, which means people need to specify some version to reference package they need.

There is also file: protocol and it's supported by npm. So this could potentially still be used there.

But I struggle to find any practical use case when 2 private packages inside a monorepo refer to each other.

You may still need to reference a private package in dev dependencies of a public package - and referencing a private package from a private package doesn't look that different to me, or am I missing something? 🤔

Do I understand correctly, that adjusting the behavior to omit all the packages without a version field applies not only to changeset add command but to other commands as well?

Yes, this would have to be taken into account in the system as a whole. It's hard for me list all the exact places right now though - but hopefully this should limit itself to just a couple of places.

Does Changesets even work when version field is absent? If it does not, skipping packages without version absolutely makes sense and my concern above can be ignored.

I've recently opened a PR to fill the missing version ( #705 ) and this has prompted the discussion that maybe instead of doing that we should ignore private package without a version. The version will still be required for publishable packages though - and we might want to throw when a version is missing in such a package.