Proposal to transfer the Corepack repo into the Node.js organization #558
Comments
|
+1 |
|
I don't like the idea of moving a TS project to the org to be added as dependency to Node.js, but we already have precedence and I think I'm somewhat alone in this position so 🤷🏻♀️ +1 to moving the repository anyway. We should have an onboarding plan to have more folks working on that project as well. |
You are not alone. |
|
Even putting aside my apprehension of having a TS project in core, I'm not really seeing how this and the linked PR are worth the extra maintenance overhead versus just leaving it in userland. 🤔 |
|
What's the concern about using typescript? @Qard ... I'm curious what additional overhead you're concerned about. There should be very little regular maintenance required... At least, definitely no more so than our current npm maintenance. |
|
TS is more an abstract concern to me--mainly just that it introduces yet another thing maintainers would need to learn. However, I'm less concerned about that in this instance, I mostly just included that to share that I had similar concerns of TS in core to others whom commented here. I'm more just questioning the value of having this particular thing in core. It's not a huge chunk of code, but it's not small either and not clear to me there's a major benefit to it being in core. It also seem a bit too magic to me, could potentially cause environment issues or user confusion. |
|
Well, let's keep in mind that moving the repo in to the node.js org is a separate matter than deciding to ship it in core. We can decide one long before deciding the other. |
|
@jasnell personally I would like to see the repository move to the org before landing the PR, but if I'm alone in that opinion I'm happy to drop that part of my objection there. |
|
Also worth noting that since there are no objections here yet from TSC or CommComm (just a raised concern on TS), we can transfer the repository in 49 hours (unless we get an objection before that). So which happens first is probably moot since it's very likely that the repo will be transferred first (given the PR is not implemented as experimental at this point).
It introduces an unnecessary contribution barrier for current Node.js collaborators who are not familiar with TypeScript. |
|
One thing we might want to consider is to also release this as |
Note that it also removes a significant part of the maintenance burden. In the end I'm fine with removing it if you feel very strongly about it, but as the one who'll keep reviewing PRs on this project, I can tell I would feel much more confident if I had a type checker running. |
|
The question of what language is used for this should be up to the developers who are building and maintaining it and should not be a factor in deciding whether the repo can move into the organization or not. |
I disagree there. The maintainers are likely to change over time. |
|
Note that I shared the project with some folks and was told that it's harder to read/follow by at least one of them for the exact reason I mentioned: TypeScript is not JavaScript. I feel strongly about it but not enough to stop progress, but the possibility of switching back to JS if TS becomes a maintenance barrier should be on the table. |
|
Also, as the main maintainer of llnode today, we should avoid having projects in the org that are only maintained by one person, it makes everything so much harder and more fragile. |
|
I think there are a few reasons this should be held for now:
I think we can resolve all of these things, but think we should before moving forward with a solution (even a good one at which many think this is). Ideas for next steps on the discussion:
I think the @nodejs/package-maintenance, @nodejs/tooling, and @nodejs/modules groups probably all have some stake in this decision. I don't think the domain fits completely in the scope of any of them, but any decision on this without feedback and input from these is one with incomplete consensus. EDIT: changed some wording above. My goal here is to make sure this is the right change for all the constituencies involved and make sure the process is inclusive and open. Wording updated to try and reflect that more clearly. |
|
|
I do think the PM WG has the most overlap. We also have a charter written, many active members, and even a place for "userland" modules to live. I think that group would be a great place to do either 1 or 2. |
I would not see this as a reason to block moving the repo in to the node.js organization. In fact, doing so would likely increase the visibility and attract more people into the conversation.
There have been clear statements made about folks from both the yarn and pnpm projects contributing and I will be participating as a contributor also.
Please clarify what you mean here. Moving this project into the node.js organization is not the same as distributing this with the node distribution. Those are separate considerations. For instance
Again, that shouldn't stop the repo from moving in under the node.js github organization. If we need a working group to manage this, then I propose establishing a new @nodejs/package-management working group that would own responsibility for both this and the existing npm client. To be clear, I disagree that this falls under the package maintenance working group charter. @cjihrig :
Yes, and as those maintainers evolve the project can evolve, including migrating to JavaScript should those maintainers decide to do so. |
From the corepack readme: "In practical terms, Corepack will let you use Yarn and pnpm without having to install them - just like what currently happens with npm, which is shipped by Node by default." This is effectively the problem that corepack is looking to solve -- essentially, helping to somewhat level the playing field across multiple package manager options. |
If the groups most closely involved do not think this is the right direction at all, do you still think so? I see no conversation saying there is consensus on this part across all the involved groups. Again, maybe I am wrong and that conversation exists and I was just not aware.
I intended to make the above argument for both parts (including it in the distribution and moving the repo), but since this was the discussion on where the repo lives, happy to leave it at that for this discussion.
I am not clear on if a WG is required, but I for sure think that the conversations I have seen so far are nowhere near consensus. My thought with it starting with a smaller group is to help give structure to the conversation. A few strong voices with time to invest are great, but also is how we got into many of these problem in the first place.
This is not a clear statement of a problem. "helping to somewhat level the playing field across multiple package manager options." is a solution still but more close to the problem I see. This seems to me as a reasonable problem statement:
And to be clear, I think this is not even the entirety of problem, just a possible first draft of a problem statement. If you take if from here, there are many possible solutions to this, of which a "package manager manager" would be one. Does this help make more sense of what I meant by "no clear problem state"? |
If we don't create a working group, and it doesn't fall under the package maintenance working group, it will be chartered under TSC, correct? IOW if there are disagreements on the repository that can't be resolved through collaboration, issues/PRs would be escalated to the TSC? |
Yes. |
Let's try to make this more actionable then so that those of us advocating for this know what the goalposts actually are... A couple questions:
|
|
Ok, here is my (maybe) actionable feedback: I don't think that Node.js shipping a tool to shim package managers serves our users or the maintainer of the project. It adds complexity and maintenance costs with no return value in the form of user happiness or productivity. There is no doubt that if you start from a problem statement of "users of yarn/pnpm have to run npm i -g yarn/pnpm, which gives npm an inherent advantage", then it is clear why corepack's approach would be one of the top solutions on the table. I don't think this is the compelling nor accurate problem statement. I don't think I alone can provide the correct problem statement, which is why I thought we should delegate to a group. But for the sake of making this more actionable, lets start with a problem statement (edited from above a bit), then think about gaps which are not addressed.
First, and foremost, is the change for an end user. Nowhere in this problem statement does it say "end users cannot do X because they first need to npm i -g yarn". If we had clear evidence that this step caused end user problems (not just the adoption of the tool, which is only an issue for the tool maintainers), then doing work to support them might be justified. I have seen no such evidence. Secondly, the problem statement talks about not having meaningful stake in the direction of the package manager. This proposal does not help this part of the problem at all, just means we have multiple to projects to keep up with which we still have no direct controlling stake in. The goal of increased competitiveness is served by this proposal, but only for the current three supported PMs. I think this part of the proposal is good, but creates new problems which do not exist today. For the node project, it means formalizing a process for including new PMs in this list. For end users it means more decision making and a major difference in behavior across node versions (older need install, so update docs and scripts, etc). For tool makers, it means new features (for example The goal of reduced vendor lock in (namely to npm) is not improved by this solution, as once a user chooses their PM, the are locked because of incompatibilities across the ecosystems. Those could be resolved by cooperation among the PMs, but relies on them being willing (again no input from the Node.js Project directly). The goal of stagnation in the tooling seems to have already been served by great work by the Yarn and pnpm folks. Yes they would like to see their adoption increase, but just shipping a shim does not solve that problem either (just like Fastify having Express compat, this is just a time thing). A proposal which would more clearly align with the problem statement above would be for node to ship with a set of tools which package managers/frameworks could build on top of and which end users could directly use in simple use cases.
It comes with a huge host of other issues, and maybe those would overcome the proposal, but we have not AFAIK had this line of conversation out to it's end. I wanted to go through the process above as an example, but I strongly believe in getting the right people in the conversation, and so please do not take my solution above comment as what I think the right direction is, just a possible one from the problem statement. |
I wouldn't be so quick to dismiss this as an argument as maintainers of both yarn and pnpm have expressed this exact concern. You may not find the issue compelling but there's no doubt that it's accurate.
While this is a problem statement that I would agree with, this is not the problem that I think is being addressed here. The proposal on the table does not change the status quo with regards to npm's privileged position.
Tool/project maintainers are Node.js users too. We cannot discount their requirements simply because they do not fall into the right group of users.
This is not a problem we're trying to solve.
Currently there is zero possible path for new package managers. With corepack there would at least be an onramp requiring nothing more than a pull request. The process would be no different than landing any other change. For end users, it's no more additional decision making than they have currently (they already decide among a wide range of tools, including yarn and pnpm).
Also not a problem we're trying to solve.
I've tried in the past to garner support for this kind of approach to no avail. There is too must resistance to changing the current user experience around npm and, frankly, I'm tired of fighting that battle. I'm interested in a solution that makes it easier for users to choose which package manager they use, that is all. |
|
Cool, that bit clears up some questions for me, thanks @jasnell. I completely respect the position that pushing through against the resistance is difficult and often not worth the effort, and my feedback in this thread is exactly that kind of resistance (I apologize for that). Unfortunately, I think I strongly believe that the problem needs to include those other things, otherwise we just smear the problem around instead of solving real things for real people.
This is one reason I think delegating to a small group would help. If you look at the work we have been discussing on the web-server-frameworks team, I think the process has worked really well to identify the problem then set about fixing it. If we did the same here, we could hopefully have similar results. EDIT: one of the critical things for the web-server-frameworks success is that we have folks from most of the important community projects which would build on top on board. Maybe we don't have that here. |
I don't think Node should ship with any such set of basic package manager building blocks. Each package manager is built with different goals and features in mind, which would make a common core very hard to integrate. As an example, Yarn's core is built to be language-agnostic, which makes it possible for us to add support for other programming languages. Because of this, it has some very basic building blocks such as What I'm trying to say is that there's no single "right way" to implement a package manager core. There are always different trade-offs involved. I believe that forcing package managers to use a common core would only hurt the ecosystem and discourage innovation. Instead of Node shipping a common core, a better idea would be for it to draft a common specification that all package managers included in Each package manager should be allowed to extend the minimum feature set because, as far as users are concerned, each package manager "lives" in its own isolated bubble until publishing, which is the only time when it gets important to enforce that a published package works with the minimal specification and, by extension, with all package managers that implement it. |
Issue nodejs/node#15244 got opened three years ago, and got revived a few times since then (each time by people unaffiliated to the package manager teams) - each time with the same request. It totals 89 comments at the moment, and it's unlikely someone has something new to add. This problem isn't new. Both me and @zkochan have been maintaining our respective projects for years, have interacted with our users more than anyone here, and have gathered their feedback. This isn't an unidentified problem either - everyone I talked with from the Node foundation has been painfully aware of the problems surrounding the npm monopoly. What it is is a problem in lack of advocated solutions. This is what we're working to fix, thanks to a solution built and supported by the actual maintainers of the relevant tools in use by our community right now. |
I don't think this point can be overstated. Corepack is being brought to the table by maintainers of both yarn and pnpm to address a years old problem that no amount of discussion in core has resolved up to this point. |
These are great points, and I think they are exactly the kind of things which should be nailed down as a part of the larger conversation. Hence my initial proposal to have a structured conversation in a smaller group with the subject matter experts like yourself. My concern is the current proposal exists, community projects exist, but Node's position is so unclear. I think a clear problem statement from the perspective of long term goals of Node would help a lot here.
Totally agree. I think the challenge of an issue on the node repo is getting engagement from the right parties who are both willing to engage and also willing to help build the solution. I actively avoid those more contentious issues, and I know many other folks do the same. The cost of engagement there is just TOO DAMN HIGH. Which is why I propose a smaller group, which I have seen work.
I very much want a solution which removes this natural monopoly. I agree this is a part of the problem, but I disagree that this solution is the right one.
Strong agree! I really don't like to be a dissenting voice, so I picked this issue rather carefully. An advanced solution to an unclear problem does not make sense for a project of this maturity IMO. I do think this is the wrong long term solution, but not because of poor decision making on the solution, because we do not have clarity on the problem. |
|
(I just realized TSC and CommComm were not pinged here, so cc @nodejs/tsc @nodejs/community-committee since this is a request to transfer a repository to the org) |
|
We may eventually get to a point where we can solve the larger problems. We may even decide that shipping corepack with the node.js binary just simply is not the right thing to do. But that is a completely separate question as to whether or not the corepack repo can move under the node.js organization. Doing so does not obligate us to any particular path moving forward and doing so helps to address one of the issues that has actually been raised in this discussion... namely, attracting additional contributors! So, let's do this: let's separate conversation threads.
Let's table 2 and 3 for now and focus on 1. There is nothing about moving the project in to the node.js organization that obligates us to any single long-term direction. |
|
Just to clarify... this issue is about adding this project under nodejs org? If so - then I think it should be done. It looks like an advanced project, that needs some incubation and experimentation, and we do have pkgjs org for just that, but at the same time - I don't think that PM WG needs to be the admin of all such incubation and experimentation projects. The whole point is that the project has maintainers and oversight (by TSC) - so what's the issue with just adding it to the org? Whether it lands in node and is distributed together, or whether it is the right way to approach the problem, or whether using TS is a hindrance to some users are entirely different discussions, that can be had between the maintainers/TSC and whoever else needs to be involved. Edit: I see James posted a similar thing while I was typing... |
👍 Will moving it into the node org mean others than the current IC's participate? If so, then I am on board. There are reasons I probably won't be an effective contributor (things brought up above among them) but if others will then I will withdrawal my opposition here and move it into that repo itself. |
|
@arcanis if you want to speed up the process, here's a checklist of files you should have before moving the repository to the org:
I see you already have a CoC, LICENSE and README. I suggest opening a PR to add a CONTRIBUTING.md and linking the CoC to the one we use on the Node.js org. Feel free to ping me on the PR to check if everything is ok. Also, that's not a requirement, but if you can change the default branch from |
|
I'm ok with the move of the repo, provided as @jasnell has stated that it's clear that it does not imply any agreement or consensus that it will become part of core, and it is in support of experimentation and the process of gaining consensus on next steps. |
|
@arcanis per our process this is now approved, you can transfer the repository to the |
|
@mmarchini Nice! I just tried but got "You don’t have the permission to create public repositories on nodejs" - do I need to do something special on my side? |
|
You got this error when trying to transfer the repo through settings? If that's the case give me admin permission to the repo and I'll transfer it (since I have write perm here). |
|
I transferred it to your account, so you should be able to re-transfer it to @nodejs 😄 |
|
lol |
The PR at nodejs/node#35398 adds a new dependency into the Node trunk: Corepack. Since once the PR has landed the expectation is that people will only use it through Node, I believe it would make sense to move it inside the Node organization.
Maintenance:
I would be ready to keep maintaining Corepack regardless the org it belongs to, especially given the low scope (cf next section), but it would let you have a better control on what the supported workflows and integrations.
Roadmap:
Corepack is expected to stay very simple, with a low amount of features. My expectation is that it's already almost feature-complete. As a result, the roadmap will mostly be to fix launch bugs, and keep the codebase in line with the Node deprecations.
The text was updated successfully, but these errors were encountered: