implement tool for publishing packages from a mono-repo #1

kevinbarabash · 2018-06-17T22:12:12Z

The reason for creating this tool is that lerna does a couple of things things which we'd like to avoid with publishing packages from the wonder-blocks mono-repo:

making changes to a dependency will cause lerna to automatically bump the version of packages that depend on it… this feels fundamentally wrong since the dependee may not be ready to use the updated version of the dependency. Also, if the dependee is using version ^1.0.0 and the dependency is bumped to 1.1.0, then there's no reason the update the dependee's version.
right now lerna is responsible for bumping versions (as well as publishing) which means that you have to commit changes without bumping versions. In repos that publish a single package this is fine b/c other packages can’t rely on in between states. This is different in mono-repos b/c you may change the version of a dependency in the same commit that you change a package to rely on that new version. This means that we should be bumping versions in the same commit that we’re making this change. This avoids having commits in the repo that you can never checkout and run b/c of an inconsistent state. The publish (and tagging) can and should be done in a separate commit.

TODO:

set up travis-ci
push the created tags to GitHub after publishing

Test Plan:

yarn test
npm link
checkout https://github.com/kevinbarabash/workspaces-test
bump versions of foo and bar packages in workspaces-test and commit changes
run antwerp --dry-run from workspaces-test folder
see that the output contains commands to tag the most recent commit with @kevinbarabash/foo@0.1.10 and @kevinbarabash/bar@0.1.11 and publish @kevinbarabash/foo and @kevinbarabash/bar packages. Notice that it doesn't try to bump or publish @kevinbarabash/foobar which relies on version ^0.1.0 of both of those packages.
run antwerp again without the --dry-run flag and verify that it's published those packages to https://npmjs.org/~kevinbarabash and that the most recent commit has indeed been tagged appropriately.

somewhatabstract

I haven't looked at the exact implementation yet. I'll do that when I get to the AirBnB.

somewhatabstract · 2018-06-17T23:56:50Z

fixtures/public-dep-public-pkg/packages/public-pkg/package.json

+    "dependencies": {
+        "@fixture/public-dep": "^0.0.1"
+    }
+}


Love all these test fixtures. Is @fixture a reserved namespace or just something you chose to use just for testing?

It's just something I made up fro testing... nothing actually gets published as part of the test tests.

somewhatabstract · 2018-06-17T23:58:27Z

index.js

+        } else {
+            console.log(`would run: ${command}`);
+        }
+    }


No exit code for a successful run?

I'm going to add a process.exit(0); but it has to come right at the end of the try block.

Or after the catch?

somewhatabstract · 2018-06-17T23:59:18Z

package.json

@@ -0,0 +1,22 @@
+{
+  "name": "antwerp",


I like it, but why antwerp?

Naming things is hard. I started out with pubtools but then I thought I'd change it to pubtool since this is really only one tool, but that's already taken. I started thinking about pubs I've been in in videos games and there was one in Hero's Quest I (aka Quest for Glory I) where you have to say a secret password to the goon guarding the entrance to the thieves' guild. One of the passwords was antwerp. I'm open to better names as this name is not very descriptive and only tangentially related to the task that the tool performs.

Also, antwerp is the name of this creature in the game:

In my journey through Benelux in the spring I visited Antwerp the city which has this really old wooden escalator which leads to a tunnel that you can walk, bike, motorbike through. Unsurprisingly it smells really woody. If you're ever in Antwerp it's a must see... along with the MAS.

somewhatabstract · 2018-06-17T23:59:32Z

package.json

+{
+  "name": "antwerp",
+  "version": "0.0.1",
+  "description": "",


Definitely needs a description! 😁

somewhatabstract · 2018-06-18T00:00:50Z

src/__test__/utils.test.js

+            "npm publish --access=public packages/public-dep",
+            "git tag @fixture/public-pkg@0.0.1",
+            "npm publish --access=public packages/public-pkg",
+        ]);


I like the ease with which we can test this without necessarily having to run it fully although there's no substitute for actually running and checking those outputted commands really work.

It's tough b/c there isn't a good way to mock out npm. At least with git I can run commands locally and verify things about the state of the local repo.

somewhatabstract · 2018-06-18T00:01:21Z

src/utils.js

+
+// We assume that if a there's a tag for the current version of a package that
+// it's been published already.  In the future we should check if that version
+// exists on npm and attempt to republish the package if it isn't.


Capture that in an issue.

Captured in #3.

I can update the comment to link back to the GitHub issue.

somewhatabstract · 2018-06-18T00:02:12Z

src/utils.js

+            // Check that depedency versions are satisified
+            // TODO: fallback to check if any tags exist that would satisfy the dep
+            if (!semver.satisfies(pkgVerMap[depName], deps[depName])) {
+                throw new Error(`${depName}@${pkgVerMap[depName]} doesn't satify ${depName}@${deps[depName]}`)


NIT: "satify" should be "satisfy"

somewhatabstract · 2018-06-18T00:02:27Z

src/utils.js

+                throw new Error(`${depName}@${pkgVerMap[depName]} doesn't satify ${depName}@${deps[depName]}`)
+            }    
+
+            // Check to see if any depedencies of public packages are private


NIT: "depedencies" should be "dependencies"

somewhatabstract · 2018-06-18T22:28:04Z

index.js

+        child_process.execSync("git push --tags");
+    } else {
+        console.log(`would run: git push --tags`);
+    }


Feel like the text git push --tags should be in a constant, and then there should be an maybeRunCommand command that takes that string like this:

function (command, description) { console.log(description); const isDryRun = argv["dry-run"]; if (isDryRun) { console.log(`would run: ${command}`); } else { child_process.execSync(command); } } That way we can reuse this for any command.

somewhatabstract · 2018-06-18T22:30:31Z

src/utils.js

+const semver = require("semver");
+
+function getTags() {
+    return child_process.execSync(`git tag`, {encoding: "utf8"}).split("\n").filter(Boolean);


I find these chains easier to read if they are lined up by the dots.

return child_process .execSync... .split... .filter...

YMMV

somewhatabstract · 2018-06-18T22:30:54Z

src/utils.js

+function getWorkspaces() {
+    const pkgJson = JSON.parse(fs.readFileSync("package.json", "utf8"));
+    return pkgJson.workspaces.reduce((result, wsGlob) => {
+        return result.concat(glob.sync(wsGlob));


braces and return are redundant.

somewhatabstract · 2018-06-18T22:34:04Z

src/utils.js

+            // bumps the version automatically
+            commands.push(`npm publish --access=public ${workspace}`);
+        }
+    }


So this getCommands call is actually doing some useful stuff. I wonder if it is clear from the name.

Perhaps generateCommandList or something would be better?

Certainly should add a comment that explains what it's doing.

Returns command list for tagging and publishing all public packages

somewhatabstract · 2018-06-18T22:35:50Z

src/__test__/utils.test.js

+    const newVersion = semver.inc(json.version, release);
+    json.version = newVersion;
+    fs.writeFileSync(pkgJsonPath, JSON.stringify(json, null, 4));
+}


So this is the step that we don't do with antwerp. antwerp assumes you've done all the version changes that you need to and then just ensure that things are tagged with those versions before publish, after asserting that the dependencies are available for those packages.

Is that right?

somewhatabstract · 2018-06-18T22:36:53Z

src/utils.js

+        pkgVerMap[pkgJson.name] = pkgJson.version;
+        depMap[pkgJson.name] = pkgJson.dependencies || {};
+        pkgJsonMap[pkgJson.name] = pkgJson;
+    }


So, this loop is ensuring that even though the dependency may not exist yet (because it's on a version of the package that isn't released), it will when antwerp is finished. Right?

Can we get this loop in its own method to make that a little clearer?

somewhatabstract · 2018-06-18T22:37:24Z

src/utils.js

+            }
+        }
+    }
+}


Also, this needs to be its own function too, I think. Either the whole loop or its contents.

somewhatabstract · 2018-06-18T22:38:02Z

src/utils.js

+    getTags,
+    getCommands,
+    getWorkspaces,
+};


Any thoughts on supporting use-cases that want to consume this as an API versus a tool?

somewhatabstract

I think there's a little refactoring to do just to get things clearer.

kevinbarabash · 2018-06-22T16:30:32Z

We have a new workflow that we're going to try with lerna so I'm going to close this for now.

implement tool for publishing packages from a mono-repo

a4d2a64

kevinbarabash requested review from jeresig, somewhatabstract and MiguelCastillo June 17, 2018 22:12

kevinbarabash added 3 commits June 17, 2018 18:21

setup travis

26dbf8f

push tags after publishing

6d850ef

don't publish private packages

5f7a9af

somewhatabstract reviewed Jun 18, 2018

View reviewed changes

address feedback from code review

0cb521d

somewhatabstract reviewed Jun 18, 2018

View reviewed changes

src/utils.js

}

}

}

}

Copy link

Member

somewhatabstract Jun 18, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, this needs to be its own function too, I think. Either the whole loop or its contents.

somewhatabstract reviewed Jun 18, 2018

View reviewed changes

somewhatabstract suggested changes Jun 18, 2018

View reviewed changes

jeresig assigned kevinbarabash Jun 20, 2018

kevinbarabash closed this Jun 22, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

implement tool for publishing packages from a mono-repo #1

implement tool for publishing packages from a mono-repo #1

kevinbarabash commented Jun 17, 2018 •

edited by somewhatabstract

somewhatabstract left a comment

somewhatabstract Jun 17, 2018

kevinbarabash Jun 18, 2018

somewhatabstract Jun 17, 2018

kevinbarabash Jun 18, 2018

somewhatabstract Jun 18, 2018

somewhatabstract Jun 17, 2018

kevinbarabash Jun 18, 2018

kevinbarabash Jun 18, 2018

somewhatabstract Jun 17, 2018

somewhatabstract Jun 18, 2018

kevinbarabash Jun 18, 2018

somewhatabstract Jun 18, 2018

kevinbarabash Jun 18, 2018

kevinbarabash Jun 18, 2018

somewhatabstract Jun 18, 2018

somewhatabstract Jun 18, 2018

somewhatabstract Jun 18, 2018 •

edited

somewhatabstract Jun 18, 2018

somewhatabstract Jun 18, 2018

somewhatabstract Jun 18, 2018

somewhatabstract Jun 18, 2018

somewhatabstract Jun 18, 2018

somewhatabstract Jun 18, 2018

somewhatabstract Jun 18, 2018

somewhatabstract left a comment

kevinbarabash commented Jun 22, 2018

implement tool for publishing packages from a mono-repo #1

implement tool for publishing packages from a mono-repo #1

Conversation

kevinbarabash commented Jun 17, 2018 • edited by somewhatabstract

somewhatabstract left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

somewhatabstract Jun 18, 2018 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

somewhatabstract left a comment

Choose a reason for hiding this comment

kevinbarabash commented Jun 22, 2018

kevinbarabash commented Jun 17, 2018 •

edited by somewhatabstract

somewhatabstract Jun 18, 2018 •

edited