Meeting Notes 2021-09-21

[afoxman]

Meeting Notes

Thank you for meeting and sharing your ideas and work! Below are my notes from the meeting. There were lots of details flying around, so there's a chance I missed something or misunderstood. Please reply with corrections/comments.

How we'll communicate

• E-mail for broad announcements
• GitHub discussion for agenda, meeting notes, and sharing ideas between meetings
• Discord for more direct communication. Please bring any interesting learning to the next group meeting!

I'll record monthly meetings and post them for all members of the working group.

Metro Module IDs

Currently numeric. Can change between builds.

AirBNB needs stability between builds. They've updated a private fork of Metro to support using a hash.
Facebook (Hermes) also needs stability, but they like the optimizations they get from using a range of nubmers from 0-N. Internally, they associate modules with their IDs, and use that to inform module assignment in future builds (where possible).

AirBNB is interested in upstreaming their work to Metro, and Facebook is on-board with that so long as numeric IDs are still available.

Developer pain points

Expo:

• Metro can be slow to start up, no loading indicator
• Webpack can start faster and can bundle faster
• Timings vary based on a number of factors
• Giving devs choice by letting them choose a bundler, while still providing an over-arching system in which it runs

Facebook:

• Watchman helps with monorepo scale issues, but still bogs down on first crawl
• Looking at the problem, but focused on internal fix
  • Save state as part of CI commit job, including Metro state
  • Load state on next CI run to pre-populate cache and avoid full crawl
  • In the design/idea-phase. No plans to open-source due to dependency on "save state" CI infra.

Q: Has anyone experimented with Metro to only watch files that are "require()"d (avoiding full FS-crawl)? Or chunked-up the crawl?

Metro uses a haste-map which needs a full crawl to discover modules.
Metro just recently added support for replacing the default haste-map with a custom implementation
Could try replacing with a "dumb" map that hits the file-system to fulfill each request
Could also integrate with more complex build systems which have their own cache

Metro now allows a custom haste-map implementation, which opens up areas for experimentation to address perf issues around FS crawling/caching.

Metro remote caching

Facebook: All the pieces are there, we just need a guide for developers to make use of them. Show them how to set up an S3 bucket, for example.

AirBNB: Already implemented this exact solution. Seeing major gains during dependency-graph computation, in particular. Down from minutes to under a minute.

Opportunity here to open-source the docs, if not the code for an S3 bucket, and have a big impact.

esbuild

Expo:

• Lots of experiments replacing some/all of Metro
• Mixed results

Microsoft:

• React-native (@rnx-kit)
  • Tried replacing transformer. Metro API wasn't compatible. Also tried SWC, but currently not great and lacking plugins.
  • Replaced serializer. Big wins from tree-shaking - smaller bundles. Also runs much faster (10x).
• Web
  • Exploring esbuild to address 10+ minute dev-server startup, among other things.
  • Result: potentially useful for inner loops, but there are blocking issues on using for production.
    • Minify sizes are larger
    • Code-splitting results in way more tiny files without a lot of control on chunking things together

A promising area for continued experimentation, including other native tools like SWC as they mature.

Summary

Metro Module IDs: AirBNB may open-source their hash-based solution. Facebook would support it.

Haste-maps and crawling the file-system: Metro config supports a custom haste-map implementation. Opens up experiments for possible haste-map / watchman replacement.

esbuild: Experiments ongoing. Expo and Microsoft working in this area for both react-native and web apps. AirBNB interested. Other native tools like SWC aren't there yet but could change things yet again.

[jondlm]

Hi there! I'm Jon, an engineer at Stripe working on Metro stuff and I'm keen to talk about this stuff with other folks working in the same space. If you all are open to it I'd like to participate here and join meetings if they're open.

Email: jdlm at stripe dot com
Discord: jdlm-stripe#3795

For context we are pretty far along the path to running Metro as a dev time bundler in one of our largest frontend codebases. Early results are showing substantial speed improvements over our existing Webpack 4 setup.

Thanks!
// Jon

[afoxman]

Welcome, Jon! Meetings are open to everyone in the group - I'll send you the invite. I'd like to hear more about the project!

You can reach me at: afoxman at microsoft dot com and as afoxman#4801 on discord.

[gwu-stripe]

Grant here—I work with Jon at Stripe. I'd also love an invite, @afoxman !