perf(gatsby-plugin-mdx): add lessBabel option, ~40% perf win

[pvdz]

This PR skips some Babel transformation steps during bootstrap in favor of less precise but very capable import/export scanning.

The mdx plugin needs to scan for the import and export symbols and paths. It is currently using Babel for this purpose. That works fine but is very very slow. We already replaced the import scanner with a regular expression approach a while back. But the export scanner had to be reverted because some sites were depending on the transformations.

So this is basically the same approach except it's tucked behind an opt-in flag. If you enable the flag, the plugin will use regular expressions to scan for export symbols and paths (which it already does for imports without flag). This will not work if you have custom syntax that affects export statements.

Currently the assumption is that any export in MDX is on a single line, or that the exported symbol is on the first line. This means that multi-line export const foo = ... is fine, since the scanner stops at the = sign. There are some limits but if you find them we can always try to support them. Just bring them to our attention.

Numbers will vary per site. I tried the very baseline gabe-fs-mdx site (available in this monorepo) at 100k pages.

Before this PR the sourcing step took 20 minutes (and before #27937 this was 50 minutes!). After this PR this step is 4 minutes!

This PR will add a lessBabel plugin option which, as you can imagine, will invoke Babel less to achieve the above projected performance improvement. At some point in the future we're likely going to make this the default, but for now we're going to look for feedback to improve the export scanner and fix forward.

Concrete numbers.

This is for gabe-fs-mdx (part of this monorepo) at 100k pages (N=100 M=8 yarn bench) on my machine under similar load:

Before #27937:

• bootstrap: 3083s
• page queries: 4818s
• total build time: 77951s

Before this PR:

• bootstrap: 1250s
• page queries: 3207s
• total build time: 4413s

After this PR:

• bootstrap: 293s
• page queries: 2929s
• total build time: 3273s

Related PRs and issues (may not be complete)

#25734
#25757
#25975
#26002
#26003
#26004
#26202

[mxstbr]

Once we get more feedback we can bump the major and flip the switch to make it opt-out instead. But that's future.

Love it! 👍 Thanks for tackling this 💯

[laurieontech]

Let's do it!