perf(gatsby-plugin-mdx): lazily fetch file nodes for remark plugins

[pvdz]

Lazily fetch the list of File nodes. Before it would always generate the list of file nodes (which does a shallow copy, which is expensive at scale). Afterwards it only does this when the files are actually read, by adding a "getter" property.

This is roughly a 45% reduction of total build time for baseline MDX sites!

Benchmark numbers on gabe-fs-mdx (no plugins), for just 10k pages:

info bootstrap finished - 110.731s
success run page queries - 176.578s - 10002/10002 56.64/s
info Done building in 297.347 sec

to

info bootstrap finished - 96.578s
success run page queries - 159.652s - 10002/10002 62.65/s
info Done building in 266.202 sec

100k page build:

Before:

• bootstrap: 3083s
• page queries: 4818s
• total build time: 77951s

After:

• bootstrap: 1250s
• page queries: 3207s
• total build time: 4413s

[vladar]

Nicely done!

One concern is that it may break in some edge case (I am not that familiar with getters ergonomics, so this concern is literally from the lack of experience working with those). But from my brief tests I couldn't find a way to make it break, so 👍

[pvdz]

Getters have been in JS since ES5 and are very safe to use from compat standpoint.

The real danger is that, potentially, plugins don't cache the access to api.files or something, meaning they'd do a repetitive fetch every time. But caching it is also not safe, for example it might break gatsby-dev when new files are sourced.

I think this is an acceptable middle ground, also because I'm not sure whether anything actually uses this at all. I couldn't find this property described in the docs either. Worst case this is a perf regression for a plugin that does use it. But it shouldn't break it.