Gatsby transform sharp throws intermittent errors

[seungpark]

Preliminary Checks

• This issue is not a duplicate. Before opening a new issue, please search existing issues: https://github.com/gatsbyjs/gatsby/issues
• This issue is not a question, feature request, RFC, or anything other than a bug report directly related to Gatsby. Please post those things in GitHub Discussions: https://github.com/gatsbyjs/gatsby/discussions

Description

Gatsby transformer sharp throws intermittent errors during the build schema process:

 ERROR #11321  API.NODE.EXECUTION

"gatsby-transformer-sharp" threw an error while running the onCreateNode
lifecycle:

Cannot set properties of undefined (setting 'dirty')

TypeError Cannot set properties of undefined (setting 'dirty')
incrementalReducer
[gatsby]/src/redux/reducers/inference-metadata.ts:95:24
inference-metadata.ts
inferenceMetadataReducer [gatsby]/src/redux/reducers/inference-metadata.ts:140:23
 redux.js:568 combination
  [snooty]/[redux]/lib/redux.js:568:29
 redux.js:296 next
  [snooty]/[redux]/lib/redux.js:296:22
 index.ts:72 
  [snooty]/[gatsby]/src/redux/index.ts:72:68
 index.js:27 
  [snooty]/[redux-thunk]/lib/index.js:27:16
 redux.js:586 boundActionCreator
  [snooty]/[redux]/lib/redux.js:586:12
 api-runner-node.js:104 doubleBoundActionCreators.<computed>
  [snooty]/[gatsby]/src/utils/api-runner-node.js:104:20
 on-node-create.js:40 Object.onCreateNode
  [snooty]/[gatsby-transformer-sharp]/on-node-create.js:40:3
 api-runner-node.js:487 runAPI
  [snooty]/[gatsby]/src/utils/api-runner-node.js:487:37


�[2K�[1A�[2K�[Gnot finished building schema - 0.453s

Accompanying gatsby-config plugins:

const plugins = [
  `gatsby-plugin-image`,
  `gatsby-plugin-sharp`,
  {
    resolve: 'gatsby-source-filesystem',
    options: {
      name: 'images',
      path: `${__dirname}/src/images`,
    },
  },
  `gatsby-transformer-sharp`, // Needed for dynamic images
  'gatsby-plugin-emotion',
];

In gatsby-node.js:

const saveFiles = async (assets) => (
   Promise.all(<save asset promise to src/images>)
);

exports.sourceNodes = async ({ actions, createContentDigest, createNodeId }) => {
   ...
   await saveFiles(assets);
   ...
})

This error is not consistent, and it seems to be complaining about a ADD_CHILD_TO_NODE action.type case, at this line.

However, when debugging the build locally, the build always succeeds and I am able to see the ImageSharp nodes (as children of File nodes) in the graphql explorer.

Reproduction Link

https://github.com/seungpark/gatsby-reproduction

Steps to Reproduce

1. Define the gatsby config as outlined in above
2. Have an asynchronous process that saves image files to the directory that is sourced from the gatsby-source-filesystem plugin
3. Run gatsby build and look for error while in step source and transform nodes
   ...

Expected Result

I would expect this to be an error or success 100% of the time. Intermittent errors have been a pain for debugging.
Ideally, this would be a successful Image Sharp node creation for each Image type file.

Actual Result

This error is thrown sometimes on the same build, but not 100%

Environment

  System:
    OS: Linux 5.10 Ubuntu 20.04.6 LTS (Focal Fossa)
    CPU: (4) x64 Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
    Shell: 5.0.17 - /bin/bash
  Binaries:
    Node: 18.19.0 - /usr/bin/node
    npm: 8.19.4 - /usr/bin/npm
  npmPackages:
    gatsby: ^5.0.0 => 5.7.0
    gatsby-plugin-emotion: ^8.0.0 => 8.7.0
    gatsby-plugin-google-tagmanager: ^5.0.0 => 5.7.0
    gatsby-plugin-image: ^3.13.0 => 3.13.0
    gatsby-plugin-layout: ^4.7.0 => 4.7.0
    gatsby-plugin-sharp: ^5.13.0 => 5.13.0
    gatsby-plugin-sitemap: ^6.0.0 => 6.7.0
    gatsby-source-filesystem: ^5.13.0 => 5.13.0
    gatsby-transformer-sharp: github:seungpark/gatsby-transformer-sharp =>5.13.0

Config Flags

No response

[olehpratsko]

Hey @seungpark ,
Your assets download script is not catching up with Gatsby nodes API at all. Use createRemoteFileNode instead.
https://www.gatsbyjs.com/plugins/gatsby-source-filesystem/#createremotefilenode

exports.sourceNodes = async ({ actions, createNodeId, createContentDigest }) => {
  const { createNode } = actions;
  ...

    for (const image of remoteImages) {
      try {
        const fileNode = await createRemoteFileNode({
          url: image.url, // string that points to the URL of the image
          parentNodeId: node.id, // id of the parent node of the fileNode you are going to create
          createNode, // helper function in gatsby-node to generate the node
          createNodeId, // helper function in gatsby-node to generate the node id
          getCache,
        });
      } catch (error) {
        console.log(error);
      }
    }

remoteImages is just array of image urls

[seungpark]

Hey @seungpark , Your assets download script is not catching up with Gatsby nodes API at all. Use createRemoteFileNode instead. https://www.gatsbyjs.com/plugins/gatsby-source-filesystem/#createremotefilenode

exports.sourceNodes = async ({ actions, createNodeId, createContentDigest }) => {
  const { createNode } = actions;
  ...

    for (const image of remoteImages) {
      try {
        const fileNode = await createRemoteFileNode({
          url: image.url, // string that points to the URL of the image
          parentNodeId: node.id, // id of the parent node of the fileNode you are going to create
          createNode, // helper function in gatsby-node to generate the node
          createNodeId, // helper function in gatsby-node to generate the node id
          getCache,
        });
      } catch (error) {
        console.log(error);
      }
    }

remoteImages is just array of image urls

@olehpratsko instead of creating my own gatsby nodes, i am using gatsby filesystem to create image gatsby nodes of files where i save the assets

[olehpratsko]

@seungpark good workaround, I'll take it to the note. Thank you for sharing.