Cannot download file with non-ascii filename from remote source

[cometkim]

Preliminary Checks

• This issue is not a duplicate. Before opening a new issue, please search existing issues: https://github.com/gatsbyjs/gatsby/issues
• This issue is not a question, feature request, RFC, or anything other than a bug report directly related to Gatsby. Please post those things in GitHub Discussions: https://github.com/gatsbyjs/gatsby/discussions

Description

when I source file nodes via createRemoteFileNode.

The actual saved filename should be a decoded string. otherwise it becomes unavailable if source url is an encoded string.

For example, sourcing a file from a340d817-7e76-4dfd-a704-74b4a526d77c_테스트.txt, its publicURL and actual base on the filesystem is going to be a340d817-7e76-4dfd-a704-74b4a526d77c_%e1%84%90%e1%85%a6%e1%84%89%e1%85%b3%e1%84%90%e1%85%b3.txt

I think publicURL is fine, but filesystem path with encoded base is not useful because typical web servers cannot retrieve it from the publicURL.

Reproduction Link

https://github.com/cometkim/gatsby-remote-file-non-ascii

Steps to Reproduce

1. yarn install & start gatsby
2. Open http://localhost:8000/
3. Click "Can I download file" link

Expected Result

200 Ok, I can see the text content "테스트"

Actual Result

404 Not found.

Environment

System:
    OS: macOS 12.2.1
    CPU: (10) x64 Apple M1 Pro
    Shell: 5.8 - /bin/zsh
  Binaries:
    Node: 16.14.0 - /var/folders/z5/shv_7yrx1cg51gkbb8jh9x180000gn/T/yarn--1652323880604-0.0013071866350218286/node
    Yarn: 1.22.17 - /var/folders/z5/shv_7yrx1cg51gkbb8jh9x180000gn/T/yarn--1652323880604-0.0013071866350218286/yarn
    npm: 8.3.1 - ~/.asdf/plugins/nodejs/shims/npm
  Languages:
    Python: 2.7.18 - /usr/bin/python
  Browsers:
    Chrome: 101.0.4951.54
    Safari: 15.3
  npmPackages:
    gatsby: ^4.13.1 => 4.14.0 
    gatsby-plugin-gatsby-cloud: ^4.13.0 => 4.14.0 
    gatsby-plugin-image: ^2.13.0 => 2.14.0 
    gatsby-plugin-manifest: ^4.13.0 => 4.14.0 
    gatsby-plugin-offline: ^5.13.0 => 5.14.0 
    gatsby-plugin-react-helmet: ^5.13.0 => 5.14.0 
    gatsby-plugin-sharp: ^4.13.0 => 4.14.0 
    gatsby-source-filesystem: ^4.13.0 => 4.14.0 
    gatsby-transformer-sharp: ^4.13.0 => 4.14.0

Config Flags

No response

[Boronare]

change gatsby-source-wordpress/src/steps/source-nodes/reate-nodes/create-remote-file-node/index.js:449 to

  const fileDownloadPromise = pushTask({
    url,
    cache,
    createNode,
    parentNodeId,
    createNodeId,
    auth,
    httpHeaders,
    ext,
    name: decodeURI(name), // << HERE
    limit
  });

will fix the problem.

[Boronare]

Ah... sorry... I had same issue while using wordpress, so I modified like that. Is it from helmet? or so? find similar path may fix it I think.
----edit----
found.
around gatsby-source-filesystem/src/create-remote-file-node.js:56

  const filename = await fetchRemoteFile({
    url,
    cache,
    auth,
    httpHeaders,
    ext,
    name: decodeURI(name), // << HERE
  })

[Boronare]

To get rid of the more fundamental problem
It'll be better to change gatsby-core-utils/src/fetch-remote-file.ts:176

      const downloadPath = createFilePath(finalDirectory, decodeURI(name), ext)

By the way, gatsby-source-wordpress doesn't refer core-utils but use almost same as its code instead...

[cometkim]

@Boronare can you confirm that #35 fixes your problem as well?

[Boronare]

@cometkim Well, this approach is opposite to my first approach, if there's 'name' param, getRemoteFileName won't be called.

[cometkim]

Ok I see, that a good catch.