gatsby-plugin-sitemap: TypeError: Cannot use 'in' operator to search for 'nodes' in undefined

[janosh]

Description

plugin-sitemap crashes build:

⠹ onPostBuild

 ERROR #11321  PLUGIN

"gatsby-plugin-sitemap" threw an error while running the onPostBuild lifecycle:

Cannot use 'in' operator to search for 'nodes' in undefined

  40 |           case 9:
  41 |             queryRecords = _context.sent;
> 42 |             filteredRecords = (0, _internals.filterQuery)(queryRecords, excludeOptions, basePath, resolveSiteUrl);
     |                                                          ^
  43 |             urls = serialize(filteredRecords);
  44 | 
  45 |             if (!(!rest.sitemapSize || urls.length <= rest.sitemapSize)) {

File: node_modules/gatsby-plugin-sitemap/gatsby-node.js:42:58



  TypeError: Cannot use 'in' operator to search for 'nodes' in undefined
  
  - internals.js:113 getNodes
    [blog]/[gatsby-plugin-sitemap]/internals.js:113:15
  
  - internals.js:46 filterQuery
    [blog]/[gatsby-plugin-sitemap]/internals.js:46:19
  
  - gatsby-node.js:42 _callee$
    [blog]/[gatsby-plugin-sitemap]/gatsby-node.js:42:58
  
  - task_queues.js:97 processTicksAndRejections
    internal/process/task_queues.js:97:5

Steps to reproduce

My config (see in repo)

{
  resolve: `gatsby-plugin-sitemap`,
  options: {
    output: `/sitemap.xml`,
    query: `{
      site {
        meta: siteMetadata {
          url
        }
      }
      pages: allSitePage {
        nodes {
          path
        }
      }
    }`,
    resolveSiteUrl: ({ site }) => site.meta.url,
    serialize: ({ site, pages }) =>
      pages.nodes.map(node => ({ url: site.meta.url + node.path })),
  }
  ,
},

Environment

  System:
    OS: macOS 10.15.5
    CPU: (16) x64 Intel(R) Core(TM) i9-9880H CPU @ 2.30GHz
    Shell: 5.7.1 - /bin/zsh
  Binaries:
    Node: 14.4.0 - /usr/local/bin/node
    Yarn: 1.22.4 - /usr/local/bin/yarn
    npm: 6.14.4 - /usr/local/bin/npm
  Languages:
    Python: 2.7.16 - /usr/bin/python
  Browsers:
    Chrome: 83.0.4103.97
    Firefox: 77.0.1
    Safari: 13.1.1
  npmPackages:
    gatsby: 2.23.1 => 2.23.1 
    gatsby-image: 2.4.6 => 2.4.6 
    gatsby-plugin-algolia: ^0.11.1 => 0.11.1 
    gatsby-plugin-catch-links: 2.3.4 => 2.3.4 
    gatsby-plugin-favicon: ^3.1.6 => 3.1.6 
    gatsby-plugin-feed: ^2.5.4 => 2.5.4 
    gatsby-plugin-google-analytics: 2.3.3 => 2.3.3 
    gatsby-plugin-lodash: ^3.3.3 => 3.3.3 
    gatsby-plugin-manifest: 2.4.10 => 2.4.10 
    gatsby-plugin-mdx: ^1.2.14 => 1.2.14 
    gatsby-plugin-netlify-cache: ^1.2.0 => 1.2.0 
    gatsby-plugin-offline: 3.2.8 => 3.2.8 
    gatsby-plugin-react-helmet: 3.3.3 => 3.3.3 
    gatsby-plugin-sharp: 2.6.10 => 2.6.10 
    gatsby-plugin-sitemap: ^2.4.5 => 2.4.5 
    gatsby-plugin-styled-components: ^3.3.3 => 3.3.3 
    gatsby-remark-autolink-headers: ^2.3.4 => 2.3.4 
    gatsby-remark-code-titles: ^1.1.0 => 1.1.0 
    gatsby-remark-copy-linked-files: ^2.3.4 => 2.3.4 
    gatsby-remark-embed-video: 3.0.10 => 3.0.10 
    gatsby-remark-emojis: ^0.4.3 => 0.4.3 
    gatsby-remark-images: 3.3.9 => 3.3.9 
    gatsby-remark-katex: ^3.3.3 => 3.3.3 
    gatsby-remark-responsive-iframe: 2.4.4 => 2.4.4 
    gatsby-remark-smartypants: 2.3.3 => 2.3.3 
    gatsby-remark-sub-sup: ^1.0.0 => 1.0.0 
    gatsby-remark-vscode: ^2.1.1 => 2.1.1 
    gatsby-source-filesystem: 2.3.10 => 2.3.10 
    gatsby-transformer-sharp: 2.5.4 => 2.5.4 
    gatsby-transformer-yaml: 2.4.3 => 2.4.3 

[vladar]

Thank you for opening this, @janosh

I think it happens because you've aliased allSitePage in your query. gatsby-plugin-sitemap relies on this key to build the site map:

gatsby/packages/gatsby-plugin-sitemap/src/internals.js

Lines 23 to 25 in 670f92f

	const { allSitePage, ...otherData } = data
	let { allPages, originalType } = getNodes(allSitePage)

And it is actually mentioned in the plugin docs:

Due to how this plugin was built it is currently expected/required to fetch the page paths from allSitePage, but you may use the allSitePage.edges.node or allSitePage.nodes query structure.

So the quick fix for you is to change pages: allSitePage back to allSitePage.

We're marking this issue as answered and closing it for now but please feel free to reopen this and comment if you would like to continue this discussion. We hope we managed to help and thank you for using Gatsby! 💜

[janosh]

@vladar Good catch! Should have noticed that.

But now I'm getting

ERROR #11321  PLUGIN

"gatsby-plugin-sitemap" threw an error while running the onPostBuild lifecycle:

Error: Protocol is required

with the following config

options: {
  output: `/sitemap.xml`,
  query: `{
    site {
      siteMetadata {
        url
      }
    }
    allSitePage {
      nodes {
        path
      }
    }
  }`,
  resolveSiteUrl: ({ site }) => site.siteMetadata.url,
  serialize: ({ site, allSitePage }) =>
    allSitePage.nodes.map(node => ({ url: site.siteMetadata.url + node.path })),
}

Both error messages weren't that helpful so far.

[vladar]

Seems similar to #12912

[vladar]

Oh well, this seems weird for me but this plugin does this:

gatsby/packages/gatsby-plugin-sitemap/src/internals.js

Lines 57 to 69 in 6a9ffcc

	return {
	...otherData,
	allSitePage: {
	[originalType]:
	originalType === `nodes`
	? allPages
	: allPages.map(page => {
	return { node: page }
	}),
	},
	site: { siteMetadata: { siteUrl } },
	}
	}

as you see it forces url key to be siteUrl. In your case it is just url. I assume if you change it in the gatsby-config from siteMetadata.url to siteMetadata.siteUrl it should work.

But I do agree that this is super confusing. Feel free to submit a PR with additional validation that will provide better error messages in these cases!

[janosh]

But I do agree that this is super confusing. Feel free to submit a PR with additional validation that will provide better error messages in these cases!

I was just about to say that. Why have a resolveSiteUrl option if the plugin throws anyway as soon as the site's url is named anything other than siteUrl.

[vladar]

@moonmeister Do you have any context on this? (sorry if tagging inappropriately)

[janosh]

I think exports.onPostBuild is doing things in the wrong order: first excluding things matching exclude and then serializing the results.

gatsby/packages/gatsby-plugin-sitemap/src/gatsby-node.js

Lines 40 to 60 in 6a9ffcc

	const filteredRecords = filterQuery(
	queryRecords,
	excludeOptions,
	basePath,
	resolveSiteUrl
	)
	const urls = serialize(filteredRecords)
	if (!rest.sitemapSize || urls.length <= rest.sitemapSize) {
	const map = sitemap.createSitemap(rest)
	urls.forEach(u => map.add(u))
	return writeFile(saved, map.toString())
	}
	const {
	site: {
	siteMetadata: { siteUrl },
	},
	} = filteredRecords
	return new Promise(resolve => {

If you do it this way around you first have to make all sorts of assumptions about the structure of the data returned by query during the exclusion. Is there any reason not to serialize first and then perform the exclusion on data in a normalized format? If not I'd happy to submit a PR to reverse the order.

[vladar]

Maybe because exclude relies on page.path which is unavailable after serialization. So we'll have to do some data juggling anyway? But I have zero context on this plugin, so just guessing.

[janosh]

But shouldn't page.path be part of the final url in any serialization function? The default is

serialize: ({ site, allSitePage }) =>
  allSitePage.nodes.map(node => {
    return {
      url: `${site.wp.generalSettings.siteUrl}${node.path}`,
      changefreq: `daily`,
      priority: 0.7,
    }
  })
}

and mine

serialize: ({ site, allSitePage }) =>
  allSitePage.nodes.map(node => ({ url: site.siteMetadata.url + node.path })),

So you should be able to exclude based on the same patterns before and after serialization.

[moonmeister]

Looks like you all discovered how internally trashed this plugin is. I ran into some of the issues you all have and did some work to try to resolve them (hence the docs note you found and the fact that you're not required to use allSitePage.edges.node but can use allSitePage.nodes)...but in doing that work i realized the whole thing needs redoing to fix some bigger issues you all have also discovered.

There is an open RFC to fix these issues that needs finalizing, please catch up on that discussion and add you're feedback, it'd be much appreciated. If we can get the RFC finalized the work shouldn't take to much effort.

[moonmeister]

Here's the PR where I tried to fix some issues, for context: #21948

[github-actions]

Hiya!

This issue has gone quiet. Spooky quiet. 👻

We get a lot of issues, so we currently close issues after 30 days of inactivity. It’s been at least 20 days since the last update here.
If we missed this issue or if you want to keep it open, please reply here. You can also add the label "not stale" to keep this issue open!
As a friendly reminder: the best way to see this issue, or any other, fixed is to open a Pull Request. Check out gatsby.dev/contribute for more information about opening PRs, triaging issues, and contributing!

Thanks for being a part of the Gatsby community! 💪💜