gatsby-source-contentful Rich Text Invalid string length

[nadiavanleur]

Description

TL;DR

I added a lot of content and started to use Contentful's links to entries when I started getting this error:

Invalid string length

  178 |   });
  179 |   contentTypeItems.forEach((contentTypeItem, i) => {
> 180 |     normalize.createNodesForContentType({
      |               ^
  181 |       contentTypeItem,
  182 |       contentTypeItems,
  183 |       restrictedNodeFields,

File: node_modules/gatsby-source-contentful/gatsby-node.js:180:15

---

My project was going absolutely fine until I added a lot more content to Contentful. I also started to use @contentful/rich-text-react-renderer, which I think might be relevant. When I tried to run npm run dev or npm run build after adding the content I got a JavaScript heap out of memory error. I fixed this by using export NODE_OPTIONS=--max_old_space_size=4096 in the command line. Then my current problem occured.

Things I have tried to fix this:

• Remove exports.createSchemaCustomization = ({...}) from gatsby-node.js
• Don't request json data from rich text from the CMS (Contentful)
• Make more memory available for Node
• Update my Gatsby version, I thought my problem was related to #17233

UPDATE

I added a bunch of console.log(...)s to the function the error says the Invalid string length message comes from (gatsby-source-contentful/normalize.js > createNodesForContentType()) and have found the error keeps coming back when looping through entryItemFields. The build stops working specifically when a fieldType is RichText. The Invalid string length error then gets thrown. However, it doesn't always do this, the first couple of times there is no error and it just continues.

The program seems to stop working on line 473: const richTextNode = prepareRichTextNode(entryNode, entryItemFieldKey, entryItemFields[entryItemFieldKey], createNodeId); of gatsby-source-contentful/normalize.js

My guess is some of the serialized rich text objects are too large to be handled.

UPDATE 2

I removed all Rich Text fields from my Contentful CMS and the problem seems to be gone. Therefore my best guess is still some of the serialized rich text objects are too large to be handled.

Steps to reproduce

• Start a gatsby project with "gatsby-starter-default"
• Install "gatsby-source-contentful"
• Personalize gatsby-node.js with exports.createPages({...}) and exports.createSchemaCustomization = ({...})
• Use rich text in contentful with inline links to entries (@contentful/rich-text-react-renderer)
• Add about 280 entries to Contentful
• Run gatsby develop or gatsby build

Expected result

I expected my build to run as usual.

Actual result

> gatsby-starter-default@0.1.0 bs /var/www/current
> npm run build && npm run serve


> gatsby-starter-default@0.1.0 build /var/www/current
> gatsby build

success open and validate gatsby-configs - 0.060s
⠋ load plugins
success load plugins - 0.817s
success onPreInit - 0.026s
success delete html and css files from previous builds - 0.033s
success initialize cache - 0.034s
success copy gatsby files - 0.069s
success onPreBootstrap - 0.013s
success createSchemaCustomization - 0.023s
Starting to fetch data from Contentful
Fetching default locale
default locale is : nl-NL
contentTypes fetched 11
Updated entries  167
Deleted entries  0
Updated assets  98
Deleted assets  0
Fetch Contentful data: 1.320s

 ERROR #11321  PLUGIN

"gatsby-source-contentful" threw an error while running the sourceNodes lifecycle:

Invalid string length

  178 |   });
  179 |   contentTypeItems.forEach((contentTypeItem, i) => {
> 180 |     normalize.createNodesForContentType({
      |               ^
  181 |       contentTypeItem,
  182 |       contentTypeItems,
  183 |       restrictedNodeFields,

File: node_modules/gatsby-source-contentful/gatsby-node.js:180:15

not finished source and transform nodes - 234.827s
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! gatsby-starter-default@0.1.0 build: `gatsby build`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the gatsby-starter-default@0.1.0 build script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/esites/.npm/_logs/2020-03-04T10_26_33_867Z-debug.log
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! gatsby-starter-default@0.1.0 bs: `npm run build && npm run serve`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the gatsby-starter-default@0.1.0 bs script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/esites/.npm/_logs/2020-03-04T10_26_33_896Z-debug.log

Environment

System:
    OS: macOS Mojave 10.14.6
    CPU: (8) x64 Intel(R) Core(TM) i7-4770HQ CPU @ 2.20GHz
    Shell: 3.2.57 - /bin/bash
  Binaries:
    Node: 10.16.1 - ~/.nvm/versions/node/v10.16.1/bin/node
    npm: 6.9.0 - ~/.nvm/versions/node/v10.16.1/bin/npm
  Languages:
    Python: 2.7.10 - /usr/bin/python
  Browsers:
    Chrome: 80.0.3987.122
    Firefox: 73.0.1
    Safari: 12.1.2
  npmPackages:
    gatsby: ^2.19.22 => 2.19.28
    gatsby-image: ^2.2.41 => 2.2.42
    gatsby-plugin-alias-imports: ^1.0.5 => 1.0.5
    gatsby-plugin-google-tagmanager: ^2.1.25 => 2.1.25
    gatsby-plugin-manifest: ^2.2.39 => 2.2.42
    gatsby-plugin-offline: ^3.0.32 => 3.0.35
    gatsby-plugin-react-helmet: ^3.1.21 => 3.1.22
    gatsby-plugin-react-svg: ^3.0.0 => 3.0.0
    gatsby-plugin-sass: ^2.1.28 => 2.1.29
    gatsby-plugin-sharp: ^2.4.5 => 2.4.5
    gatsby-plugin-sitemap: ^2.2.27 => 2.2.27
    gatsby-remark-images-contentful: ^2.1.31 => 2.1.32
    gatsby-source-contentful: ^2.1.85 => 2.1.89
    gatsby-source-filesystem: ^2.1.46 => 2.1.48
    gatsby-transformer-sharp: ^2.3.13 => 2.3.16
  npmGlobalPackages:
    gatsby-cli: 2.8.29

[MadhaviS]

I am also facing the exact issue

[vladar]

Hi @nadiavanleur !

Sorry to hear you're running into an issue. To help us best begin debugging the underlying cause, it is incredibly helpful if you're able to create a minimal reproduction. This is a simplified example of the issue that makes it clear and obvious what the issue is and how we can begin to debug it.

You can probably use contentful starter to prepare a reproduction site and contentful-export tool to fetch the content that produces this error.

If you're up for it, we'd very much appreciate if you could provide a minimal reproduction and we'll be able to take another look.

Thanks for using Gatsby! 💜

[github-actions]

Hiya!

This issue has gone quiet. Spooky quiet. 👻

We get a lot of issues, so we currently close issues after 30 days of inactivity. It’s been at least 20 days since the last update here.
If we missed this issue or if you want to keep it open, please reply here. You can also add the label "not stale" to keep this issue open!
As a friendly reminder: the best way to see this issue, or any other, fixed is to open a Pull Request. Check out gatsby.dev/contribute for more information about opening PRs, triaging issues, and contributing!

Thanks for being a part of the Gatsby community! 💪💜

[thijs-van-der-velden]

@vladar

I am running into a similar issue. My build fails with a Javascript heap out of memory error. I've narrowed this down to the call

function prepareRichTextNode(node, key, content, createNodeId) {
  const str = stringify(content);

The issue is that the contentful sdk link resolver creates a nested traversable object. As all links are resolved, an RTE link to a piece of content with multiple links could lead to the resulting object containing the entire site... json-stringify-safe tries its best to convert this to a json string but ultimately runs out of memory.

I have confirmed this is the issue by disabling the link resolver on the contentful client.

As for how to resolve this issue... I'm not sure that I see a simple way of doing it. Ultimately, the implementation of outputting the RTE content as JSON is simply not compatible with the idea of being able to retrieve fields of entries linked in the RTE. Either the gql content types for RTE content need to be defined so that RTEs can leverage gql, or the JSON content needs to be trimmed somehow...

Unfortunately due to the nature of the issue, creating a minimal reproduction is both complicated (the issue only occurs when you have a lot of content) and unreliable (an out of memory error depends on the amount of memory available to the environment).

[github-actions]

Hiya!

This issue has gone quiet. Spooky quiet. 👻

We get a lot of issues, so we currently close issues after 30 days of inactivity. It’s been at least 20 days since the last update here.
If we missed this issue or if you want to keep it open, please reply here. You can also add the label "not stale" to keep this issue open!
As a friendly reminder: the best way to see this issue, or any other, fixed is to open a Pull Request. Check out gatsby.dev/contribute for more information about opening PRs, triaging issues, and contributing!

Thanks for being a part of the Gatsby community! 💪💜

[github-actions]

Hey again!

It’s been 30 days since anything happened on this issue, so our friendly neighborhood robot (that’s me!) is going to close it.
Please keep in mind that I’m only a robot, so if I’ve closed this issue in error, I’m HUMAN_EMOTION_SORRY. Please feel free to reopen this issue or create a new one if you need anything else.
As a friendly reminder: the best way to see this issue, or any other, fixed is to open a Pull Request. Check out gatsby.dev/contribute for more information about opening PRs, triaging issues, and contributing!

Thanks again for being part of the Gatsby community! 💪💜