[Misc] Research caching to improve yarn build times

[zstix]

Descriptions

Running yarn build or yarn develop can take over 2 minutes to run for a fresh install and the repository currently only represents 10% of the total amount of content. This is a bit of a deterrent for new people who want to jump in and contribute.

We need to do some research into how to lower this build time and ensure we can scale up the amount of content without exponentially increasing build times. Caching images and queries may be one way to achieve this, but it doesn't solve the issue for new developers.

Acceptance Criteria

Since this is somewhat vague, we will probably want to timebox this effort to no more than 4 hours (for now).

• Identify options for reducing the build time
• Decide on which options we would like to take
• Either execute the options, or add them to our backlog

[zstix]

We have done some research on this issue during a previous MMF: #32

We may be able to utilize incremental builds to help improve this process.

It's possible that this issue has been solved by the Gatsby community.

[zstix]

https://www.gatsbyjs.com/docs/conditional-page-builds/

[zstix]

https://bulletninja.com/enable-gatsby-incremental-builds-aws-amplify

Here's some more information about that: https://bulletninja.com/enable-gatsby-incremental-builds-aws-amplify

[caylahamann]

With the update of Gatsby to the newest version, caching has dramatically improved.

• yarn develop without caching ~ 3 minutes (largest culprits are running page queries takking 75 seconds and generating image thumbnails which take 125.578s)
• yarn develop with caching - ~ 8 seconds

So there is a long initial start, but as long as someone doesn't run gatsby clean or makes changes to the gatsby apis (which content creators are not likely to do) this shouldn't be too much of an issue. Possible ways to improve build times:

• Cloudinary as an asset management tool. We can easily integrate into gatsby via a plugin, the tooling in the website seems to be simple and easy to use, and we would be able to add transform the images via our migration script. This might take a bit of work, but it should cut down on the generating of image thumbnails (especially as migration continues).
• Conditional Builds this wouldn't help our development process, but would ensure slightly faster builds in amplify.

[caylahamann]

without images and no caching - 141.587781166 s
with images and no caching - 179.265728646 s

We could also limit the number of pages built during development. If a contributor knows that they only want to develop on a certain page, we can put that as an input and then just modify the query for development.

[caylahamann]

• make ticket with pricing info for cloudinary

[caylahamann]

followed up with this ticket: #257