Proposal: download Cypress to the common cached folder

[bahmutov]

Feature

Instead of downloading and unzipping Cypress into ./node_modules/cypress/dist/ download it into a global folder for better caching. This will allow

• better caching of dependencies on CI
• reusing same version of Cypress if already downloaded once for some project

To determine output location use https://github.com/LinusU/node-cachedir

var cachedir = require('cachedir')
var path = cachedir('cypress')
// `path` now contains the path under which you should store cached files

• cypress run, open, verify

  • take arg --binary-folder (looks like: /foo/bar/Cypress.app (macOS) or /foo/bar/Cypress (linux, win), to override default path to cypress binary folder (default is (cypressCache)/<cli version>/Cypress.app)
  • can override folder of binary with env var CYPRESS_BINARY_FOLDER=/path/to/Cypress.app
    • warn user if set

• cypress install

  • Checks (cypressCache)/<cli version> for already installed binary folder (Cypress.app on macOS, Cypress on linux/win), and if binary version matches (installVersion), do nothing.
  • (installVersion) defaults to cli version, is overridden by env var CYPRESS_BINARY_VERSION during install
    • can be either semvar: 1.2.3, url: www.cypress.io/download/1.2.3, or zip: /path/to/cypress.zip
  • if cypress install --force, force reinstall to (cypressCache)/<cli version>
  • (cypressCache) defaults to:
    • ~/Library/Caches/Cypress (macOS)
    • ~/.cache/Cypress (linux)
    • /AppData/Local/Cypress/Cache (win)
  • (cypressCache) can be overridden with env var CYPRESS_CACHE_DIRECTORY
  • before install, ensure write access to (cypressCache)

• custom error message for failed access to (cypressCache)

• docs

  • Installing Cypress
  • command line usage
  • update info about global install

• manual testing all use cases

  • cypress install
    • CYPRESS_BINARY_VERSION
      • url
        • skip if has any cache
      • semvar
      • zip
        • skip if has any cache
        • show progress for unzipping
      • --force
    • --force
  • --binary-folder
    • cypress run
    • cypress open
    • cypress verify
    • invalid
  • CYPRESS_CACHE_DIRECTORY
    • valid
    • invalid, error message

• add local code coverage with nyc

  • send to coveralls in CI

Stretch goal

• can we cache ffmpeg separately to avoid bundling the same version over and over?

[ntwb]

Either this is a duplicate of #1233 or vice-versa, not overly concerned which issue is closed but they are for the most part duplicates

[zkat]

psst: If you use make-fetch-happen you'll be able to tie the cypress download cache to npm's own cache (by pointing it at process.env.npm_config_cache in the run script).

[EirikBirkeland]

The Cypress.io download always doubles the time my yarn install takes, on a 25 mbit connection. There are ways to get around, but would greatly appreciate caching.

[bahmutov]

Can you cache node_modules ?

…

Sent from my iPhone

On Mar 20, 2018, at 11:22, Eirik Birkeland ***@***.***> wrote: The Cypress.io download always doubles the time my yarn install takes, on a 25 mbit connection. There are ways to get around, but would greatly appreciate caching. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

[ncjones]

To work around this issue we build a docker image with our test suite's dependencies pre-installed.

[KellyR-STCU]

Would love to see this put into a global location. My team is really excited about Cypress, but we've got to optimize the build time. I tried out npm ci and that actually shaved 2 minutes off the install process. Preventing this download from needing to occur repeatedly would shave about another 40-45 seconds off.

[EirikBirkeland]

@bahmutov I can cache node_modules on CI server, but for local builds we often change our dependencies because we are jumping between branches.

[garytryan]

Also having this problem with slow install times for local development using Yarn.

[kuceb]

currently in progress in PR #1582

(my hard drive needed the space) -->

[kuceb]

This also enables us to ship our docker images with Cypress pre-cached, which will help simplify CI setup!
Opened issue in docker images

[brian-mann]

Yeah it's true but we'll need to update our build process to include it - but we should it's a good idea.

Open a separate issue for that and we can work on that while you finish this PR.

[bahmutov]

we need to think about caching Cypress in the docker container - because it takes up a lot of space. If we do second split (Electron vs our code), then YES, @bkucera that would be super good idea to include multiple versions pre-cached because they would not add much weight

[brian-mann]

@bahmutov good point... that's another "+" in the corner of splitting out those two things vs caching the entire app bundle.

[brian-mann]

Released in 3.0.0.

[maxime1992]

According to Gleb's comment, this issue should include:

better caching of dependencies on CI
reusing same version of Cypress if already downloaded once for some project

I just tried to:

• upgrade cypress within my project to 3.0
• make sure that ~/.cache/Cypress/3.0.0 exists
• rm -rf node_modules
• yarn again

Cypress has been downloaded again instead of using the local version, available in cypress global cache folder. Am I missing something? Is that not the expected behaviour and maybe I should open a new issue?

That's a great feature, looking forward to it, congrats :)

[kuceb]

@maxime1992 yep, that was a bug, fixed in 3.0.1 !

[jeserkin]

If one installs it globally on machine and versions match (with project), then installing with npm shouldn't pull it again and install on project level, correct?

[kuceb]

@jeserkin correct

[jeserkin]

@bkucera Is it possible, that installing dependencies with npm ci, somehow mitigates existence check and still downloads cypress based on project level description?
At the moment I have npm install -g cypress@3.3.2 pre-installed on docker image. Later I run npm ci in container, that is based on given image and it does following:

HH:mm:ss  > cypress@3.3.2 postinstall /build/xtss/node_modules/cypress
HH:mm:ss  > node index.js --exec install
HH:mm:ss  
HH:mm:ss  Installing Cypress (version: 3.3.2)
HH:mm:ss  
HH:mm:ss  �[?25l[12:31:14]  Downloading Cypress     [started]
HH:mm:ss  [12:31:15]  Downloading Cypress      0% 0s [title changed]
HH:mm:ss  [12:31:15]  Downloading Cypress      3% 3s [title changed]
HH:mm:ss  [12:31:15]  Downloading Cypress      8% 2s [title changed]
HH:mm:ss  [12:31:15]  Downloading Cypress      12% 2s [title changed]
HH:mm:ss  [12:31:15]  Downloading Cypress      18% 2s [title changed]
HH:mm:ss  [12:31:15]  Downloading Cypress      24% 2s [title changed]
HH:mm:ss  [12:31:15]  Downloading Cypress      29% 1s [title changed]
HH:mm:ss  [12:31:16]  Downloading Cypress      34% 1s [title changed]
HH:mm:ss  [12:31:16]  Downloading Cypress      41% 1s [title changed]
HH:mm:ss  [12:31:16]  Downloading Cypress      47% 1s [title changed]
HH:mm:ss  [12:31:16]  Downloading Cypress      52% 1s [title changed]
HH:mm:ss  [12:31:16]  Downloading Cypress      60% 1s [title changed]
HH:mm:ss  [12:31:16]  Downloading Cypress      68% 1s [title changed]
HH:mm:ss  [12:31:16]  Downloading Cypress      74% 0s [title changed]
HH:mm:ss  [12:31:16]  Downloading Cypress      81% 0s [title changed]
HH:mm:ss  [12:31:16]  Downloading Cypress      89% 0s [title changed]
HH:mm:ss  [12:31:16]  Downloading Cypress      96% 0s [title changed]
HH:mm:ss  [12:31:16]  Downloaded Cypress      [title changed]
HH:mm:ss  [12:31:16]  Downloaded Cypress      [completed]
HH:mm:ss  [12:31:16]  Unzipping Cypress       [started]