Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add clearCache cli command #4430

Merged
merged 14 commits into from
Oct 11, 2017
Merged

Conversation

tabrindle
Copy link
Contributor

Summary

Occasionally Jest's cache needs to be cleared for debugging purposes.

This seems to be a fairly common question, to say nothing about caching being a difficult thing:

https://stackoverflow.com/questions/44866994/how-to-clear-jest-cache

It has also been mentioned in issues here: #3705

Even with a PR to explain how to do so. #4232

Seems silly to have a multi step operation to clear a cache - so I wrote a bash script to do it
jest --showConfig | grep cacheDirectory | awk -F\" '{ print $4 }' | xargs rm -rf which in the end is dumb because jest has a cli already, and parsing JSON in bash like this is a bad idea.

Test plan

jest --clearCache

This should do nothing more than delete the cacheDirectory and exit 0.

@thymikee
Copy link
Collaborator

thymikee commented Sep 5, 2017

Thanks! Can you make sure yarn test passes? There's lint issue reported by CI

Copy link
Collaborator

@thymikee thymikee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need an integration test for this.

clearCache: {
default: undefined,
description:
'Clear default directory Jest uses for cache and exits.' +
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: Clears or exit, whatever the convention is right now (hard to access on a phone)

clearDirectory(configs[0].cacheDirectory);
process.stdout.write(`Cleared ${configs[0].cacheDirectory}\n`);
process.exit(0);
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should be able to clear the cache for certain project or all projects, right now it's only for the first one.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wasn't sure how this worked: should I just map over the configs array?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yup, should be fine

const clearDirectory = (path: string) => {
if (fs.existsSync(path)) {
fs.readdirSync(path).map(file => {
const filePath = `${path}/${file}`;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This won't work on windows, use path.join or path.resolve

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Didnt even think of that. Will research.

@tabrindle
Copy link
Contributor Author

@thymikee I'm really not sure where to start on the integration test- Im assuming we just need to mock jest for the config object it generates, run the clearDirectory function then verify the directory is empty. Open to suggestions.

@thymikee
Copy link
Collaborator

thymikee commented Sep 6, 2017

You can check integration_tests for in this repo to see how to write one. You don't mock anything, just spawn a real jest instance with desired config and test the output

@@ -31,11 +33,26 @@ const createDirectory = (path: string) => {
}
};

const clearDirectory = (directoryPath: string) => {
Copy link
Member

@SimenB SimenB Sep 7, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not just rimraf? (or del or one other of the many other modules that does this)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only because I don't generally add dependencies to someone else's project unless they mention it. rimraf is fine.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Migrated to rimraf.

@tabrindle
Copy link
Contributor Author

@thymikee Added an integration test, and it's passing. Refactored to use rimraf. Should be read for re-review.

Test Suites: 166 passed, 166 total
Tests:       1574 passed, 1574 total
Snapshots:   691 passed, 691 total
Time:        48.293s, estimated 53s
Ran all test suites.
=============================== Coverage summary ===============================
Statements   : 53.72% ( 3766/7010 )
Branches     : 55.61% ( 2131/3832 )
Functions    : 51.1% ( 746/1460 )
Lines        : 57.18% ( 3689/6451 )
================================================================================

Looks like appveyor is failing, but it doesn't appear to have anything to do with me. Everything else is passing.

@thymikee
Copy link
Collaborator

thymikee commented Sep 8, 2017

Looks like you didn't push the changes?

@tabrindle
Copy link
Contributor Author

Derp. Sorry.

@codecov-io
Copy link

codecov-io commented Sep 8, 2017

Codecov Report

Merging #4430 into master will not change coverage.
The diff coverage is n/a.

Impacted file tree graph

@@           Coverage Diff           @@
##           master    #4430   +/-   ##
=======================================
  Coverage   57.13%   57.13%           
=======================================
  Files         194      194           
  Lines        6565     6565           
  Branches        3        3           
=======================================
  Hits         3751     3751           
  Misses       2814     2814

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1ece140...4680586. Read the comment docs.

Copy link
Collaborator

@thymikee thymikee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry to hold this, but it just needs a bit more work. We're almost there 🙂

let clearCacheError;

configs.map(config => {
rimraf(config.cacheDirectory, [], () => {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm afraid this is completely wrong and doesn't work at all.
Please refer to rimraf tests on how to use it:
https://github.com/isaacs/rimraf/blob/d84fe2cc6646d30a401baadcee22ae105a2d4909/test/basic.js#L48-L55.

Also I see no point in using async API, since we're not doing anything else in this time. Please use rimraf.sync, this way you'll avoid temporary clearCacheError variable and perf overhead of async call.

Copy link
Contributor Author

@tabrindle tabrindle Sep 8, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure what I was thinking - process exited before async call finished. Will switch to sync. Will be easier than using a promise all similar.

`--cacheDirectory=${CACHE}`,
]);

expect(fs.existsSync(CACHE)).toBe(false);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a false positive. This folder doesn't exist, because Jest didn't run any tests. We need to populate this cache earlier with a sample run, then run it once more. Or just create the dir and make sure it is present prior to running test.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry I thought the folder was created when calling runJest regardless of whether there are tests or not. Will add a stub test, verify the cache dir has contents, then runJest again with clearCache

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you add

expect(fs.existsSync(CACHE)).toBe(true);

above the runJest call?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the second test correct? Makes sense.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct!

@tabrindle
Copy link
Contributor Author

No problem, if it's not right you shouldn't pass it :)

@@ -67,6 +68,15 @@ const runCLI = async (
outputStream,
);

if (argv.clearCache) {
configs.map(config => {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

forEach over map. (You ignore the return value anyways)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I usually prefer map to prevent accidental mutation. I suppose here it doesn't matter as the process is about to end. I will defer to you on this.

default: undefined,
description:
'Clears the configured Jest cache directory and then exits.' +
'Default directory can be found by calling jest --showConfig',
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing space before "Default"?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch.


const CACHE = path.resolve(os.tmpdir(), 'clear_cache_directory');

skipOnWindows.suite();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Necessary, or copypasta? (Just asking, fine if needed 😄)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

skipOnWindows? I assumed it was part of the test convention as I saw it everywhere. I can remove it if it isn't necessary.

@tabrindle
Copy link
Contributor Author

There is currently an issue on master where we get the following stack trace messing up quite a few tests --including mine-- where tests are looking for a specific stderr output.

Looks related to yargs, but the last breaking upgrade to yargs was before I started on this PR.

YError: Invalid first argument. Expected string or function but received boolean.
    at argumentTypeError (/Users/tabrindle/Developer/jest/packages/jest-cli/node_modules/yargs/lib/argsert.js:70:9)
    at /Users/tabrindle/Developer/jest/packages/jest-cli/node_modules/yargs/lib/argsert.js:52:39
    at Array.forEach (<anonymous>)
    at module.exports (/Users/tabrindle/Developer/jest/packages/jest-cli/node_modules/yargs/lib/argsert.js:45:21)
    at Object.Yargs.self.version (/Users/tabrindle/Developer/jest/packages/jest-cli/node_modules/yargs/yargs.js:718:5)
    at _buildArgv (/Users/tabrindle/Developer/jest/packages/jest-cli/build/cli/index.js:118:3)
    at Object.<anonymous> (/Users/tabrindle/Developer/jest/packages/jest-cli/build/cli/index.js:37:18)
    at Generator.next (<anonymous>)
    at step (/Users/tabrindle/Developer/jest/packages/jest-cli/build/cli/index.js:45:2466)
    at /Users/tabrindle/Developer/jest/packages/jest-cli/build/cli/index.js:45:2696

@felipeochoa felipeochoa mentioned this pull request Oct 4, 2017
3 tasks
@SimenB
Copy link
Member

SimenB commented Oct 8, 2017

@tabrindle this PR seems like it's gotten lots of commits it shouldn't have. Care to rebase?

Regarding you issue, does it help to do yarn clean-all && yarn (make sure you're on yarn@1.1.0 as well)? Seems like something is weird with your dependencies locally, so maybe doing a clean install will help 🙂

- map over all configs
- rename path arg to directoryPath for less confusion
- yarn run jest -- -t 'jest --clearCache'

$ node ./packages/jest-cli/bin/jest.js "-t" "jest --clearCache"
 PASS  integration_tests/__tests__/clear_cache.test.js

Test Suites: 165 skipped, 1 passed, 1 of 166 total
Tests:       1573 skipped, 1 passed, 1574 total
Snapshots:   0 total
- actually create directory with real test
- check for directory, then delete
- use rimraf.sync
- remove temp var
@tabrindle
Copy link
Contributor Author

tabrindle commented Oct 11, 2017

@SimenB Thanks for the tip - Rebased & looks like my new tests are passing, and the only test failing is the same one failing on master.

 FAIL  packages/jest-worker/src/__tests__/child.test.js
  ● lazily requires the file

    TypeError: Cannot read property 'concat' of undefined
      
      at handle (node_modules/worker-farm/lib/child/index.js:44:24)
      at emitOne (events.js:120:20)
      at process.emit (events.js:210:7)
          at Promise (<anonymous>)

@SimenB
Copy link
Member

SimenB commented Oct 11, 2017

CI is happy, that's great. The failing test you see is tracked in #4630

@tabrindle
Copy link
Contributor Author

In an effort to double check, I was able to confirm the new cache directory creation, and it's deletion via jest -t "jest --clearCache"

On my machine: /var/folders/9h/tl11z92d59sd2zw5c3zcxn740000gn/T/clear_cache_directory is created then immediately destroyed via integration test.

/**
* Copyright (c) 2014-present, Facebook, Inc. All rights reserved.
*
* This source code is licensed under the BSD-style license found in the
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

change the license header to be MIT, please

Copy link
Contributor Author

@tabrindle tabrindle Oct 11, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also found jest/integration_tests/__tests__/pass_with_no_tests.test.js with BSD license. Should I take care of that here?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes please

@SimenB
Copy link
Member

SimenB commented Oct 11, 2017

CI is red. Circle is not friendly to mobile, but I'm guessing a lint error?

@tabrindle
Copy link
Contributor Author

tabrindle commented Oct 11, 2017

Indeed - appears .vscode/settings.json has prettier configured "editor.formatOnSave": true, which spaced out the brackets. I can look into this later.

edit: didnt have prettier-vscode installed before, and somehow it was still formatting - once installed the config seems to work correctly.

@tabrindle
Copy link
Contributor Author

@thymikee @SimenB I believe this is ready to go - sorry for the long process!

Copy link
Member

@SimenB SimenB left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thanks for sticking with us

@cpojer cpojer merged commit d82925e into jestjs:master Oct 11, 2017
@cpojer
Copy link
Member

cpojer commented Oct 11, 2017

Awesome! Can you also send a PR to update the documentation with this arg?

@SimenB
Copy link
Member

SimenB commented Oct 11, 2017

Haha.

image

@tabrindle
Copy link
Contributor Author

Will do!

tabrindle added a commit to tabrindle/jest that referenced this pull request Oct 11, 2017
@tabrindle tabrindle mentioned this pull request Oct 11, 2017
cpojer pushed a commit that referenced this pull request Oct 11, 2017
* add to changelog for PR #4430

* add —clearCache to docs
@github-actions
Copy link

This pull request has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.
Please note this issue tracker is not a help forum. We recommend using StackOverflow or our discord channel for questions.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators May 13, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants