feat(jest-haste-map): handle injected scm clocks #10966
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
SimenB
reviewed
Dec 22, 2020
SimenB
reviewed
Dec 22, 2020
CHANGELOG.md
Outdated
| @@ -15,6 +15,7 @@ | |||
| - `[jest-runner]` [**BREAKING**] Run transforms over `testRunnner` ([#8823](https://github.com/facebook/jest/pull/8823)) | |||
| - `[jest-runtime, jest-transform]` share `cacheFS` between runtime and transformer ([#10901](https://github.com/facebook/jest/pull/10901)) | |||
| - `[jest-transform]` Pass config options defined in Jest's config to transformer's `process` and `getCacheKey` functions ([#10926](https://github.com/facebook/jest/pull/10926)) | |||
| - `[jest-haste-map]` Handle injected scm clocks ([#10966](https://github.com/facebook/jest/pull/10966)) | |||
SimenB
approved these changes
Dec 22, 2020
|
@SimenB Yeah, I'm not sure about documentation. I'll go ahead and at least explain it further in code comments for anyone in the future that might be poking around for a similar use-case. |
|
This pull request has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs. |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Jest's haste map supports incremental update. What this means is that when you first run Jest, a full rebuild of the map is done, but future invocations will only update changed files, which is pretty quick.
At Facebook, due to the size of our monorepo, we've taken this one step further. The first run of Jest downloads a version of the haste map that was created for a revision near yours. We then update it from there. This is done because doing a full rebuild takes several minutes.
To support this, we've been using a forked version of the Watchman crawler that supported doing an "SCM query". You can read about it here: https://facebook.github.io/watchman/docs/scm-query.html
The reason this is necessary is because Watchman clocks are not portable across systems; the haste map you create on your machine cannot by default be run on mine. The downloaded haste map contains a clock that references source control mergebases, which are portable.
This PR allows Jest, when it sees a specific type of clock that represents a source-control query, to handle it properly.
Rather than maintaining our own custom crawler, I'd like to upstream this. Upstreaming will make it simpler to distribute this to the variety of places that need it and may benefit others with a similar use-case; if any of them use hg and Watchman :)
Test plan
I added a unit test for this behavior. Specifically, the tricky part is that Watchman responds slightly differently to the query, with a divergence in
is_fresh_instanceandclockfields. I also killed a variety ofanytypes in this file for an extra layer of protection, since the crux of this diff involved changing response types.Additionally, I tested this end-to-end against FB infra to be sure it'll work properly in that context.