Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Core: Remove dependence on file-system-cache #29256

Merged
merged 15 commits into from
Oct 7, 2024
Merged

Conversation

ndelangen
Copy link
Member

@ndelangen ndelangen commented Oct 1, 2024

Closes #29168

What I did

I embeddedn file-system-cache into our own source code.

Checklist for Contributors

Testing

The changes in this PR are covered in the following automated tests:

  • stories
  • unit tests
  • integration tests
  • end-to-end tests

Manual testing

This section is mandatory for all contributions. If you believe no manual test is necessary, please state so explicitly. Thanks!

Documentation

  • Add or update documentation reflecting your changes
  • If you are deprecating/removing a feature, make sure to update
    MIGRATION.MD

Checklist for Maintainers

  • When this PR is ready for testing, make sure to add ci:normal, ci:merged or ci:daily GH label to it to run a specific set of sandboxes. The particular set of sandboxes can be found in code/lib/cli/src/sandbox-templates.ts

  • Make sure this PR contains one of the labels below:

    Available labels
    • bug: Internal changes that fixes incorrect behavior.
    • maintenance: User-facing maintenance tasks.
    • dependencies: Upgrading (sometimes downgrading) dependencies.
    • build: Internal-facing build tooling & test updates. Will not show up in release changelog.
    • cleanup: Minor cleanup style change. Will not show up in release changelog.
    • documentation: Documentation only changes. Will not show up in release changelog.
    • feature request: Introducing a new feature.
    • BREAKING CHANGE: Changes that break compatibility in some way with current major version.
    • other: Changes that don't fit in the above categories.

🦋 Canary release

This pull request has been released as version 0.0.0-pr-29256-sha-cd1d83f5. Try it out in a new sandbox by running npx storybook@0.0.0-pr-29256-sha-cd1d83f5 sandbox or in an existing project with npx storybook@0.0.0-pr-29256-sha-cd1d83f5 upgrade.

More information
Published version 0.0.0-pr-29256-sha-cd1d83f5
Triggered by @JReinhold
Repository storybookjs/storybook
Branch norbert/replace-fscache
Commit cd1d83f5
Datetime Mon Oct 7 10:48:15 UTC 2024 (1728298095)
Workflow run 11213928182

To request a new release of this pull request, mention the @storybookjs/core team.

core team members can create a new canary release here or locally with gh workflow run --repo storybookjs/storybook canary-release-pr.yml --field pr=29256

name before after diff z %
createSize 0 B 0 B 0 B - -
generateSize 78.7 MB 78.7 MB 0 B 1.37 0%
initSize 151 MB 150 MB -792 kB -1.55 -0.5%
diffSize 72.6 MB 71.8 MB -792 kB -1.78 -1.1%
buildSize 6.77 MB 6.77 MB 0 B -0.57 0%
buildSbAddonsSize 1.5 MB 1.5 MB 0 B -0.58 0%
buildSbCommonSize 195 kB 195 kB 0 B - 0%
buildSbManagerSize 1.83 MB 1.83 MB 0 B -0.56 0%
buildSbPreviewSize 270 kB 270 kB 0 B -0.58 0%
buildStaticSize 0 B 0 B 0 B - -
buildPrebuildSize 3.8 MB 3.8 MB 0 B -0.57 0%
buildPreviewSize 2.97 MB 2.97 MB 0 B -0.85 0%
testBuildSize 0 B 0 B 0 B - -
testBuildSbAddonsSize 0 B 0 B 0 B - -
testBuildSbCommonSize 0 B 0 B 0 B - -
testBuildSbManagerSize 0 B 0 B 0 B - -
testBuildSbPreviewSize 0 B 0 B 0 B - -
testBuildStaticSize 0 B 0 B 0 B - -
testBuildPrebuildSize 0 B 0 B 0 B - -
testBuildPreviewSize 0 B 0 B 0 B - -
name before after diff z %
createTime 17.4s 22.9s 5.4s 1.51 🔺23.8%
generateTime 22.4s 19.3s -3s -109ms -0.7 -16.1%
initTime 16.4s 13.3s -3s -134ms -0.95 -23.5%
buildTime 8.8s 8.1s -675ms -1.45 🔰-8.2%
testBuildTime 0ms 0ms 0ms - -
devPreviewResponsive 11.8s 6s -5s -766ms -0.94 -94.7%
devManagerResponsive 6.7s 3.8s -2s -951ms -1.2 -77.6%
devManagerHeaderVisible 825ms 542ms -283ms -0.7 -52.2%
devManagerIndexVisible 893ms 576ms -317ms -0.67 -55%
devStoryVisibleUncached 2.5s 1.3s -1s -146ms 0.19 -83.5%
devStoryVisible 901ms 575ms -326ms -0.68 -56.7%
devAutodocsVisible 767ms 477ms -290ms -0.54 -60.8%
devMDXVisible 839ms 477ms -362ms -0.59 -75.9%
buildManagerHeaderVisible 897ms 499ms -398ms -0.53 -79.8%
buildManagerIndexVisible 898ms 557ms -341ms -0.3 -61.2%
buildStoryVisible 952ms 558ms -394ms -0.45 -70.6%
buildAutodocsVisible 736ms 522ms -214ms -0.25 -41%
buildMDXVisible 815ms 508ms -307ms -0.24 -60.4%

Greptile Summary

This PR removes the dependency on file-system-cache and replaces it with a custom implementation embedded in the project's source code, addressing issues such as size, dependency on ramda, typing problems, and versioning inconsistencies.

  • Implemented FileSystemCache class in code/core/src/common/utils/file-cache.ts to replace external dependency
  • Updated code/builders/builder-webpack5/src/index.ts and src/preview/iframe-webpack.config.ts to use new cache implementation
  • Removed file-system-cache from dependencies in code/core/package.json
  • Modified code/core/src/cli/dev.ts to adjust for new cache implementation
  • Updated type imports in code/core/src/types/modules/core-common.ts to use local FileSystemCache type

@ndelangen ndelangen changed the title remove file-system-cache by embedding it into our source Maintenance: Remove dependence on file-system-cache Oct 1, 2024
@ndelangen ndelangen self-assigned this Oct 1, 2024
@ndelangen ndelangen added maintenance User-facing maintenance tasks ci:normal labels Oct 1, 2024
@ndelangen ndelangen requested a review from JReinhold October 1, 2024 14:35
Copy link

nx-cloud bot commented Oct 1, 2024

☁️ Nx Cloud Report

CI is running/has finished running commands for commit fbf7a92. As they complete they will appear below. Click to see the status, the terminal output, and the build insights.

📂 See all runs for this CI Pipeline Execution


✅ Successfully ran 1 target

Sent with 💌 from NxCloud.

@ndelangen ndelangen marked this pull request as ready for review October 2, 2024 19:38
Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

8 file(s) reviewed, 3 comment(s)
Edit PR Review Bot Settings

Comment on lines +17 to +18
content?: any;
value?: any;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

style: Consider using a more specific type than 'any' for content and value

}

public async setMany(items: CacheItem[], options?: CacheSetOptions): Promise<void> {
await Promise.all(items.map((item) => this.set(item.key, item.content ?? item.value, options)));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

style: This could potentially cause performance issues with large numbers of items. Consider batching

Comment on lines +139 to +140
.map((data) => JSON.parse(data))
.filter((entry) => entry.content && !this.isExpired(entry, now));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

style: This approach might be inefficient for large caches. Consider implementing pagination or streaming

Copy link
Contributor

@JReinhold JReinhold left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code looks good to me, only a single remark.
We need to test this in Windows before I'd feel confident merging, giving how critical path this is.

code/core/src/common/utils/file-cache.ts Show resolved Hide resolved
@ndelangen ndelangen merged commit 2d3ee31 into next Oct 7, 2024
57 checks passed
@ndelangen ndelangen deleted the norbert/replace-fscache branch October 7, 2024 12:24
@github-actions github-actions bot mentioned this pull request Oct 7, 2024
6 tasks
@shilman shilman changed the title Maintenance: Remove dependence on file-system-cache Core: Remove dependence on file-system-cache Oct 8, 2024
@JReinhold JReinhold added dependencies maintenance User-facing maintenance tasks and removed maintenance User-facing maintenance tasks dependencies labels Oct 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ci:normal maintenance User-facing maintenance tasks
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Investigate migrating away from file-system-cache
2 participants