Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DDW-1227] Fix out-of-memory errors on ci.iog.io #3145

Merged
merged 6 commits into from
Aug 16, 2023

Conversation

michalrus
Copy link
Member

@michalrus michalrus commented Aug 10, 2023

We’re OOM-ing on https://ci.iog.io/, and therefore have cache misses – developers have to build devshells locally (slow), instead of downloading them from the cache. Let’s reduce the memory footprint of the evaluation

This PR fixes that by reducing the number of times we evaluate Nixpkgs. After the changes, they'll only be evaluated once per each target system. Previously they were evaluated once per each pair of Cardano network and target system (i.e. for each installer), and also multiple times for devshells (we have separate devshells for each Cardano network).

IMO, it doesn’t require a QA run, since it has no functional changes, just build changes (and not significant ones)

You can see that it is a safe transformation in terms of the actual builds, because:

  • the main change is to move the cluster parameter down 1 level (previously we had internal.x86_64-windows.mainnet.unsignedInstaller, now we have internal.x86_64-windows.unsignedInstaller.mainnet) – this is done by wrapping some attrs with genCluster (cluster: …), and later referencing them with the suffix .${cluster},
  • some symbol names have changed: newCommoncommon, daedalusPkgscommon,
  • code from old-default.nix has been moved to common.nix,
  • old-shell.nix has been renamed to devshells.nix and it now takes pkgs and common from inputs.self.internal.*, preventing previous reevaluations

You can also see that it worked here: https://ci.iog.io/build/598723#tabs-constituents

Todos

n/a

Screenshots

n/a

Testing Checklist


Review Checklist

Basics

  • PR assigned to the PR author(s)
  • input-output-hk/daedalus-dev and input-output-hk/daedalus-qa assigned as PR reviewers
  • If there are UI changes, Alexander Rukin assigned as an additional reviewer
  • PR has appropriate labels (release-vNext, feature/bug/chore, WIP)
  • PR link is added to a Jira ticket, ticket moved to In Review
  • PR is updated to the most recent version of the target branch (and there are no conflicts)
  • PR has a good description that summarizes all changes
  • PR contains screenshots (in case of UI changes)
  • CHANGELOG entry has been added to the top of the appropriate section (Features, Fixes, Chores) and is linked to the correct PR on GitHub
  • There are no missing translations (running yarn manage:translations produces no changes)
  • Text changes are proofread and approved (Jane Wild / Amy Reeve)
  • Japanese text changes are proofread and approved (Junko Oda)
  • Storybook works and no stories are broken (yarn storybook)
  • In case of dependency changes yarn.lock file is updated

Code Quality

  • Important parts of the code are properly commented and documented
  • Code is properly typed with typescript types
  • React components are split-up enough to avoid unnecessary re-renderings
  • Any code that only works in main process is neatly separated from components

Testing

  • New feature/change is covered by acceptance tests
  • New feature/change is manually tested and approved by QA team
  • All existing acceptance tests are still up-to-date
  • New feature/change is covered by Daedalus Testing scenario
  • All existing Daedalus Testing scenarios are still up-to-date

After Review

  • Update Slack QA thread by marking it with a green checkmark

Copy link
Contributor

@angerman angerman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, though, @tgunnoe should have a look as well!

@michalrus michalrus removed the request for review from dermetfan August 11, 2023 10:39
@tgunnoe
Copy link
Member

tgunnoe commented Aug 11, 2023

I couldn't find anything to complain about. good work!

@marcin-mazurek marcin-mazurek merged commit 883cbed into develop Aug 16, 2023
@michalrus michalrus deleted the fix/ddw-1227-hydra-oom branch August 16, 2023 08:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants