Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

UTxO-HD heap increase in latest benchmarks #1192

Closed
jasagredo opened this issue Jul 29, 2024 · 5 comments · Fixed by #1232
Closed

UTxO-HD heap increase in latest benchmarks #1192

jasagredo opened this issue Jul 29, 2024 · 5 comments · Fixed by #1232
Assignees
Labels

Comments

@jasagredo
Copy link
Contributor

jasagredo commented Jul 29, 2024

Live data increased from 3800MB to 4200MB in value workload

image

and from 3400MB to 4600MB in plutus workload

image

@jasagredo jasagredo transferred this issue from IntersectMBO/cardano-node Jul 29, 2024
@jasagredo jasagredo changed the title Live data increased from 3800MB to 4200MB in value workload and from 3400MB to 4600MB in plutus workload. UTxO-HD heap increase in latest benchmarks Jul 29, 2024
@jasagredo jasagredo moved this to 🏗 In progress in Consensus Team Backlog Jul 29, 2024
@jasagredo jasagredo self-assigned this Jul 29, 2024
@jasagredo
Copy link
Contributor Author

jasagredo commented Aug 12, 2024

Local cluster runs seem unreliable to reproduce this behavior. As the difference is more drastic in the forge-stress-pre-plutus workload we are running the following clusters:

  • utxo-hd-9.1 with -hT
    image (2)
  • utxo-hd-9.1 with -hi and -finfo-table-map -fdistinct-constructor-tables Nix needs adaptation to build with info table profiling.
  • 9.1.0 with -hT
    image

It is important to note that the max memory usage seems to be the Genesis loading phase, which influences the heap size for the overall run:
image (3)

@jasagredo
Copy link
Contributor Author

As a side note, #1194 doesn't seem to have any notable benefit in the memory usage.

@jasagredo
Copy link
Contributor Author

The graphs above show that on local runs, there is only an 80 MB difference on BabbageTxOut which cannot be unpacked in GHC 8.10.7, see

| BabbageTxOut !(Value (LedgerState (ShelleyBlock (Praos c) (BabbageEra c))))

@jasagredo jasagredo moved this from 🏗 In progress to 🚫 Help needed in Consensus Team Backlog Aug 14, 2024
@jasagredo
Copy link
Contributor Author

This is blocked on us being able to get profiling information from the cluster runs. We need the help from the Perf team.

@jasagredo
Copy link
Contributor Author

Done in #1232

@github-project-automation github-project-automation bot moved this from 🚫 Help needed to ✅ Done in Consensus Team Backlog Sep 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Status: ✅ Done
Development

Successfully merging a pull request may close this issue.

1 participant