Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove compacting of TxIn. Add Keyed instances for hashes #2530

Merged
merged 3 commits into from
Nov 2, 2021

Conversation

lehins
Copy link
Collaborator

@lehins lehins commented Oct 25, 2021

This PR requires: IntersectMBO/cardano-base#243

@lehins lehins requested a review from TimSheard October 25, 2021 18:50
@lehins
Copy link
Collaborator Author

lehins commented Oct 25, 2021

@redxaxder Here is the reason why we need to remove compacted TxIn, we need an instance of Keyed for TxId

@TimSheard if you are ok with this PR feel free to "Squash and Merge| it, it is opened into your ts-compact branch.

@lehins lehins force-pushed the lehins/ts-compact-integration branch 2 times, most recently from 277388c to 1a1496b Compare October 26, 2021 15:04
@lehins lehins force-pushed the ts-compact branch 3 times, most recently from 3ed31b6 to 02f85c1 Compare October 26, 2021 16:26
@iohk-bors iohk-bors bot deleted the branch master October 26, 2021 17:54
@iohk-bors iohk-bors bot closed this Oct 26, 2021
@lehins lehins reopened this Oct 26, 2021
@lehins lehins changed the base branch from ts-compact to master October 26, 2021 17:56
@lehins lehins force-pushed the lehins/ts-compact-integration branch from 1a1496b to 835ce93 Compare October 26, 2021 18:02
@lehins lehins requested a review from redxaxder October 26, 2021 18:03
@lehins
Copy link
Collaborator Author

lehins commented Oct 26, 2021

After some experiments we found out that using HashMap that was implemented in #2520 is qyuite a bit more memory efficient then other approaches that involved manual unpacking of TxIn. Here is a comparison:

  Case                                  Max        MaxOS         Live       Allocated     GCs
  IntMap (KeyMap TxId ())       232,127,304  503,316,480  232,127,304  42,208,026,560  40,753
  KeyMap TxId (IntMap TxId ())  305,991,112  627,048,448  305,991,112  46,528,677,952  44,916
  IntMap (Map TxId ())          324,725,624  701,497,344  324,725,624  34,313,290,760  33,026
  Map TxIn ()                   383,746,832  896,532,480  383,746,832  33,154,708,560  31,937

In order for us to be able and use the most efficient from the top of the list, this PR is needed.

@lehins lehins force-pushed the lehins/ts-compact-integration branch from 835ce93 to d7dde93 Compare October 26, 2021 18:11
@lehins lehins force-pushed the lehins/ts-compact-integration branch from d7dde93 to fa20ff3 Compare October 26, 2021 18:47
Copy link
Contributor

@nc6 nc6 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Everything looks reasonable. I have two questions, but they're not really review comments:

  • The comment about needing a Keyed instance doesn't really motivate the non-unpacking.
  • Why the change from Word64 to Int for the index? Is this motivated by wanting to use IntMap?

{-# UNPACK #-} !Word64 -> -- Index
TxIn crypto
TxInCompactOther :: !(TxId crypto) -> {-# UNPACK #-} !Word64 -> TxIn crypto
data TxIn crypto = TxInCompact !(TxId crypto) {-# UNPACK #-} !Int
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why the change to Int here?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because we will use it with IntMap, which needs an Int, rather than a Word64. I don't think it would a problem, right? Considering that max value on mainnet is 249 and I don't see tx index going larger than maxBound :: Int any time in the foreseeable future 😄

@lehins lehins merged commit a8700c4 into master Nov 2, 2021
@iohk-bors iohk-bors bot deleted the lehins/ts-compact-integration branch November 2, 2021 09:33
nc6 added a commit to IntersectMBO/cardano-node that referenced this pull request Nov 4, 2021
Recent ledger changes
(IntersectMBO/cardano-ledger#2530) have
restricted the range for transaction input indices. Since these are
unlikely to be large anyway, restrict the generated range.
erikd pushed a commit to IntersectMBO/cardano-node that referenced this pull request Nov 4, 2021
Recent ledger changes
(IntersectMBO/cardano-ledger#2530) have
restricted the range for transaction input indices. Since these are
unlikely to be large anyway, restrict the generated range.
erikd pushed a commit to IntersectMBO/cardano-node that referenced this pull request Nov 4, 2021
Recent ledger changes
(IntersectMBO/cardano-ledger#2530) have
restricted the range for transaction input indices. Since these are
unlikely to be large anyway, restrict the generated range.
erikd pushed a commit to IntersectMBO/cardano-node that referenced this pull request Nov 5, 2021
Recent ledger changes
(IntersectMBO/cardano-ledger#2530) have
restricted the range for transaction input indices. Since these are
unlikely to be large anyway, restrict the generated range.
erikd pushed a commit to IntersectMBO/cardano-node that referenced this pull request Nov 6, 2021
Recent ledger changes
(IntersectMBO/cardano-ledger#2530) have
restricted the range for transaction input indices. Since these are
unlikely to be large anyway, restrict the generated range.
Jimbo4350 pushed a commit to IntersectMBO/cardano-node that referenced this pull request Nov 8, 2021
Recent ledger changes
(IntersectMBO/cardano-ledger#2530) have
restricted the range for transaction input indices. Since these are
unlikely to be large anyway, restrict the generated range.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants