Skip to content
This repository has been archived by the owner on Feb 12, 2024. It is now read-only.

feat: store blocks by multihash instead of CID #3124

Merged
merged 12 commits into from
Jul 2, 2020

Conversation

achingbrain
Copy link
Member

@achingbrain achingbrain commented Jun 25, 2020

Updates the ipfs-repo dep to a version that stores blocks by multihash instead of CID to support CIDv1 and CIDv0 access to the same block.

New features:

  • Adds a --multihash argument to the cli command ipfs refs local which prints the base32 encoded multihash of each block

BREAKING CHANGES:

  • ipfs.refs.local now returns a v1 CID with the raw codec for every block and not the original CID by which it was added to the blockstore

Updates the `ipfs-repo` dep to a version that stores blocks by
multihash instead of CID to support CIDv1 and CIDv0 access to the
same block.
@achingbrain achingbrain linked an issue Jun 25, 2020 that may be closed by this pull request
@achingbrain
Copy link
Member Author

Supersedes #2522

@achingbrain achingbrain requested a review from hugomrdias June 26, 2020 06:24
@achingbrain
Copy link
Member Author

@aschmahmann could I get a 👍 or 👎 on the cli change please? It would be a shame to ship this and then have go-ipfs implement something different later down the line.

The change is:

  • ipfs refs local used to print out CIDs of blocks in your repo
  • Now it prints v1 raw CIDs instead
  • There's a new --multihash argument to ipfs refs local which prints base32 encoded multihashes instead of the v1 raw CIDs

@aschmahmann
Copy link
Contributor

@achingbrain I'm having trouble finding which issue describes what we agreed on here. This change seems reasonable to me, but mostly because I'm not sure what people are using ipfs refs local for... I recall we use ipfs refs {cid} with the preload nodes but that code path is left unchanged.

Any pointers?

@achingbrain
Copy link
Member Author

The issue & decision is here: ipfs/specs#242 (comment)

Also relevant: ipfs/kubo#6815

@aschmahmann
Copy link
Contributor

Neither of those issues seem to explicitly reference the changes to ipfs refs local. Although I guess it's probably implied by those issues since there may be orphaned (e.g. unreferenced by a pin) blocks that we just can't figure out the CIDs for.

@achingbrain
Copy link
Member Author

I guess it's probably implied by those issues since there may be orphaned (e.g. unreferenced by a pin) blocks that we just can't figure out the CIDs for

Exactly, it's more of a practical implication of dropping the original CID a block was stored by.

packages/ipfs/src/cli/commands/refs-local.js Outdated Show resolved Hide resolved
packages/ipfs/test/cli/refs-local.js Outdated Show resolved Hide resolved
@aschmahmann
Copy link
Contributor

Seems reasonable 👍 from me then. If @hsanjuan or @Stebalien have a second to confirm that'd be great, since they've been more involved with this than I have. However, if they're busy then no complaints from me

Co-authored-by: Hugo Dias <hugomrdias@gmail.com>
@Stebalien
Copy link
Member

@achingbrain that's the planned change for go-ipfs as well.

@achingbrain achingbrain merged commit 03b17f5 into master Jul 2, 2020
@achingbrain achingbrain deleted the feat/store-blocks-by-multihash branch July 2, 2020 12:27
@oed
Copy link
Contributor

oed commented Jul 17, 2020

So does this change mean that all data already stored in an ipfs node will not be retrievable locally when one updates to v0.48?
Is there a data migration strategy that can be used here?

@achingbrain
Copy link
Member Author

No, there should be no change. A blockstore migration takes place when you first start a node running 0.48.0 on a non-migrated repo to convert all the keys so what is currently retrievable should still be retrievable after the upgrade.

@github-actions github-actions bot mentioned this pull request Feb 4, 2022
SgtPooki referenced this pull request in ipfs/js-kubo-rpc-client Aug 18, 2022
Updates the `ipfs-repo` dep to a version that stores blocks by multihash instead of CID to support CIDv1 and CIDv0 access to the same block.

New features:

- Adds a `--multihash` argument to the cli command `ipfs refs local` which prints the base32 encoded multihash of each block

BREAKING CHANGES:

- `ipfs.refs.local` now returns a v1 CID with the raw codec for every block and not the original CID by which it was added to the blockstore

Co-authored-by: Hugo Dias <hugomrdias@gmail.com>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Migration of Blockstore to use multihash instead of CID as key
5 participants