Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Storage Deals #117

Merged
merged 22 commits into from
Aug 8, 2019
Merged

Storage Deals #117

merged 22 commits into from
Aug 8, 2019

Conversation

magik6k
Copy link
Contributor

@magik6k magik6k commented Aug 2, 2019

No description provided.

chain/deals/client.go Outdated Show resolved Hide resolved
chain/deals/types.go Outdated Show resolved Hide resolved
chain/deals/types.go Outdated Show resolved Hide resolved
chain/deals/types.go Outdated Show resolved Hide resolved
chain/deals/types.go Outdated Show resolved Hide resolved
@magik6k magik6k force-pushed the feat/deals branch 3 times, most recently from e7382dc to a96fe81 Compare August 6, 2019 12:22
@magik6k magik6k marked this pull request as ready for review August 6, 2019 23:20
if err := f.Close(); err != nil {
return cid.Undef, err
}
commP, err := sectorbuilder.GeneratePieceCommitment(f.Name(), 6)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

😬 We need to get better interfaces than this from the proofs team.

chain/deals/client.go Outdated Show resolved Hide resolved
return
}

// TODO: Review: Not signed?
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mmm, this is a good point. We need a signed deal response from the miner, and we're not getting it right here.

We need to clean up the spec around this.

return err
}
if has {
// TODO: uncomment after deals work
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lol. do deals work now?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's basically 'allow duplicates'. I'd leave this in for now, at least until we implement payments, because it makes debugging really annoying

chain/deals/types.go Outdated Show resolved Hide resolved
lib/bytesink/fifo.go Outdated Show resolved Hide resolved
@@ -0,0 +1,245 @@
package nsbsnet
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is just a straight-up copy of the bitswap network package with the protocol ID prefix changes integrated... right? I'd say we should upstream this ASAP

bitswapNetwork := network.NewFromIpfsHost(host, rt)
// prefix protocol for chain bitswap
// (so bitswap uses /chain/ipfs/bitswap/1.0.0 internally for chain sync stuff)
bitswapNetwork := nsbsnet.NewFromIpfsHost(host, rt, "/chain")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hrm, why do we need to segregate this at all? It seems to me that what we really want is just to use sessions and pre-seed it with a specific peer ID, but keep that all on the same bitswap (one use case here is for ipfs peers to be able to read filecoin chain data natively)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly because blockstore for chain and client are separate, which makes implementing gc easier and is nice for badger gc too.

I think We'll never use bitswap for fetching to client blockstore, so we may be able to use tiered blockstore to virtually merge them.. Or just merge them and deal with gc later

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

using a tiered blockstore seems fine to me. alternatively, what we're doing here can work for now, and seems not too difficult to change moving forward (entangling things should be easier than disentangling them)

Copy link
Member

@whyrusleeping whyrusleeping left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall, this is a good step in the right direction. Before merging, I think we should do a few things:

  • break the long functions up a bit more. I commented on most of the places i think this should happen
  • Don't use the bytepipebuffer thing. I think its just pulling technical debt into the project, and writing the data to a temp file for now and pushing hard on getting better interfaces from the sector builder is the right approach.
  • Ideally we don't have to have separate bitswap instances, I get that its somewhat necessary right now due to datastore separation concerns. At the very least, lets upstream the bitswap network changes and use them here. That should be a pretty uncontroversial change and i think you and steven can get it merged and shipped really quickly.

After that, feel free to merge.

Copy link
Member

@whyrusleeping whyrusleeping left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

quick pass through, LGTM (aware of all the TODOs)

@whyrusleeping
Copy link
Member

oops, i caused merge conflicts by merging #126

@magik6k magik6k merged commit 61a058a into master Aug 8, 2019
@Kubuxu Kubuxu deleted the feat/deals branch May 13, 2020 01:54
nonsense added a commit that referenced this pull request Nov 6, 2020
dumikau pushed a commit to protofire/lotus that referenced this pull request Jun 8, 2023
* add ipc scripts

* resolve validators when returnin subnet actor state

* fix scripts. add docs. new export key method
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants