Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

No Filestore On Storage Client #107

Merged
merged 4 commits into from
Feb 4, 2020

Conversation

hannahhoward
Copy link
Collaborator

Goals

Remove the need to write a temporary file for the piece on the storage client, who likely does not want to commit extra hard drives space

Implementation

  • use an updated version of go-car that can calculate an estimate of car file size without actually writing the car
  • calculate the car file size without writing
  • write the actual car to an OS.Pipe in a go routine, which is then read when generating commP by the sector builder
  • this avoids writing a temporary file other than the one used by OS.Pipe

Copy link
Contributor

@laser laser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you have my blessings

@hannahhoward hannahhoward force-pushed the feat/no-filestore-on-storage-client branch from b40c0e5 to 391c11b Compare February 4, 2020 22:04
@codecov-io
Copy link

Codecov Report

Merging #107 into master will increase coverage by 0.59%.
The diff coverage is 64.87%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #107      +/-   ##
==========================================
+ Coverage   34.71%   35.29%   +0.59%     
==========================================
  Files          36       36              
  Lines        2706     2726      +20     
==========================================
+ Hits          939      962      +23     
+ Misses       1610     1604       -6     
- Partials      157      160       +3
Impacted Files Coverage Δ
storagemarket/impl/provider.go 0% <0%> (ø) ⬆️
storagemarket/impl/client.go 0% <0%> (ø) ⬆️
storagemarket/impl/provider_states.go 0% <0%> (ø) ⬆️
storagemarket/impl/client_utils.go 33.9% <0%> (+1.65%) ⬆️
pieceio/pieceio.go 81.97% <75%> (-2.4%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 87675b3...391c11b. Read the comment docs.

support commP generation with no intermediate filestore via OS.Pipe -- can be used on storage client
if not storage provider
Remove the need for a filestore in the client by using os.Pipe based piece commitment generation
defer solving writing all cid locations to a seperate ticket
@hannahhoward hannahhoward force-pushed the feat/no-filestore-on-storage-client branch from 391c11b to 0e847a5 Compare February 4, 2020 22:08
@hannahhoward hannahhoward merged commit f3ab53a into master Feb 4, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants