Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

storage: more. #279

Merged
merged 14 commits into from
Nov 11, 2021
Merged

storage: more. #279

merged 14 commits into from
Nov 11, 2021

Commits on Oct 30, 2021

  1. storage: many more adapters; a new filesystem storage implementation;…

    … benchmark helper functions; and actual benchmark tables, runnable, which compare several storage implementations.
    warpfork committed Oct 30, 2021
    Configuration menu
    Copy the full SHA
    67c65ae View commit details
    Browse the repository at this point in the history
  2. storage: extract readmes about adapters.

    Mostly deredundantizing, but made a few improvements as well.
    warpfork committed Oct 30, 2021
    Configuration menu
    Copy the full SHA
    4d9e5f2 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    125c81f View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    ab1d2c7 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    fd20a2f View commit details
    Browse the repository at this point in the history
  6. storage: update fsstore defaults to be the same as flatfs as used by …

    …default configuration of go-ipfs.
    
    Include links to where the relevant ipfs code can be found.
    (It's not all in very obvious places, and not all in the go-ipfs repo.)
    
    Extracted some of the b32 encoder to a package function...
    but, although this does avoid some repeated initialization,
    it was all CPU stuff.  Benchmarks indicate that it made no real
    difference.  So it's mostly a stylistic change.
    warpfork committed Oct 30, 2021
    Configuration menu
    Copy the full SHA
    6ea5215 View commit details
    Browse the repository at this point in the history

Commits on Nov 2, 2021

  1. Configuration menu
    Copy the full SHA
    532c290 View commit details
    Browse the repository at this point in the history

Commits on Nov 8, 2021

  1. don't use nested b.Run calls

    One level with fmt.Sprintf is enough for what we need,
    and is also easier to follow.
    mvdan committed Nov 8, 2021
    Configuration menu
    Copy the full SHA
    9269140 View commit details
    Browse the repository at this point in the history

Commits on Nov 10, 2021

  1. storage benchmarks: use testing package's tmpdirs.

    ... which is actually important, not just stylistic: this also makes
    the benchmarks much more correct, because we were previously
    doing the cleanup work on the clock!  Now the cleanup is done by
    the test system, off the clock.
    
    This fixes results to actually be, um, correct.  *facepalm*.
    warpfork committed Nov 10, 2021
    Configuration menu
    Copy the full SHA
    f020fd6 View commit details
    Browse the repository at this point in the history
  2. linking: check for io.Closer.

    The storage APIs now require this and caution of potential resource
    leakage otherwise.
    warpfork committed Nov 10, 2021
    Configuration menu
    Copy the full SHA
    ba94a80 View commit details
    Browse the repository at this point in the history
  3. storage benchmarks: it would be preferable to actually use the shardi…

    …ng parameters that are in fact "default" when comparing these.
    warpfork committed Nov 10, 2021
    Configuration menu
    Copy the full SHA
    888bbea View commit details
    Browse the repository at this point in the history

Commits on Nov 11, 2021

  1. Configuration menu
    Copy the full SHA
    893d825 View commit details
    Browse the repository at this point in the history
  2. storage: switch storage.StreamingReadableStorage.GetStream to return …

    …an io.ReadCloser instead of just io.Reader.
    
    This seems more likely to encourge good code that doesn't leak
    resources than just admonishing people to check via a comment.
    
    On the downside, sometimes this requires implementations to wrap stuff
    in no-op closers.  The trade is probably worth it.
    
    The code in LinkSystem which uses these things still doesn't change.
    It's still using the `linking.Block*Opener` functions, which still
    return just io.Reader.  And we're considerably less free to change
    those, because there are already quite a few implementations of those
    types in the wild, and we don't want to break them.
    warpfork committed Nov 11, 2021
    Configuration menu
    Copy the full SHA
    60b0f8c View commit details
    Browse the repository at this point in the history
  3. storage: avoid io.NopCloser; it was relatively recently introduced.

    (I think our policy is roughly "support the last two significant go
    versions", and by that, we might technically be allowed to use this,
    but, eh; there's minimal cost to being conservative today.)
    warpfork committed Nov 11, 2021
    Configuration menu
    Copy the full SHA
    e99bcd1 View commit details
    Browse the repository at this point in the history