Skip to content
This repository has been archived by the owner on Oct 23, 2022. It is now read-only.

test: common fs and mem pinstore tests #360

Merged
merged 25 commits into from
Sep 4, 2020

Conversation

koivunej
Copy link
Collaborator

@koivunej koivunej commented Sep 4, 2020

Continues the long chain of pin PRs from #358. Refactors the previously custom tempdir hack and system polluting repo tests into a macro based which can be re-generated for each datastore implementation. Of course this revealed issues on both fs and mem pinstores but they were more about me not understanding the rules before three implementations :) Lets fix later if we find out more inconsistencies.

The macro approach is the only one I can see, looks pretty bad and if there are errors those will be even uglier.

Joonas Koivunen and others added 25 commits September 4, 2020 17:32
this is implemented using a blockstore alike sharding implementation.
for recursive pins, a json Vec<String> is stored and used mostly
linearly (eh, something to improve). for direct pins, empty files are
used. the pin files have different file extensions.

lets see if this works in windows as well.
this allows using the owned permit and more importantly moving it to the
threadpool thread, which is not necessarily bound to the task lifetime.
there is just one permit per datastore at the moment, but making this
per cid for example shouldn't be too difficult.

having a global lock will be needed later on, not sure how to implement
it though.
358: feat: fs pinstore r=aphelionz a=koivunej

Adds filesystem based pin store. See adec0a8 for TL;DR.

Builds upon rs-ipfs#357. Only last N commits are new after including 159a1c5. Even windows seems ok. base32lower is used to encode the cid names.

TODO:
- [x] at least the tests written for ipfs::repo::mem but will post another PR

The differences to other impls are not explained as I am not sure if I am up to date with the other implementations.



Co-authored-by: Joonas Koivunen <joonas@equilibrium.co>
my first attempt with macro generated tests: lots of quite confusing to
debug compiler errors but the final outcome looks ok.
@koivunej koivunej force-pushed the align_fs_and_mem_pinstore_tests branch from 11ab6dc to 2b82d88 Compare September 4, 2020 15:30
Copy link
Contributor

@aphelionz aphelionz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bors r+

@koivunej koivunej marked this pull request as ready for review September 4, 2020 15:49
@bors
Copy link
Contributor

bors bot commented Sep 4, 2020

Build succeeded:

@bors bors bot merged commit 2365748 into rs-ipfs:master Sep 4, 2020
@koivunej koivunej mentioned this pull request Sep 5, 2020
@koivunej koivunej deleted the align_fs_and_mem_pinstore_tests branch September 24, 2020 12:55
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants