simplify complex usage of crates-index

[ToBinio]

I believe there should be an easier way to use sparse::make_cache_request and sparse::parse_cache_reqsponse because as the examples show it currently is a bit weird.

to be exact some update_crate() function which simply does the request for you.

especially when thinking about adding some generic/abstract index (#128 & #75) something like this would be required.

I could also imagine moving this functionality later or directly implementing this into the abstract index.

note:
I already kinda opened this discussion before #127

[Byron]

I see where you imagine this journey to go and am happy to support it - it makes sense and should be quite achievable, but hope you can update the title to maybe encapsulate the bigger picture.

To my mind, since #128 and #75 were mentioned, this ideation thread is really about…

• being able to have 'the one' index that is the one cargo would use as per its configuration
• one can make crate-queries without having to care about the underlying implementation
• ideally, fuzzy-matching is built-in or at least easily achievable

Regarding fuzzy-name searches, I have just added another comment that proposes a primitive iterator to help with name generation and solve an immediate need while giving a lot of control to the caller (e.g. crate_index::Names::new("the-crate-I-look-for").take(10).find(|name| index.crate_(name)) to limit the amount of work done).

Regarding abstraction, the current implementation particularly of the SparseIndex is wonderful as it dodges the async question entirely with just one dependency. Otherwise it's the question of sync or async, then it's the question of which framework to use, and different applications have different preferences that, in all honesty, I don't want to maintain here either.

If there should be an abstraction, I think it should be a trait, but that trait also has to then make a choice of async or sync implementations, probably it needs both to satisfy SparseIndex realistically. Error handling would also have to be integrated, either with boxed errors or with a generic one. It's solvable, but its complex as well.

Thus I recommend to go with primitives first, and find a solution for #128, and further find a way to abstract fuzzy matching (admittedly, I am in love with the NamePermutationIterator idea). From there, I'd go with concrete implementations of the interactions with the SparseIndex to show what it takes. These should go into their own crates, like crates-index-ureq, crates-index-requwest and so on, and from there one should be able to see a couple of patterns that then guide the creation of the Index trait that could reasonably well abstract over GitIndex and SparseIndex functionality. Probably it would even be feasible for each of these crates-index-<impl> crates to have their own abstraction, be it a trait or a concrete type, tailored to what makes sense there.

These crates-index-<impl> crates could be maintained here if there would be code-owners for these, but maybe they should just be owned by the community to keep traffic and issues away from here. In any case, this crate would contain the primitives that these community crates can use.

Having said that, I don't see such an Index trait emerging here anytime soon, but encourage experimentation with community crates. Maybe one of these then gets transferred back to this repository once it's clear that it works well enough and the API has settled.

[ToBinio]

I couldn't think of a better title...

How realistic do you think such a community crate would be without any "help" from this crate? Because I don't think the change of something like this happening is very big.
I myself would love to create one but I don't think I am ready for this task and I know how quickly I jump between projects.
so maybe creating ureq and request as "official" ones would be best.
but this is something for the future...

About using the correct index based on the config. Do you think this should also be handled in separate crates?
if not how would you image the API?
either it could simply be a static function that gets a path and returns which one to use.
or it could return an enum with contains a SparseIndex and GitIndex variants and could already implement some generic functions e.g. getCrate(). But at that point, we are basically back to a generic Index implementation just without the HTTP part.

[Byron]

About using the correct index based on the config. Do you think this should also be handled in separate crates?
if not how would you image the API?
either it could simply be a static function that gets a path and returns which one to use.

Yes, this functionality should be coming from this crate. It should be as simple as possible, probably providing the local path if it exists along with the variant of index that cargo would use. If that requires executing cargo to check its version, that must definitely be opt-in (avoid running programs unless the caller knows this can happen). It definitely shouldn't create an instance of the index - this has to be deferred to the crate that wants to provide some 'generic' index.

---

This means for this crate to support everything else, I think what's needed is two primitives

• a function to tell you which index type cargo would chose
  • I think this is also needed for a sub-set of crates-index users - many will be fine making a choice and using one or the other. Knowing which index cargo would use would allow to reuse more local state and possibly work better in offline mode situations as one would use the most frequently used index.
• an iterator for possible names of a crate to support creating 'fuzzy' lookups