Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gateway: Try feather for incremental files verifications in gateway #139

Open
Jorropo opened this issue Jan 30, 2023 · 3 comments
Open

gateway: Try feather for incremental files verifications in gateway #139

Jorropo opened this issue Jan 30, 2023 · 3 comments
Labels
P1 High: Likely tackled by core team if no one steps up topic/gateway Issues related to HTTP Gateway

Comments

@Jorropo
Copy link
Contributor

Jorropo commented Jan 30, 2023

The goal is to handle efficiently handle links between the left and middle column, for that I'll experiment by porting https://github.com/Jorropo/go-featheripfs to be used in the gateway code (as well as some modernising).

It is pretty unclear how much breaking I'll have to do. The simplest option is to fork the gateway code (not viable long term). Else I can try to find some commun interface without compromising on speed too much (I don't belive in it).
The other (my preferred) option is to drop go-unixfs from the gateway, this will include challenges (ipld compat), but will allows us to use a server driven interface internally (see ipfs#12).


Done crateria:

  • Have a unixfs client that is is usable to implement the gateway API spec that decodes an IPIP402 stream using a fixed (low) amount of memory for arbitrarily sized files.
@Jorropo Jorropo added need/triage Needs initial labeling and prioritization and removed need/triage Needs initial labeling and prioritization labels Jan 30, 2023
@Jorropo Jorropo self-assigned this Jan 30, 2023
@BigLep
Copy link
Contributor

BigLep commented Jan 31, 2023

By "left and middle" column I assume you're meaning columns 2 and 3 in https://www.notion.so/pl-strflt/Project-Rhea-decentralized-IPFS-gateway-3d5906e7a0d84bea800d5920005dfea6#9d1477c5b8d84790bcbb56c0c569f15a right?

image

@Jorropo Jorropo changed the title Try feather for incremental files verifications in gateway gateway: Try feather for incremental files verifications in gateway Jan 31, 2023
@Jorropo Jorropo added the P1 High: Likely tackled by core team if no one steps up label Jan 31, 2023
@lidel lidel added the topic/gateway Issues related to HTTP Gateway label Mar 3, 2023
@BigLep
Copy link
Contributor

BigLep commented May 18, 2023

2023-05-18 conversation: @Jorropo is going to update the issue description to be very clear about the done criteria. We should spell out the expected inputs and output. The goal is to avoid someone at code review being surprised about what we're doing.

@BigLep
Copy link
Contributor

BigLep commented Jun 13, 2023

Implementation work happening in #347

@Jorropo Jorropo removed their assignment Mar 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
P1 High: Likely tackled by core team if no one steps up topic/gateway Issues related to HTTP Gateway
Projects
No open projects
Status: 🏃‍♀️ In Progress
Development

No branches or pull requests

3 participants