refactor: on-demand remote pin status checks #1903
Conversation
This is bare minimum to restore remote pinning with rate-limited Pinata. Details in #1900 (comment) - The expensive 'ipfs pin remote service ls --stat' and autopin MFS policy is executed only on Settings screen - Files screen does not make any requests to remote pinning service anymore. - Remote pin status check is triggered by opening "Set pinning" modal via context menu or the pin icon, or by clicking on "Pin Status" column header. – We cache aggressively and assume pins on remote service are not changed by other means. The cache lives in memore, reloading the webui will purge cached states. - Clicking on "Pin Status" header bypassess remote pin status cache and forces online status re-check for all items in current directory. - Errors returned by pinning service are now correctly reported to the user. The CLI and ipfs-webui users will now see the same error message.
src/bundles/pinning.js
Outdated
| notPins.delete(pinCid) | ||
| adds.push({ id: `${service.name}:${pinCid}`, ...pin }) | ||
| try { | ||
| // TODO: execute remote.ls with +1m backoff per service when response Type == "error" and Message includes "429 Too Many Requests" |
There was a problem hiding this comment.
ℹ️ Curiously, hardcore caching is enough to send less than 6 requests per minute (I was not able to trigger rate-limiting at Pinata).
👉 What happens on 429 Too Many Requests?
If the user overdoses caffeine and pins more than 6 things in one minute, they will get a meaningful error via notify.js (yellow Toaster stating):
Unable to set Pinning at Pinata: Error: reason: "Rate limit exceeded", details: "You have exceeded the API rate limit. Wait a minute and slow down your requests.": 429 Too Many Requests
Having this, I decided to not implement any throttling on the client. There is no point in ipfs-desktop or go-ipfs trying to hide the throttling by visually slowing pin checks down – that would give an impression of "IPFS Desktop being slow" while it is remote HTTP server's limitation.
But it could be better!
I agree! If anyone is interested in improving remote pinning implementation, a good starting point is #1752 where existing technical debt is described, and a clean way to rewrite it. Figuring out UX for handling rate-limiting (HTTP 429) would be a small part of that work (e.g. option for user to "dismiss & keep trying in background")
this leverages createCacheBundle (src/bundles/index.js) and persistActions (src/bundles/pinning.js) for persisting remote pin states across page reloads new users should not see any functional difference, old users may need to manually go to "Set pinning" to trigger status fetch, but that is the best we can do and a fair compromise, given that before this remote pins at Pinata did not work at all due to throttling (#1900)
we added two variables, and old key had none. using a different key ensures users use english instead of old translation
|
Did as many tests I could. Fast-tracking this, as we need to restore remote pinning on Pinata in go-ipfs 0.13 and ipfs-desktop 0.19. |
This is bare minimum to restore remote pinning with rate-limited Pinata and close #1900
Not a perfect solution, but a fast-tracked fix to restore pinning functionality at all (right now Pinata is effectively broken in webui and ipfs-desktop – details in #1900 (comment)).
Key changes:
pinningbundle state with remote pin states withcreateCacheBundle(src/bundles/index.js) andpersistActions(src/bundles/pinning.js), which store data in browser's IndexedDB (usesmoney-clipas cache backend)ipfs pin remote service ls --statchecks are executed only on Settings screen – Files does not need them.