Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

@uppy/transloadit: implement Server-sent event API #4098

Merged
merged 11 commits into from
Jul 13, 2023
Merged

@uppy/transloadit: implement Server-sent event API #4098

merged 11 commits into from
Jul 13, 2023

Conversation

aduh95
Copy link
Contributor

@aduh95 aduh95 commented Sep 12, 2022

Could replace Socket.io to get Assembly status update without HTTP polling. (See https://developer.mozilla.org/en-US/docs/Web/API/EventSource, and https://caniuse.com/eventsource)

@arturi arturi requested a review from mifi September 26, 2022 23:20
Copy link
Contributor

@mifi mifi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure why I was requested to review this, as it doesn't seem to be finished, but it seems like a good idea if websockets are hard to get working.
should we remove websockets code or do we need to support both at the same time?

packages/@uppy/transloadit/src/Assembly.js Show resolved Hide resolved
@aduh95 aduh95 marked this pull request as ready for review January 20, 2023 15:53
@arturi arturi self-requested a review January 26, 2023 15:38
@arturi arturi requested a review from mifi February 2, 2023 16:10
}

if (e.data.startsWith('assembly_upload_finished:')) {
const file = JSON.parse(e.data.slice('assembly_upload_finished:'.length))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

try/catch JSON.parse in case of an error?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure it'd be useful, that would only happen if there's a bug on Transloadit side, at which point throwing an error seems like the correct behavior.

@arturi
Copy link
Contributor

arturi commented Feb 2, 2023

Would it make sense to add a setTimeout to do fetchStatus if no events were emitted from SSE for X seconds?

packages/@uppy/transloadit/src/Assembly.js Outdated Show resolved Hide resolved
packages/@uppy/transloadit/src/Assembly.js Show resolved Hide resolved
packages/@uppy/transloadit/src/Assembly.js Outdated Show resolved Hide resolved
packages/@uppy/transloadit/src/Assembly.js Outdated Show resolved Hide resolved
packages/@uppy/transloadit/src/Assembly.js Outdated Show resolved Hide resolved
@aduh95 aduh95 requested a review from arturi July 10, 2023 14:38

this.#sse.addEventListener('assembly_result_finished', (e) => {
const [stepName, result] = JSON.parse(e.data)
this.emit('result', stepName, result)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't it be this?

Suggested change
this.emit('result', stepName, result)
this.emit('transloadit:result', stepName, result)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

socket.on('assembly_result_finished', (stepName, result) => {
this.emit('result', stepName, result)
if (!this.status.results[stepName]) {
this.status.results[stepName] = []
}
this.status.results[stepName].push(result)
})

}

if (e.data === 'assembly_uploading_finished') {
this.emit('executing')
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
this.emit('executing')
this.emit('transloadit:assembly-executing')

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is copied from

socket.on('assembly_uploading_finished', () => {
this.emit('executing')
})

Changing the event would be a breaking change, right?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gotta be careful here indeed, as some of the events are Assembly internal events, not Uppy Core event emitter, it's confusing.

}

if (e.data === 'assembly_upload_meta_data_extracted') {
this.emit('metadata')
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

internal only?

Suggested change
this.emit('metadata')
// used only internally to do X
this.emit('metadata')

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No idea, I've copied it from

socket.on('assembly_upload_meta_data_extracted', () => {
this.emit('metadata')
this.#fetchStatus({ diff: false })
})

@mifi
Copy link
Contributor

mifi commented Jul 13, 2023

LGTM except I'm a bit confused why some events are handled in this.#sse.addEventListener('message'), e.g. assembly_uploading_finished and assembly_upload_meta_data_extracted, while others in their own eventListeners, e.g. this.#sse.addEventListener('assembly_upload_finished')? is there a good reason why everything isn't just in the message event? should it be documented why?

@aduh95
Copy link
Contributor Author

aduh95 commented Jul 13, 2023

The upstream is sending events, sometimes as 'message' (which is the default), sometimes as a custom event. But event must send data with them, while Transloadit.com could work around this by sending an empty string or a dummy one, sending it as message sounds like a cleaner way of doing things.

should it be documented why?

Yes, on Transloadit side. We're waiting to have a successful implementation with Uppy before officialising the API (after that, it will be quite hard to make breaking changes if we have to)

@aduh95 aduh95 merged commit 9ac5842 into main Jul 13, 2023
@aduh95 aduh95 deleted the eventsource branch July 13, 2023 15:18
@github-actions github-actions bot mentioned this pull request Jul 13, 2023
github-actions bot added a commit that referenced this pull request Jul 13, 2023
| Package                | Version | Package                | Version |
| ---------------------- | ------- | ---------------------- | ------- |
| @uppy/aws-s3-multipart |   3.5.0 | @uppy/locales          |   3.2.3 |
| @uppy/box              |   2.1.2 | @uppy/onedrive         |   3.1.2 |
| @uppy/companion        |   4.7.0 | @uppy/provider-views   |   3.4.0 |
| @uppy/companion-client |   3.2.1 | @uppy/react            |   3.1.3 |
| @uppy/core             |   3.3.1 | @uppy/status-bar       |   3.2.2 |
| @uppy/dashboard        |   3.4.2 | @uppy/transloadit      |   3.2.0 |
| @uppy/dropbox          |   3.1.2 | @uppy/utils            |   5.4.1 |
| @uppy/google-drive     |   3.2.0 | uppy                   |  3.12.0 |

- @uppy/transloadit: fix error message (Antoine du Hamel / #4572)
- @uppy/provider-views: add support for remote file paths (Mikael Finstad / #4537)
- @uppy/transloadit: implement Server-sent event API (Antoine du Hamel / #4098)
- @uppy/aws-s3-multipart: add support for signing on the client (Antoine du Hamel / #4519)
- @uppy/react: allow `id` from props (Merlijn Vos / #4570)
- @uppy/aws-s3-multipart: fix lint warning (Antoine du Hamel / #4569)
- @uppy/status-bar: listen to `upload` event instead of button click (Antoine du Hamel / #4563)
- @uppy/aws-s3-multipart: fix support for non-multipart PUT upload (Antoine du Hamel / #4568)
- @uppy/companion: fix esm imports in production/transpiled builds (Dominik Schmidt / #4561)
- @uppy/locales: fix expression and spelling errors in es_ES (Rubén / #4567)
- meta: upgrade dev dependencies (dependabot\[bot\])
- meta: Don't use triage label (Artur Paikin / #4552)
- meta: update Cypress (Antoine du Hamel / #4562)
- @uppy/box,@uppy/companion,@uppy/dropbox,@uppy/google-drive,@uppy/onedrive,@uppy/provider-views: Load Google Drive / OneDrive lists 5-10x faster & always load all files (Merlijn Vos / #4513)
- @uppy/locales: Add missing pt-BR locales for ImageEditor plugin (Mateus Cruz / #4558)
Murderlon added a commit that referenced this pull request Jul 25, 2023
* main:
  @uppy/aws-s3-multipart: refresh file before calling user-defined functions (#4557)
  Release: uppy@3.13.0 (#4595)
  Add i18n to CONTRIBUTING.md (#4591)
  Add VirtualList to ProviderView (#4566)
  @uppy/provider-views: fix race conditions with folder loading (#4578)
  @uppy/status-bar: fix ETA when status bar is installed during upload (#4588)
  @uppy/provider-views: fix infinite folder loading  (#4590)
  examples/aws: client-side signing (#4463)
  Bump word-wrap from 1.2.3 to 1.2.4 (#4586)
  e2e: increase `requestTimeout` to 16s (#4587)
  @uppy/locales: update zh_TW translation (#4583)
  @uppy/aws-s3-multipart: fix crash on pause/resume (#4581)
  @uppy/aws-s3-multipart: do not access `globalThis.crypto` on the top-level (#4584)
  Release: uppy@3.12.0 (#4574)
  @uppy/transloadit: fix error message (#4572)
  @uppy/provider-views: add support for remote file paths (#4537)
  @uppy/transloadit: implement Server-sent event API (#4098)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants