GSoC Infragram.org full-screen UI and video upload Discussion and Planning

[Forchapeatl]

Hi, everyone! This is the Planning Issue for the Infragram.org: full-screen UI and video upload for infragram.org. The project is in collaboration with the Outreachy proposal: Design a new full-screen user interface for Infragram.org . All are open for discussion and improvement.

Process Video Locally

• Allow conversion of entire video.
• Implement video controls on canvas.
• Include another Canvas Element ( second Canvas ).
• Record video on process at second canvas.
• Save processed video file.
• Integrate the above functionalities to with UI of outreachy collaborator. @stephaniequintana

Allow selecting multiple output resolutions as well

• Dynamically change the size of second canvas to fit required resolution.
• Integrate the above functionalities to with UI of outreachy collaborator.

Integrate Teamsys latest camera selection API
Issue #39

• Download Temasys/AdapterJS library.
• Change all instances of the word webRtcOptions.context to webRtcDetectedType
• Remove code if (webRtcOptions.context === "flash")
  Reference 1 Reference 2

Accept multiple webcam resolutions

• Maintain css canvas size.
• Change HTML canvas size to fit requested resolution.
• Integrate the above functionalities to with UI of outreachy collaborator.

Write UI tests for new Infragram UI Jest

• write test for infragram UI functionalities.

Allow zooming/panning of video

• Zooming : Add uScale of uniform vect2 to shader.frag and shader.vert.
• Panning : Add uPan of uniform vect2 to shader.frag and shader.vert .
• Implement above functionalities to infragram drawscreen function.
• Integrate the above functionalities to with UI of outreachy collaborator.

Miscellaneous

• Add option to close webcam input
• Enable image and video stream mirroring
• Add file filters on input to optimize infrared analysis

[TildaDares]

This looks great @Forchapeatl🎉. You're off to a great start!

[TildaDares]

I've created labels for planning and gsoc, so you can add them if you want. I love seeing colors on Github so I like to add labels wherever I can😅.

[jywarren]

Hi @Forchapeatl - this is looking great. I wanted to ask for some clarification on the video portion - are we equally supporting uploaded video (through drag and drop) and live recorded video? Do they use almost all the same functions? If you can just give me a quick summary of this I'd love that!

Also I wanted to say, i know the panning has been challenging. If you want to take a break from it that's fine! The Temasys adapter is a high priority for us, so if you wanted to try taking that one on i think it'd be wonderful -- it'll unlock Safari/iOS functionality, for one -- and should allow for other improvements. What do you think? And/or, am I behind on things at all regarding this? Thank you!

[Forchapeatl]

Hello @jywarren . Thank you . Yes it works on recorded videos and live videos.
Please have a look at it here https://forchapeatl.github.io/infragram/index3.html#

Okay . I will commence with Temasys immediately . Thank you.

[RaviAnand111]

Hey @Forchapeatl @jywarren , If there is anything can I help you too ?

[jywarren]

Sounds great thank you Forcha!

@RaviAnand111 so helpful, thanks! Would you like to test out the version Forcha linked w both uploaded and live videos and let us know how it works?

[Forchapeatl]

Friday 29th July 2022.
Hi @jywarren.
Please here are a few of my worries

1. I have made analysis on this issue Update to Use Temasys/AdapterJS for getUserMedia (brings iOS + Safari compatibility) spectral-workbench.js#172 I tested the Publiclab Spectral Workbench on an Apple phone (IOS 11). (chrome and safari). but all I got was a blank webcam stream. Please am I missing something ?

On Chrome (IOS)

[RaviAnand111]

hey @Forchapeatl @jywarren , I tried to open the link @Forchapeatl gave https://forchapeatl.github.io/infragram/index3.html# , but I am unable to get any video, on uploading I am neither able to see an image nor video. In live video function as well, there is no video on screen just black screen.
I am using firefrox and Manjaro Arch Linux OS.
Thanks

[Forchapeatl]

hello @RaviAnand111 you are off to a great start. It should work fine on Chrome.

[jywarren]

Oh hmm, @Forchapeatl is this possibly the error you're seeing on Safari? (BTW we may want to move to a different more specifically named issue for this?) publiclab/spectral-workbench.js#266

I believe it had been working in Safari as of publiclab/spectral-workbench.js#172 (review), so I hope we can figure out what happened if it is no longer!

[jywarren]

@RaviAnand111 can you confirm that basic webcam access is working on your system with https://webrtc.github.io/samples/src/content/getusermedia/gum/ ? Thank you!

[Forchapeatl]

@jywarren . Please have a look at this comment #39 (comment) .

[RaviAnand111]

hey @jywarren, I tried to open this link https://webrtc.github.io/samples/src/content/getusermedia/gum/ and it seems to work neither on chrome nor on firefox. I tried to experiment with dev tools but got confused myself.

Hey @Forchapeatl, I tried this link https://forchapeatl.github.io/infragram/index3.html# to run on chrome and it is working fine n chrome, the webcam is visible there, but I guess we would need to work on it for firefox.
Thanks

[jywarren]

Hmm, its strange that the webrtc demo/example isn't running -- but if https://forchapeatl.github.io/infragram/index3.html is working then your webcam drivers are probably OK. But the webrtc.github.io implementation should be the canonical/official correct usage, so that's concerning. I wonder if we should hold off for now on testing on Arch Linux, as it's not as common a platform? The original question here was Safari support, as well.

Alternatively @RaviAnand111 if you can find an implementation that is working on chrome and firefox from an official source or a commonly used library, we can try using their syntax. For example, https://p5js.org/examples/dom-video-capture.html is what I had suggested Forcha look at as an example. Does that work for you in multiple browsers @RaviAnand111?

[RaviAnand111]

Hey @jywarren, I tried https://p5js.org/examples/dom-video-capture.html and it is working fine on firefox but not on chrome and chromium browser. I don't know what to conclude from this.

I guess I should try to find some official library which can be used for firefox and chrome on Arch Linux.
I think you are right, Arch Linux (Manjaro) is not a common OS and is used by very few so we can hold off its testing and work on mainstream OS, and by the sidelines I can try to find some resources for Arch Linux and inform you guys if I find any.
how does that sound ?
Thanks

[Forchapeatl]

Hello @RaviAnand111 . No worries. This https://p5js.org/examples/dom-video-capture.html example uses invert filters. It appears the filter intensity styles differently on various browsers. I send gratitude.