Screen sharing by selection does not work

[shamun]

General screen sharing seems to be working but the screen sharing by selection (see screenshot below) is not working. In Google Chrome Stable or Canary while sharing screen you get a selection dialog window where you can select which screen you want to share or which running application you want to share. That is missing in the current nodewebkit version last tested with 12(alpha).

[warrickhunter]

It seems Chrome's desktop capture API is not available in node-webkit.
Does anyone know if this or a similar API can be made available in node-webkit?
Perhaps make available a collection screens/windows etc with the streamId to pass to getUserMedia().

[shamun]

@warrickhunter: node-webkit works for desktop capture API.

The problem that i mentioned is about the small popup window in my screenshot (white window) you can see. That is only missing in node-webkit.

See the issue #576 where Roger Wang mentioned he is working on it, but there was no status/updates regarding that issue yet.

@rogerwang commented on Dec 3, 2014
Hi @jbescoyez , what do you mean by "window picker"? I'll look into it. Thanks.

[warrickhunter]

@shamun: yes I agree with you, the popup is missing. Just to clarify, I was referring to Chrome's desktop capture api which, in Chrome, is accessed by calling chrome.desktopCapture.chooseDesktopMedia(). chrome.desktopCapture is undefined in node-webkit.
Yes it is possible to capture the whole desktop by making the correct call to navigator.webkitGetUserMedia(), however there's no way that I know of to select a particular window/screen. It would be nice if this were added to node-webkit.
Perhaps there's some other OS level API which can gather information about screens/windows? I'm not involved in the development of node-webkit so I don't know where to start.

[shamun]

@warrickhunter: Roger wang was working on it i think.

have you seen his comment on issue #576 (posted on Dec 3, 2014), maybe he have it solved already but not yet committed for other dependencies issue.

[shamun]

What is the Status of this issue please?

[rogerwang]

This is planned to be fixed in nw13. But I'll try to fix it in nw12.

[shamun]

@rogerwang: thank you very much. looking forward to it. you are genius.

[jtg-gg]

@rogerwang I am half way doing it on OSX platform, shall I finish this ?

[rogerwang]

@jtg-gg please go ahead. thanks. looking forward to your implementation.

[redgecombe]

@rogerwang @jtg-gg
I had started implementing something around this as well for use in our app. My approach was, as suggested above, to add a JS API to allow for enumerating and selecting the windows and screens, rather than implement the chooseDesktopMedia API. I think I'm getting close, but it would be great if there was an officially supported API.

@jtg-gg, were you doing an API, or a full picker window?

Great project by the way, thanks.

[shamun]

@to all:

Many Thanks for your contribution. I want to suggest you guys one thing to consider regarding this issue.

(if its possible and easy for you guys else please ignore it.)

1 - Add the new window picker or running application picker just like Google Chrome/Canary does
2 - But also keep the old screen sharing like right now as it was implemented by Roger wang the reasons are as following

Scenario Storm (real life): Why keep method 1 (old) and method 2 (new):

Method 1 (OLD):

As it is now is very handy too in many scenarios.
when it comes to make applications using Python, Go, Java to launch the NodeWebKit on the fly and without ANY human/user input start the screen share and make applications like TEAMVIEWER or LOGMEIN or VNC

Method 2 (NEW which is upcoming soon):

New feature like Google Chrome, Canary has one disadvantage which i faced with some Bank application requirements.

In some BANK (Belgium banks) i need to make an application just like Roger Wang made as in Method 1, they want to execute our apps (Python, Java, Go application) and then simply start screen sharing which was impossible with Google Chrome, Canary because you need be human/real user clicking what to share (we need to automate it for there ATM Machines).

But Roger Wang made impossible to possible with his excellent Method 1 implementation.

So i think it will be wise programming to keep this both feature's
(Google Chrome, Canary do not listen to anyone properly they keep or cut or remove often, we found nodewebkit is a trusted and safe world for us, in a sense we are not breaking many things which were removed often by Google Chrome, Canary).

let us choose as third party programmers of nodewebkit to choose, which method (1 or 2) we nee to use, maybe for compatibility we can set the method 2 as default then it will be compatible with other Chrome, Canary applications.

Summary:

Therefore, is it possible we can keep the existing screen share as it is and if we need Window picker like Chrome we add some additional friendly parameters or so?

Both way of screen sharing has its own advantages, it will be smart programming if you guys keep as it is now and beside that include the new feature. Again please ignore if its too complicated and time consuming.

[jtg-gg]

@redgecombe I am porting google chrome screen picker UI, so it should behave exactly like chrome's screen picker

[jtg-gg]

I've just made a pull request #3163
@rogerwang please review, and merge if you are ok

[shamun]

Not working yet.

1. Downloaded:

http://dl.nwjs.io/live-build/03-09-2015/dd7d7ad-80ebf09-be948af-459755a-2bdc251-c114bb7/nwjs-v0.12.0-win-x64.zip
(windows 8.1 64-bit.)

2. using nodewebkit launched: https://talky.io/rogerwang-and-jtg-gg

3. using google chrome from another pc connected to: https://talky.io/rogerwang-and-jtg-gg

4. using nodewebkit now clicking "share screen" does nothing

(only from Google chrome i can share screen, but i should be able to do this also from nodewebkit, its not working)

[tommoor]

@shama I believe the API is different in NW therefore that example wouldn't work as it uses a chrome extension to achieve screensharing.

[shamun]

@tommoor: main goal was to use it with talky.

can you please show me an example how to make the talky compatible with nodewebkit. else this feature will be pointless for our case. if not possible we will make our own chrome version. please confirm.

[jtg-gg]

@shamun the api is different, in chrome extension, the javascript will call chrome.chooseDesktopMedia
while in nwjs, it is gui.Screen.chooseDesktopMedia, you can try to make a wrapper from chrome.chooseDesktopMedia to gui.Screen.chooseDesktopMedia

this is the documentation for nwjs
https://github.com/nwjs/nw.js/wiki/Screen#screenchoosedesktopmedia-array-of-desktopcapturesourcetype-sources-function-callback

[shamun]

Please note it works for me the way i wanted just if i use Google Chrome itself. But i cant do the similar with NodeWebkit.

Example with Google Chrome itself:
Step 1: Chrome.exe --app="https://github.com/nwjs"
Step 2: you will notice now all works without re-programming any extensions

Can you please kindly take care about this, so that we do not have to re-program chrome web store extensions?

[shamun]

Then only problem is now i need exactly the same features of:

Chrome.exe --app="https://example.com"

but with nodewebkit features on it such as : frame: true / false, width, height, always-on-top, resizable, toolbar, title thats it, then you can allow anyone to not change there stable/reliable chrome extensions but make it work with nodewebkit?

EDIT (NodeWebKit Fallback Reference): Using InnoSetup and Google Chrome offline installer for 32/64-bit https://support.google.com/installer/answer/126299?hl=en

[DivyaKMenon]

Hi,
I am using Node-Webkit screen-sharing feature. I did the same as mentioned in https://github.com/nwjs/nw.js/wiki/Screen#screenchoosedesktopmedia-array-of-desktopcapturesourcetype-sources-function-callback. Window chooser is working. But when I choose a window, it's always executing the failure callback. And is giving error:
NavigatorUserMediaError {constraintName: "", message: "", name: "InvalidStateError"}
When I searched I found that if I use HTTPS protocol it will work. But can I make my screen-sharing work without https protocol? If so, can anyone please tell me how I can fix it?

[porsager]

I did some testing with NW 0.12.1 and got it working using these constraints.

gui.Screen.chooseDesktopMedia(["window","screen"], function(streamId) {
  var constraints = {
    audio: false,
    video: {
      mandatory: {
        chromeMediaSource: 'desktop',
        chromeMediaSourceId: streamId,
        maxWidth: 1920,
        maxHeight: 1080
      },
      optional:[{
        googTemporalLayeredScreencast: true
      }]
    }
  }

  window.navigator.webkitGetUserMedia(constraints, success, error)
})

[DivyaKMenon]

Ya even i did the same..
I got it now. Thanks, :)

gui.Screen.chooseDesktopMedia(
        ["window","screen"], function(streamId) {
             var vid_constraint = {
                  mandatory: {
                    chromeMediaSource: 'desktop', 
                    chromeMediaSourceId: streamId, 
                    maxWidth: 1920, 
                    maxHeight: 1080,
                    minFrameRate: 1,
                    maxFrameRate: 5
                  }, 
                  optional:[]
                };
                navigator.webkitGetUserMedia({audio:false,video: vid_constraint},
                    function(stream){
                        document.getElementById('video_1').src = URL.createObjectURL(stream);
                        stream.onended = function() { console.log("Ended"); };
                    }, 
                    function(error){
                        console.log('failure',error);
                });
    });

I also have another doubt. When I did the above code, it showed me my own screen in the video tag. But its a screenshot. Not a video screen-sharing. Is it not possible to stream it as video and not screen-shots?

[porsager]

You need to either do a play() on the video after setting the url, or setting autoplay on it.
eg:

document.getElementById('video_1').src = URL.createObjectURL(stream);
document.getElementById('video_1').play()

or in the tag like

<video autoplay></video>

[DivyaKMenon]

Thanks a lot @porsager . Tried that. And it worked :)

[DivyaKMenon]

Hi ,
I tried setting flag : "chromium-args": "--auto-select-desktop-capture-source='Entire screen'" for selecting the screen automatically on initializing media. It's working fine. But my problem is that my screen is flickering. Once I initialize the media, an empty window picker shows and then goes. It happens like in 1 second. Can I enable a flag or anything so that the flickering doesn't happen and the window picker is hidden all the time?

[rogerwang]

This is fixed and released with v0.13.0-alpha0

[mscreenie]

Is this fixed in 12.2? Thanks!

[rogerwang]

no. It's will not be in 0.12. The fix benefits from the change we made in 0.13

[matthewrobb]

What is the performance characteristics of nw.js screen capture? Is it viable for screen mirroring low latency video games?

[metal3d]

https://github.com/nwjs/nw.js/wiki/Screen works well only if I set audio to false.
Audio only works too... but both desktop + audio doesn't work. Any idea on how I can do ? The goal is to capture desktop + mic then allow to save stream somewhere (I will probably use https://github.com/streamproc/MediaStreamRecorder)

[agustinprod]

--auto-select-desktop-capture-source='Entire screen' doesn't work on windows if the user don't have english locale installed, any way to solve this?

[agustinprod]

I found a way to capture user desktop without prompt on all platforms (tested on 0.12.2):

var sourceId = /darwin/.test(process.platform) ? 'screen:' + gui.Screen.screens[0].id : 'screen:0';

navigator.webkitGetUserMedia({
    audio: false,
    video: {
        mandatory: {
            chromeMediaSource: 'desktop',
            chromeMediaSourceId: sourceId,
            maxWidth: 2000,
            maxHeight: 1000
        },
        optional:[]
    }
    }, function(screenStream) {

    },
    function(err) {
        console.error('getUserMedia failed!' , err, new Date)
    }
);

[sn0opr]

Is it possible to computre the screen and stream it to and RTMP server for example ?

[bhavyaw]

@killgt ...Many Thanks. This Works Perfectly!!

[Ajeey]

@shamun Did you succeed in making sharing working( https://talky.io) with node-webkit and chrome?