If the video resolution comes back 16:9 it doesn't work

[ryansit]

If the video resolution comes back 16:9, or something not 4:3 then the face recognition doesn't work.

I am able to detect the resolution and change the initial canvas size before I start tracking, but that doesn't help.

[auduno]

Hi, just out of curiosity, did you figure out if this was a bug in clmtrackr, or was it something else?

[ryansit]

So in the current sample code it doesn't take into consideration the possibility of a different aspect ratio. In the case where the video is returned at 16:9 or something else then I think what is happening is the tracking dimensions somewhere is off, so it doesn't lock onto the face correctly, because it is trying to match on some warped image.

There are a number of solutions, but in my solution I listen for the loadedmetadata event, then once I have the actual working video dimensions, I changed the video and overlay sizes to match the ratio of the video. The ratio thing happens more on 1) Android is 1:1, and 2) some newer Windows laptops that default to 16:9.

Some other handy things is setting up navigator.mediaDevices.enumerateDevices so you can choose your cam, could be helpful for Chrome on Android. Plus you can now use the more standard navigator.mediaDevices.getUserMedia that will get everything working in https://developer.apple.com/safari/technology-preview/ and the latest Chrome recommends it too now.

[jitinl]

Thanks for this, changing the video and overlay sizes on the video's loadedmetadata function, and then starting clmtracker works very well for this issue. In my experience the browser doesn't always follow the size/resolution specified in getUserMedia, so that is not reliable.

  video.onloadedmetadata=function(){
        var w = video.videoWidth;
        var h = video.videoHeight;
        video.height=h;
        video.width=w;
        drawCanvas.height = h;
        drawCanvas.width = w;
        ctracker.init(pModel);
        ctracker.start(video);
        drawLoop();
    }

[ryansit]

Yes it actually doesn't know the real resolution until loadedmetadata. When the video ends up 4:3 then it doesn't really matter because clmtrackr downscales and it stays at 4:3 anyways. But if it is some other aspect then clmtrackr loads as 4:3, then once the actual video is started it switches to another aspect like 16:9 and clmtrackr still tries to track in a 4:3.

[auduno]

Thanks for the report, I'll have a look at updating clmtrackr examples to support other aspect ratios over the next weeks.

[auduno]

Hi, I've now updated the examples in dev branch to handle any camera aspect ratio. I've done some testing of it, but I don't have access to a device with camera aspect ratio other than 4:3, so if you have a chance to test it out and let me know if it works, that would be great.

[jitinl]

Hi,

I tested one of the new examples in the dev branch (facedeform.html), and it doesn't seem to work on my phone in portrait orientation (probably 3:4). It works fine in landscape (4:3) and also on my desktop. I haven't got a chance to debug why this is happening yet.

Another thing - if I open the page while in portrait mode (3:4), and then switch to landscape, and click on start - it fails to work even in 4:3. If however I open the page while I am in landscape mode (4:3), and stay in landscape mode, it works fine.
Maybe you are checking the height/width on page load, so if the video dimensions change after the page has loaded, but before the user presses start you don't register that change.

Update: I checked out the issue, and the problem is with the CSS styling for the #videoel, #overlay, #webgl elements. The width and height specified in the CSS take precedence over the vid.width and vid.height, etc. properties. Removing the width and height for these elements from the CSS fixes the issue.

Thanks!

[auduno]

Thanks for the update and debugging! I'll fix the CSS issue as soon as possible, will also have a look at handling dynamic proportions.

[jitinl]

Sure, I can also work on that and make a PR if you want.

[auduno]

If you have time to make a PR that would be awesome!

[jitinl]

Okay, thank you. I will do it this weekend.

[jitinl]

Hi, I made a small pull request in this regard - #121
Thanks.

[auduno]

The examples in the dev branch now also handle dynamic resizing of camera input, e.g. due to device reorientation or similar, so I'll consider this issue closed.