Camera image not loading in live demo

[nserrino]

I followed the instructions here https://www.jetson-ai-lab.com/vit/tutorial_nanoowl.html?=&linkId=100000237007328 on my Jetson device.

ls /dev/video* /dev/video0 /dev/video1

After port-forwarding to my local machine to open the demo in Chrome, the camera feed looks broken:

Here are the logs from my command:
root@jetson001:/opt/nanoowl/examples/tree_demo# python3 tree_demo.py ../../data/owl_image_encoder_patch32.engine /usr/local/lib/python3.8/dist-packages/transformers/utils/hub.py:123: FutureWarning: Using TRANSFORMERS_CACHEis deprecated and will be removed in v5 of Transformers. UseHF_HOME instead. warnings.warn( config.json: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████| 4.42k/4.42k [00:00<00:00, 685kB/s] model.safetensors: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████| 613M/613M [00:31<00:00, 19.5MB/s] preprocessor_config.json: 100%|████████████████████████████████████████████████████████████████████████████████████████████████| 392/392 [00:00<00:00, 77.2kB/s] tokenizer_config.json: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████| 775/775 [00:00<00:00, 703kB/s] vocab.json: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████| 1.06M/1.06M [00:00<00:00, 4.16MB/s] merges.txt: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████| 525k/525k [00:00<00:00, 3.87MB/s] special_tokens_map.json: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████| 460/460 [00:00<00:00, 352kB/s] /usr/local/lib/python3.8/dist-packages/torch/functional.py:504: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at /opt/pytorch/pytorch/aten/src/ATen/native/TensorShape.cpp:3483.) return _VF.meshgrid(tensors, **kwargs) # type: ignore[attr-defined] [01/22/2024-23:13:43] [TRT] [W] Using an engine plan file across different models of devices is not recommended and is likely to affect performance or even cause errors. 100%|███████████████████████████████████████| 338M/338M [00:19<00:00, 18.5MiB/s] INFO:root:Opening camera. [ WARN:0] global /opt/opencv/modules/videoio/src/cap_gstreamer.cpp (1760) handleMessage OpenCV | GStreamer warning: Embedded video playback halted; module v4l2src0 reported: Internal data stream error. [ WARN:0] global /opt/opencv/modules/videoio/src/cap_gstreamer.cpp (888) open OpenCV | GStreamer warning: unable to start pipeline [ WARN:0] global /opt/opencv/modules/videoio/src/cap_gstreamer.cpp (480) isPipelinePlaying OpenCV | GStreamer warning: GStreamer: pipeline have not been created INFO:root:Loading predictor. ======== Running on http://0.0.0.0:7860 ======== (Press CTRL+C to quit) [ WARN:1] global /opt/opencv/modules/videoio/src/cap_v4l.cpp (1004) tryIoctl VIDEOIO(V4L2:/dev/video0): select() timeout. INFO:root:handle_index_get INFO:aiohttp.access:127.0.0.1 [22/Jan/2024:23:16:25 +0000] "GET / HTTP/1.1" 200 235 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36" INFO:root:Websocket connected. INFO:aiohttp.access:127.0.0.1 [22/Jan/2024:23:16:25 +0000] "GET /favicon.ico HTTP/1.1" 404 172 "http://localhost:9000/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36" INFO:root:handle_index_get INFO:aiohttp.access:127.0.0.1 [22/Jan/2024:23:16:57 +0000] "GET / HTTP/1.1" 304 176 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36" INFO:aiohttp.access:127.0.0.1 [22/Jan/2024:23:16:25 +0000] "GET /ws HTTP/1.1" 101 0 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36" INFO:root:Websocket connected. INFO:aiohttp.access:127.0.0.1 [22/Jan/2024:23:16:57 +0000] "GET /ws HTTP/1.1" 101 0 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36" INFO:root:Websocket connected.

[jaybdub]

Hi @nserrino ,

Thanks for reaching out!

Could you try changing CAMERA_DEVICE = 1 here:

nanoowl/examples/tree_demo/tree_demo.py

Line 43 in 60c5d9f

CAMERA_DEVICE = 0

Best,
John