You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The testing of the video decoders is in a catastrophic state. Almost all tests for read_video, VideoReader or the GPU decoder are being skipped on our CI, for one reason or another.
The GPU tests are never run because we never build with support for the nvidia lib. (I guess it's sort of OK considering the state of that GPU decoder itself... we should probably just remove that decoder entirely.)
Failed to load video Python extension: '/lib64/libstdc++.so.6: version `CXXABI_1.3.8' not found (required by /opt/conda/envs/ci/lib/./libopenh264.so.5)'
Although for some Python versions, we don't even get an error message.
The tests in test_io.py are being run. They only test read_video on the pyav backend AFAICT.
The lack of test for those decoders isn't new and has been going on for a LONG time. In #7646 we already noticed part of the problem.
Part of the reason this hasn't been flagged earlier is because our tests are setup incorrectly: they don't tell you if they can't be run! If for some reason the ops cannot be loaded, or if pyav isn't present, or if there was a problem with the ffmpeg build, then the tests are simply skipped. I understand this is because we can't be testing those APIs on all platforms, but one bad consequence is that when the build fails (and it FAILS), then the tests are just skipped and CI is still happy with that, so we never know. We think the tests are green when in reality they were never run.
So, beyond bringing those tests back, we have to find a way to be properly alerted when those tests aren't run when we expect them to be run.
On #8161 we have decided to at least stop pretendying, and explicitly excluded those files from being tested until the situation gets resolved. We should probably start with bringing back the read_video tests.
The text was updated successfully, but these errors were encountered:
The testing of the video decoders is in a catastrophic state. Almost all tests for
read_video
,VideoReader
or the GPU decoder are being skipped on our CI, for one reason or another.The GPU tests are never run because we never build with support for the nvidia lib. (I guess it's sort of OK considering the state of that GPU decoder itself... we should probably just remove that decoder entirely.)
The other tests (
read_video
andVideoReader
) are never run either. @pmeier's [DEBUG] video extension not available in CI #8157 suggests that the video ops cannot be loaded with the following error:Although for some Python versions, we don't even get an error message.
test_io.py
are being run. They only testread_video
on the pyav backend AFAICT.The lack of test for those decoders isn't new and has been going on for a LONG time. In #7646 we already noticed part of the problem.
Part of the reason this hasn't been flagged earlier is because our tests are setup incorrectly: they don't tell you if they can't be run! If for some reason the ops cannot be loaded, or if pyav isn't present, or if there was a problem with the ffmpeg build, then the tests are simply skipped. I understand this is because we can't be testing those APIs on all platforms, but one bad consequence is that when the build fails (and it FAILS), then the tests are just skipped and CI is still happy with that, so we never know. We think the tests are green when in reality they were never run.
So, beyond bringing those tests back, we have to find a way to be properly alerted when those tests aren't run when we expect them to be run.
On #8161 we have decided to at least stop pretendying, and explicitly excluded those files from being tested until the situation gets resolved. We should probably start with bringing back the
read_video
tests.The text was updated successfully, but these errors were encountered: