[🐛 Bug]: Not all test execution session is recorded as mp4

[novalagung]

What happened?

I'm using docker selenium and enabling the s3 auto upload. it works great. however recently I found out that not all sessions are being recorded as mp4. I was checking some failing test case executions, somehow the generated mp4 video is missing. Not all failing test cases, only a small number of them.

My question is, what is the potential root cause of it? Is there a particular type of failing test execution error that can cause the video not to be generated?

Command used to start Selenium Grid with Docker (or Kubernetes)

Docker

version: "3"
services:
  chrome:
    image: selenium/node-chrome:4.18.1-20240224
    shm_size: 2gb
    depends_on:
      - selenium-hub
    environment:
      - SE_EVENT_BUS_HOST=selenium-hub
      - SE_EVENT_BUS_PUBLISH_PORT=4442
      - SE_EVENT_BUS_SUBSCRIBE_PORT=4443
      - SE_NODE_MAX_SESSIONS=10
      - SE_VNC_NO_PASSWORD=1
      - SCREEN_WIDTH=1920
      - SCREEN_HEIGHT=1080
    restart: always

  chrome_video:
    image: selenium/video:ffmpeg-6.1-20240224
    volumes:
      - ./videos:/videos
      - ./rclone.conf:/home/seluser/.config/rclone/rclone.conf
    depends_on:
      - chrome
    environment:
      - DISPLAY_CONTAINER_NAME=chrome
      - SE_VIDEO_FILE_NAME=auto
      - SE_VIDEO_UPLOAD_ENABLED=true
      - SE_VIDEO_INTERNAL_UPLOAD=true
      - SE_UPLOAD_DESTINATION_PREFIX=s3://mybucket
      - RCLONE_CONFIG_S3_PROVIDER=AWS
      - RCLONE_CONFIG_S3_TYPE=s3
      - RCLONE_CONFIG_S3_ENV_AUTH=true
      - RCLONE_CONFIG_S3_REGION=
      - RCLONE_CONFIG_S3_ACCESS_KEY_ID=
      - RCLONE_CONFIG_S3_SECRET_ACCESS_KEY=
      - UPLOAD_OPTS=--s3-no-check-bucket
    restart: always

  selenium-hub:
    image: selenium/hub:4.18.1-20240224
    container_name: selenium-hub
    ports:
      - "4442:4442"
      - "4443:4443"
      - "4444:4444"
    restart: always

Relevant log output

### selenium-grid:

00:38:16.517 INFO [LocalDistributor.newSession] - Session created by the Distributor. Id: 2f24e30989e72299b3eae7009a4775e6
00:38:25.843 INFO [LocalSessionMap.lambda$new$0] - Deleted session from local Session Map, Id: 2f24e30989e72299b3eae7009a4775e6
00:38:25.843 INFO [GridModel.release] - Releasing slot for session id 2f24e30989e72299b3eae7009a4775e6

### selenium-grid-chrome:

00:38:16.515 INFO [LocalNode.newSession] - Session created by the Node. Id: 2f24e30989e72299b3eae7009a4775e6, Caps: Capabilities {acceptInsecureCerts: false, browserName: chrome, browserVersion: 122.0.6261.69, chrome: {chromedriverVersion: 122.0.6261.69 (81bc525b6a36..., userDataDir: /tmp/.org.chromium.Chromium...}, fedcm:accounts: true, goog:chromeOptions: {debuggerAddress: localhost:40111}, networkConnectionEnabled: false, pageLoadStrategy: normal, platformName: linux, proxy: Proxy(), se:bidiEnabled: false, se:cdp: ws://172.25.0.3:4444/sessio..., se:cdpVersion: 122.0.6261.69, se:vnc: ws://172.25.0.3:4444/sessio..., se:vncEnabled: true, se:vncLocalAddress: ws://172.25.0.3:7900, setWindowRect: true, strictFileInteractability: false, timeouts: {implicit: 0, pageLoad: 300000, script: 30000}, unhandledPromptBehavior: dismiss and notify, webauthn:extension:credBlob: true, webauthn:extension:largeBlob: true, webauthn:extension:minPinLength: true, webauthn:extension:prf: true, webauthn:virtualAuthenticators: true}
00:38:25.491 WARN [SeleniumSpanExporter$1.lambda$export$3] - {"traceId": "1a521a6ec1ce968fc6b9bf700299c5b4","eventTime": 1724978305491369546,"eventName": "HTTP request execution complete","attributes": {"http.flavor": 1,"http.handler_class": "org.openqa.selenium.remote.http.Route$PredicatedRoute","http.host": "172.25.0.3:5555","http.method": "POST","http.request_content_length": "2","http.scheme": "HTTP","http.status_code": 400,"http.target": "\u002fsession\u002f2f24e30989e72299b3eae7009a4775e6\u002felement\u002ff.A9617F55C6C5A10B538CFB592A657CEF.d.9E7EBA32520FF57863E46438A6D0856B.e.114\u002fclick","http.user_agent": "selenium\u002f4.15.2 (python linux)"}}
00:38:25.842 INFO [SessionSlot.stop] - Stopping session 2f24e30989e72299b3eae7009a4775e6
00:38:26.205 INFO [LocalNode.newSession] - Session created by the Node. Id: 833fb56d9cf32148771decb0435d730b, Caps: Capabilities {acceptInsecureCerts: false, browserName: chrome, browserVersion: 122.0.6261.69, chrome: {chromedriverVersion: 122.0.6261.69 (81bc525b6a36..., userDataDir: /tmp/.org.chromium.Chromium...}, fedcm:accounts: true, goog:chromeOptions: {debuggerAddress: localhost:39967}, networkConnectionEnabled: false, pageLoadStrategy: normal, platformName: linux, proxy: Proxy(), se:bidiEnabled: false, se:cdp: ws://172.25.0.3:4444/sessio..., se:cdpVersion: 122.0.6261.69, se:vnc: ws://172.25.0.3:4444/sessio..., se:vncEnabled: true, se:vncLocalAddress: ws://172.25.0.3:7900, setWindowRect: true, strictFileInteractability: false, timeouts: {implicit: 0, pageLoad: 300000, script: 30000}, unhandledPromptBehavior: dismiss and notify, webauthn:extension:credBlob: true, webauthn:extension:largeBlob: true, webauthn:extension:minPinLength: true, webauthn:extension:prf: true, webauthn:virtualAuthenticators: true}
00:38:35.248 WARN [SeleniumSpanExporter$1.lambda$export$3] - {"traceId": "6e86a895e786baa4f0f8f5d3c1da87d1","eventTime": 1724978315247853681,"eventName": "HTTP request execution complete","attributes": {"http.flavor": 1,"http.handler_class": "org.openqa.selenium.remote.http.Route$PredicatedRoute","http.host": "172.25.0.3:5555","http.method": "POST","http.request_content_length": "2","http.scheme": "HTTP","http.status_code": 400,"http.target": "\u002fsession\u002f833fb56d9cf32148771decb0435d730b\u002felement\u002ff.328784E2A4CAB9ED06A5F78F06E5ACBF.d.BA01AEB49CF5E4F501D25E88D9EC4E36.e.116\u002fclick","http.user_agent": "selenium\u002f4.15.2 (python linux)"}}

### selenium-grid-chrome_video:
no log

Operating System

docker linux

Docker Selenium version (image tag)

4.18.1-20240224

Selenium Grid chart version (chart version)

4.18.1

[github-actions]

@novalagung, thank you for creating this issue. We will troubleshoot it as soon as we can.

---

Info for maintainers

Triage this issue by using labels.

If information is missing, add a helpful comment and then I-issue-template label.

If the issue is a question, add the I-question label.

If the issue is valid but there is no time to troubleshoot it, consider adding the help wanted label.

If the issue requires changes or fixes from an external project (e.g., ChromeDriver, GeckoDriver, MSEdgeDriver, W3C), add the applicable G-* label, and it will provide the correct link and auto-close the issue.

After troubleshooting the issue, please add the R-awaiting answer label.

Thank you!

[VietND96]

Is there a particular type of failing test execution error that can cause the video not to be generated?

Yes, it could be since record start/stop relies on node status endpoint response.
Can you try selenium/video:latest or recent tag to see any improvement on this.

[novalagung]

@VietND96 sure, I'll post another update here asap if I found anything

[novalagung]

Finally, I managed to get this issue fixed by adding some updates on the code level, on how the selenium driver is being managed.

I'm using pytest and changed the default driver() scope to session, and now it looks good.

@pytest.fixture()
def driver(request, scope="session"):

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.