-
Notifications
You must be signed in to change notification settings - Fork 575
UnreachableBrowserException in middle of testing #1090
Comments
More investigating, it does seem like here is the output of /wd/hub when it comes back up (Not sure if the errors on this page are any help :\ ) {
"sessionId": null,
"status": 13,
"value": {
"class": "java.lang.NullPointerException",
"error": "unknown error",
"message": null,
"stackTrace": [
{
"className": "org.openqa.grid.internal.ExternalSessionKey",
"fileName": "ExternalSessionKey.java",
"lineNumber": 79,
"methodName": "fromWebDriverRequest"
},
{
"className": "org.openqa.grid.web.servlet.handler.WebDriverRequest",
"fileName": "WebDriverRequest.java",
"lineNumber": 58,
"methodName": "extractSession"
},
{
"className": "org.openqa.grid.web.servlet.handler.RequestHandler",
"fileName": "RequestHandler.java",
"lineNumber": 240,
"methodName": "getSession"
},
{
"className": "org.openqa.grid.web.servlet.handler.RequestHandler",
"fileName": "RequestHandler.java",
"lineNumber": 123,
"methodName": "process"
},
{
"className": "org.openqa.grid.web.servlet.DriverServlet",
"fileName": "DriverServlet.java",
"lineNumber": 85,
"methodName": "process"
},
{
"className": "org.openqa.grid.web.servlet.DriverServlet",
"fileName": "DriverServlet.java",
"lineNumber": 63,
"methodName": "doGet"
},
{
"className": "javax.servlet.http.HttpServlet",
"fileName": "HttpServlet.java",
"lineNumber": 687,
"methodName": "service"
},
{
"className": "javax.servlet.http.HttpServlet",
"fileName": "HttpServlet.java",
"lineNumber": 790,
"methodName": "service"
},
{
"className": "org.seleniumhq.jetty9.servlet.ServletHolder",
"fileName": "ServletHolder.java",
"lineNumber": 865,
"methodName": "handle"
},
{
"className": "org.seleniumhq.jetty9.servlet.ServletHandler$CachedChain",
"fileName": "ServletHandler.java",
"lineNumber": 1655,
"methodName": "doFilter"
},
{
"className": "io.prometheus.client.filter.MetricsFilter",
"fileName": "MetricsFilter.java",
"lineNumber": 170,
"methodName": "doFilter"
},
{
"className": "org.seleniumhq.jetty9.servlet.ServletHandler$CachedChain",
"fileName": "ServletHandler.java",
"lineNumber": 1642,
"methodName": "doFilter"
},
{
"className": "org.seleniumhq.jetty9.servlet.ServletHandler",
"fileName": "ServletHandler.java",
"lineNumber": 533,
"methodName": "doHandle"
},
{
"className": "org.seleniumhq.jetty9.server.handler.ScopedHandler",
"fileName": "ScopedHandler.java",
"lineNumber": 146,
"methodName": "handle"
},
{
"className": "org.seleniumhq.jetty9.security.SecurityHandler",
"fileName": "SecurityHandler.java",
"lineNumber": 548,
"methodName": "handle"
},
{
"className": "org.seleniumhq.jetty9.server.handler.HandlerWrapper",
"fileName": "HandlerWrapper.java",
"lineNumber": 132,
"methodName": "handle"
},
{
"className": "org.seleniumhq.jetty9.server.handler.ScopedHandler",
"fileName": "ScopedHandler.java",
"lineNumber": 257,
"methodName": "nextHandle"
},
{
"className": "org.seleniumhq.jetty9.server.session.SessionHandler",
"fileName": "SessionHandler.java",
"lineNumber": 1595,
"methodName": "doHandle"
},
{
"className": "org.seleniumhq.jetty9.server.handler.ScopedHandler",
"fileName": "ScopedHandler.java",
"lineNumber": 255,
"methodName": "nextHandle"
},
{
"className": "org.seleniumhq.jetty9.server.handler.ContextHandler",
"fileName": "ContextHandler.java",
"lineNumber": 1340,
"methodName": "doHandle"
},
{
"className": "org.seleniumhq.jetty9.server.handler.ScopedHandler",
"fileName": "ScopedHandler.java",
"lineNumber": 203,
"methodName": "nextScope"
},
{
"className": "org.seleniumhq.jetty9.servlet.ServletHandler",
"fileName": "ServletHandler.java",
"lineNumber": 473,
"methodName": "doScope"
},
{
"className": "org.seleniumhq.jetty9.server.session.SessionHandler",
"fileName": "SessionHandler.java",
"lineNumber": 1564,
"methodName": "doScope"
},
{
"className": "org.seleniumhq.jetty9.server.handler.ScopedHandler",
"fileName": "ScopedHandler.java",
"lineNumber": 201,
"methodName": "nextScope"
},
{
"className": "org.seleniumhq.jetty9.server.handler.ContextHandler",
"fileName": "ContextHandler.java",
"lineNumber": 1242,
"methodName": "doScope"
},
{
"className": "org.seleniumhq.jetty9.server.handler.ScopedHandler",
"fileName": "ScopedHandler.java",
"lineNumber": 144,
"methodName": "handle"
},
{
"className": "org.seleniumhq.jetty9.server.handler.StatisticsHandler",
"fileName": "StatisticsHandler.java",
"lineNumber": 174,
"methodName": "handle"
},
{
"className": "org.seleniumhq.jetty9.server.handler.HandlerWrapper",
"fileName": "HandlerWrapper.java",
"lineNumber": 132,
"methodName": "handle"
},
{
"className": "org.seleniumhq.jetty9.server.Server",
"fileName": "Server.java",
"lineNumber": 503,
"methodName": "handle"
},
{
"className": "org.seleniumhq.jetty9.server.HttpChannel",
"fileName": "HttpChannel.java",
"lineNumber": 364,
"methodName": "handle"
},
{
"className": "org.seleniumhq.jetty9.server.HttpConnection",
"fileName": "HttpConnection.java",
"lineNumber": 260,
"methodName": "onFillable"
},
{
"className": "org.seleniumhq.jetty9.io.AbstractConnection$ReadCallback",
"fileName": "AbstractConnection.java",
"lineNumber": 305,
"methodName": "succeeded"
},
{
"className": "org.seleniumhq.jetty9.io.FillInterest",
"fileName": "FillInterest.java",
"lineNumber": 103,
"methodName": "fillable"
},
{
"className": "org.seleniumhq.jetty9.io.ChannelEndPoint$2",
"fileName": "ChannelEndPoint.java",
"lineNumber": 118,
"methodName": "run"
},
{
"className": "org.seleniumhq.jetty9.util.thread.strategy.EatWhatYouKill",
"fileName": "EatWhatYouKill.java",
"lineNumber": 333,
"methodName": "runTask"
},
{
"className": "org.seleniumhq.jetty9.util.thread.strategy.EatWhatYouKill",
"fileName": "EatWhatYouKill.java",
"lineNumber": 310,
"methodName": "doProduce"
},
{
"className": "org.seleniumhq.jetty9.util.thread.strategy.EatWhatYouKill",
"fileName": "EatWhatYouKill.java",
"lineNumber": 168,
"methodName": "tryProduce"
},
{
"className": "org.seleniumhq.jetty9.util.thread.strategy.EatWhatYouKill",
"fileName": "EatWhatYouKill.java",
"lineNumber": 126,
"methodName": "run"
},
{
"className": "org.seleniumhq.jetty9.util.thread.ReservedThreadExecutor$ReservedThread",
"fileName": "ReservedThreadExecutor.java",
"lineNumber": 366,
"methodName": "run"
},
{
"className": "org.seleniumhq.jetty9.util.thread.QueuedThreadPool",
"fileName": "QueuedThreadPool.java",
"lineNumber": 765,
"methodName": "runJob"
},
{
"className": "org.seleniumhq.jetty9.util.thread.QueuedThreadPool$2",
"fileName": "QueuedThreadPool.java",
"lineNumber": 683,
"methodName": "run"
},
{
"className": "java.lang.Thread",
"fileName": "Thread.java",
"lineNumber": 748,
"methodName": "run"
}
],
"stacktrace": "java.lang.NullPointerException\n\tat org.openqa.grid.internal.ExternalSessionKey.fromWebDriverRequest(ExternalSessionKey.java:79)\n\tat org.openqa.grid.web.servlet.handler.WebDriverRequest.extractSession(WebDriverRequest.java:58)\n\tat org.openqa.grid.web.servlet.handler.RequestHandler.getSession(RequestHandler.java:240)\n\tat org.openqa.grid.web.servlet.handler.RequestHandler.process(RequestHandler.java:123)\n\tat org.openqa.grid.web.servlet.DriverServlet.process(DriverServlet.java:85)\n\tat org.openqa.grid.web.servlet.DriverServlet.doGet(DriverServlet.java:63)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:687)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:790)\n\tat org.seleniumhq.jetty9.servlet.ServletHolder.handle(ServletHolder.java:865)\n\tat org.seleniumhq.jetty9.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1655)\n\tat io.prometheus.client.filter.MetricsFilter.doFilter(MetricsFilter.java:170)\n\tat org.seleniumhq.jetty9.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)\n\tat org.seleniumhq.jetty9.servlet.ServletHandler.doHandle(ServletHandler.java:533)\n\tat org.seleniumhq.jetty9.server.handler.ScopedHandler.handle(ScopedHandler.java:146)\n\tat org.seleniumhq.jetty9.security.SecurityHandler.handle(SecurityHandler.java:548)\n\tat org.seleniumhq.jetty9.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)\n\tat org.seleniumhq.jetty9.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)\n\tat org.seleniumhq.jetty9.server.session.SessionHandler.doHandle(SessionHandler.java:1595)\n\tat org.seleniumhq.jetty9.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)\n\tat org.seleniumhq.jetty9.server.handler.ContextHandler.doHandle(ContextHandler.java:1340)\n\tat org.seleniumhq.jetty9.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)\n\tat org.seleniumhq.jetty9.servlet.ServletHandler.doScope(ServletHandler.java:473)\n\tat org.seleniumhq.jetty9.server.session.SessionHandler.doScope(SessionHandler.java:1564)\n\tat org.seleniumhq.jetty9.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)\n\tat org.seleniumhq.jetty9.server.handler.ContextHandler.doScope(ContextHandler.java:1242)\n\tat org.seleniumhq.jetty9.server.handler.ScopedHandler.handle(ScopedHandler.java:144)\n\tat org.seleniumhq.jetty9.server.handler.StatisticsHandler.handle(StatisticsHandler.java:174)\n\tat org.seleniumhq.jetty9.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)\n\tat org.seleniumhq.jetty9.server.Server.handle(Server.java:503)\n\tat org.seleniumhq.jetty9.server.HttpChannel.handle(HttpChannel.java:364)\n\tat org.seleniumhq.jetty9.server.HttpConnection.onFillable(HttpConnection.java:260)\n\tat org.seleniumhq.jetty9.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305)\n\tat org.seleniumhq.jetty9.io.FillInterest.fillable(FillInterest.java:103)\n\tat org.seleniumhq.jetty9.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)\n\tat org.seleniumhq.jetty9.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)\n\tat org.seleniumhq.jetty9.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)\n\tat org.seleniumhq.jetty9.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)\n\tat org.seleniumhq.jetty9.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)\n\tat org.seleniumhq.jetty9.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)\n\tat org.seleniumhq.jetty9.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765)\n\tat org.seleniumhq.jetty9.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683)\n\tat java.lang.Thread.run(Thread.java:748)\n"
}
} more readable stack trace
|
To solve this issue we made a method to retry the driver connection multiple times. static WebDriver executeRemoteDriver (capabilities, testName){
def webdriver
def i = 0
while (webdriver == null && i < 30) {
try{
webdriver = new RemoteWebDriver(new URL("http://10.204.128.218:32222/wd/hub"), capabilities)
} catch(Exception e){
if(webdriver == null){
System.out.println("Retrying driver with test: $testName date: ${LocalDateTime.now()}")
}
}
i++
}
return webdriver
} I'm suspecting more of a k8s network issue than a zalenium issue. As a result of this method we will have the remote driver try to connect about 3 times before a sucessful connection Do you guys recommend a specific network plugin for zalenium? We are using Weave. |
Overall After switching to Flannel from Weave test execution has stabilized. No more problems launching browsers or browsers quiting during tests |
🐛 Bug Report
During testing we will have chrome instances launch successfully to start testing.
after a couple of tests automation has trouble launching new chrome instances.
This error will fail almost all of our test repeatally.
Here is the error
we have a 4 node k8s cluster that seems healthy
Here is an error output that might be related found in the pod logs
Here is some of the code to launch the remote web driver.
We also have a method that cleans up the driver after a test is done with it (this is supposed to be compatible with parallel testing. We have tested commenting out this method and we still get the same error)
To Reproduce
Steps to reproduce the behavior (including the docker command/docker-compose/Kubernetes manifests to start Zalenium):
Manifest specified at the bottom
Expected behavior
A clear and concise description of what you expected to happen.
Successful launch of new browsers with a stable connection between them.
Test script reproducing this issue (when applicable)
Unfortunalty I can't share the repo, but I can seek out more logs and try as many differn't scenarios as needed.
Environment
OS:
Zalenium Image Version(s):
Docker version:
If using Kubernetes, specify your environment, and if relevant your manifests:
zalenium deploy yaml created from helm template:
Thank you for any help, I know this issue is ambiguous, but I am willing to provide any extra help I can.
The text was updated successfully, but these errors were encountered: