Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Increase stop-domain command timeout in tests #24812

Merged
merged 1 commit into from
Feb 15, 2024
Merged

Increase stop-domain command timeout in tests #24812

merged 1 commit into from
Feb 15, 2024

Conversation

avpinchuk
Copy link
Contributor

This should decrease frequency of failures of the application-tests.

@avpinchuk avpinchuk added this to the 7.0.13 milestone Feb 14, 2024
@avpinchuk avpinchuk self-assigned this Feb 14, 2024
@avpinchuk
Increase stop-domain command timeout in tests
9dde62a 
Signed-off-by: Alexander Pinčuk <alexander.v.pinchuk@gmail.com>
@avpinchuk avpinchuk force-pushed the increase-stop-timeout branch from 65f71ef to 9dde62a Compare February 14, 2024 20:57
@dmatej
Copy link
Contributor

dmatej commented Feb 14, 2024

I am quite sure that domain should stop much faster than in 10 seconds if everything is alright, especially with --kill and --force.

@dmatej
Copy link
Contributor

dmatej commented Feb 14, 2024

To me it seems there is some different issue - the shutdown hook from admin-tests is not executed or is executed too late sometimes, for some reason, and just on Jenkins.

14:14:34  13:14:34.427359    INFO                 main               org.glassfish.main.itest.tools.asadmin.Asadmin.exec exec(timeout=30,000, detached=false, args=[start-domain, --debug])
14:14:35  There is a process already using the admin port 4,848 -- it probably is another instance of a GlassFish server.
14:14:35  Command start-domain failed.

Or maybe these is yet different issue ... I know I wrote this code, but why the hell the change-admin-password command failed?

14:14:33  [INFO] Running org.glassfish.main.test.app.security.multirolemapping.MultiRoleMappingTest
14:14:34  13:14:33.958583    INFO                 main      org.glassfish.main.itest.tools.GlassFishTestEnvironment.<clinit> Using basedir: /home/jenkins/agent/workspace/_test-using-jenkinsfile_PR-24811/appserver/tests/application
14:14:34  13:14:33.962103    INFO                 main      org.glassfish.main.itest.tools.GlassFishTestEnvironment.<clinit> Expected GlassFish directory: /home/jenkins/agent/workspace/_test-using-jenkinsfile_PR-24811/appserver/tests/application/target/glassfish7/glassfish
14:14:34  13:14:33.963955    INFO                 main               org.glassfish.main.itest.tools.asadmin.Asadmin.exec exec(timeout=5,000, detached=false, args=[change-admin-password])
14:14:34  Command change-admin-password failed.
14:14:34  
14:14:34  Authentication failed for user: admin (Usually, this means invalid user name and/or password)
14:14:34  
14:14:34  
14:14:34  Admin password NOT changed.

https://ci.eclipse.org/glassfish/job/glassfish_build-and-test-using-jenkinsfile/job/PR-24811/7/execution/node/37/log/

@dmatej
Copy link
Contributor

dmatej commented Feb 14, 2024 •
edited
Loading

And the next build was successful:

21:42:19  [INFO] Running org.glassfish.main.test.app.security.multirolemapping.MultiRoleMappingTest
21:42:19  20:42:18.970950    INFO                 main      org.glassfish.main.itest.tools.GlassFishTestEnvironment.<clinit> Using basedir: /home/jenkins/agent/workspace/_test-using-jenkinsfile_PR-24811/appserver/tests/application
21:42:19  20:42:18.974180    INFO                 main      org.glassfish.main.itest.tools.GlassFishTestEnvironment.<clinit> Expected GlassFish directory: /home/jenkins/agent/workspace/_test-using-jenkinsfile_PR-24811/appserver/tests/application/target/glassfish7/glassfish
21:42:19  20:42:18.975770    INFO                 main               org.glassfish.main.itest.tools.asadmin.Asadmin.exec exec(timeout=5,000, detached=false, args=[change-admin-password])
21:42:19  Command change-admin-password executed successfully.
21:42:19  
21:42:19  
21:42:19  
21:42:19  Admin password changed.

I know that asadmin sometimes creates files in user's home directory ... could it be related?
EDIT: Nope, those are mostly related to gfclient and clusters.

@avpinchuk
Copy link
Contributor Author

You're right. This is a temporary solution. But now we have a problem with admin console. I'm working on it.

You can reproduce the problem with these steps:

  • start domain
  • touch the admin console
  • stop domain
  • start domain and immediately stop domain
  • check the tail of the server.log

Shortly, admin console try to start when GlassFish already stopped.

@avpinchuk
Copy link
Contributor Author

  • start-domain, then pause 10 sec, then stop-domain - Waiting finished after 149 ms.
  • start-domain, then immediate stop-domain - Waiting finished after 5,260 ms.

on my local machine

@avpinchuk
Copy link
Contributor Author

This is a part of the server.log:

[2024-02-15T01:29:36.622664+03:00] [GF 7.0.13-SNAPSHOT] [INFO] [] [jakarta.enterprise.logging.stdout] [tid: _ThreadID=22 _ThreadName=GlassFish Shutdown Hook] [levelValue: 800] [[
  Completed shutdown of GlassFish runtime]]

[2024-02-15T01:29:37.335099+03:00] [GF 7.0.13-SNAPSHOT] [SEVERE] [] [] [tid: _ThreadID=104 _ThreadName=Thread-11] [levelValue: 1000] [[
  NCLS-CORE-00029
MultiException stack 1 of 3
java.lang.IllegalStateException: Service com.sun.enterprise.v3.services.impl.GrizzlyService was started at level 0 but it has a run level of 10.  The full descriptor is SystemDescriptor(
	implementation=com.sun.enterprise.v3.services.impl.GrizzlyService
	contracts={com.sun.enterprise.v3.services.impl.GrizzlyService,org.glassfish.api.container.RequestDispatcher}
	scope=org.glassfish.hk2.runlevel.RunLevel
	qualifiers={}
	descriptorType=CLASS
	descriptorVisibility=NORMAL
	metadata=runLevelValue={10},Bundle-SymbolicName={org.glassfish.main.core.kernel},Bundle-Version={7.0.13.SNAPSHOT}
	rank=50
	loader=OsgiPopulatorPostProcessor.HK2Loader(OSGiModuleImpl:: Bundle = [org.glassfish.main.core.kernel [67]], State = [READY],1338010098)
	proxiable=null
	proxyForSameScope=null
	analysisName=null
	id=888
	locatorId=0
	identityHashCode=1276276154
	reified=true)
	at org.glassfish.hk2.runlevel.internal.AsyncRunLevelContext.validate(AsyncRunLevelContext.java:422)
	at org.glassfish.hk2.runlevel.internal.AsyncRunLevelContext.findOrCreate(AsyncRunLevelContext.java:275)
	at org.glassfish.hk2.runlevel.RunLevelContext.findOrCreate(RunLevelContext.java:65)
	at org.jvnet.hk2.internal.Utilities.createService(Utilities.java:2109)
	at org.jvnet.hk2.internal.ServiceHandleImpl.getService(ServiceHandleImpl.java:93)
	at org.jvnet.hk2.internal.ServiceLocatorImpl.getService(ServiceLocatorImpl.java:690)
	at org.jvnet.hk2.internal.ThreeThirtyResolver.resolve(ThreeThirtyResolver.java:55)
	at org.jvnet.hk2.internal.ClazzCreator.resolve(ClazzCreator.java:197)
	at org.jvnet.hk2.internal.ClazzCreator.resolveAllDependencies(ClazzCreator.java:234)
	at org.jvnet.hk2.internal.ClazzCreator.create(ClazzCreator.java:387)
	at org.jvnet.hk2.internal.SystemDescriptor.create(SystemDescriptor.java:466)
	at org.jvnet.hk2.internal.SingletonContext$1.compute(SingletonContext.java:59)
	at org.jvnet.hk2.internal.SingletonContext$1.compute(SingletonContext.java:47)
	at org.glassfish.hk2.utilities.cache.Cache$OriginThreadAwareFuture$1.call(Cache.java:74)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
	at org.glassfish.hk2.utilities.cache.Cache$OriginThreadAwareFuture.run(Cache.java:131)
	at org.glassfish.hk2.utilities.cache.Cache.compute(Cache.java:176)
	at org.jvnet.hk2.internal.SingletonContext.findOrCreate(SingletonContext.java:98)
	at org.jvnet.hk2.internal.Utilities.createService(Utilities.java:2109)
	at org.jvnet.hk2.internal.ServiceHandleImpl.getService(ServiceHandleImpl.java:93)
	at org.jvnet.hk2.internal.ServiceHandleImpl.getService(ServiceHandleImpl.java:67)
	at org.glassfish.internal.data.EngineInfo.getContainer(EngineInfo.java:62)
	at com.sun.enterprise.v3.server.ApplicationLifecycle.startContainers(ApplicationLifecycle.java:1064)
	at com.sun.enterprise.v3.server.ApplicationLifecycle.setupContainerInfos(ApplicationLifecycle.java:782)
	at com.sun.enterprise.v3.server.ApplicationLifecycle.deploy(ApplicationLifecycle.java:413)
	at com.sun.enterprise.v3.server.ApplicationLoaderService.processApplication(ApplicationLoaderService.java:404)
	at com.sun.enterprise.v3.admin.adapter.InstallerThread.load(InstallerThread.java:211)
	at com.sun.enterprise.v3.admin.adapter.InstallerThread.run(InstallerThread.java:82)
MultiException stack 2 of 3
java.lang.IllegalArgumentException: While attempting to resolve the dependencies of com.sun.enterprise.web.WebContainer errors were found
	at org.jvnet.hk2.internal.ClazzCreator.resolveAllDependencies(ClazzCreator.java:247)
	at org.jvnet.hk2.internal.ClazzCreator.create(ClazzCreator.java:387)
	at org.jvnet.hk2.internal.SystemDescriptor.create(SystemDescriptor.java:466)
	at org.jvnet.hk2.internal.SingletonContext$1.compute(SingletonContext.java:59)
	at org.jvnet.hk2.internal.SingletonContext$1.compute(SingletonContext.java:47)
	at org.glassfish.hk2.utilities.cache.Cache$OriginThreadAwareFuture$1.call(Cache.java:74)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
	at org.glassfish.hk2.utilities.cache.Cache$OriginThreadAwareFuture.run(Cache.java:131)
	at org.glassfish.hk2.utilities.cache.Cache.compute(Cache.java:176)
	at org.jvnet.hk2.internal.SingletonContext.findOrCreate(SingletonContext.java:98)
	at org.jvnet.hk2.internal.Utilities.createService(Utilities.java:2109)
	at org.jvnet.hk2.internal.ServiceHandleImpl.getService(ServiceHandleImpl.java:93)
	at org.jvnet.hk2.internal.ServiceHandleImpl.getService(ServiceHandleImpl.java:67)
	at org.glassfish.internal.data.EngineInfo.getContainer(EngineInfo.java:62)
	at com.sun.enterprise.v3.server.ApplicationLifecycle.startContainers(ApplicationLifecycle.java:1064)
	at com.sun.enterprise.v3.server.ApplicationLifecycle.setupContainerInfos(ApplicationLifecycle.java:782)
	at com.sun.enterprise.v3.server.ApplicationLifecycle.deploy(ApplicationLifecycle.java:413)
	at com.sun.enterprise.v3.server.ApplicationLoaderService.processApplication(ApplicationLoaderService.java:404)
	at com.sun.enterprise.v3.admin.adapter.InstallerThread.load(InstallerThread.java:211)
	at com.sun.enterprise.v3.admin.adapter.InstallerThread.run(InstallerThread.java:82)
MultiException stack 3 of 3
java.lang.IllegalStateException: Unable to perform operation: resolve on com.sun.enterprise.web.WebContainer
	at org.jvnet.hk2.internal.ClazzCreator.create(ClazzCreator.java:422)
	at org.jvnet.hk2.internal.SystemDescriptor.create(SystemDescriptor.java:466)
	at org.jvnet.hk2.internal.SingletonContext$1.compute(SingletonContext.java:59)
	at org.jvnet.hk2.internal.SingletonContext$1.compute(SingletonContext.java:47)
	at org.glassfish.hk2.utilities.cache.Cache$OriginThreadAwareFuture$1.call(Cache.java:74)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
	at org.glassfish.hk2.utilities.cache.Cache$OriginThreadAwareFuture.run(Cache.java:131)
	at org.glassfish.hk2.utilities.cache.Cache.compute(Cache.java:176)
	at org.jvnet.hk2.internal.SingletonContext.findOrCreate(SingletonContext.java:98)
	at org.jvnet.hk2.internal.Utilities.createService(Utilities.java:2109)
	at org.jvnet.hk2.internal.ServiceHandleImpl.getService(ServiceHandleImpl.java:93)
	at org.jvnet.hk2.internal.ServiceHandleImpl.getService(ServiceHandleImpl.java:67)
	at org.glassfish.internal.data.EngineInfo.getContainer(EngineInfo.java:62)
	at com.sun.enterprise.v3.server.ApplicationLifecycle.startContainers(ApplicationLifecycle.java:1064)
	at com.sun.enterprise.v3.server.ApplicationLifecycle.setupContainerInfos(ApplicationLifecycle.java:782)
	at com.sun.enterprise.v3.server.ApplicationLifecycle.deploy(ApplicationLifecycle.java:413)
	at com.sun.enterprise.v3.server.ApplicationLoaderService.processApplication(ApplicationLoaderService.java:404)
	at com.sun.enterprise.v3.admin.adapter.InstallerThread.load(InstallerThread.java:211)
	at com.sun.enterprise.v3.admin.adapter.InstallerThread.run(InstallerThread.java:82)
]]

[2024-02-15T01:29:37.336963+03:00] [GF 7.0.13-SNAPSHOT] [SEVERE] [] [jakarta.enterprise.system.core] [tid: _ThreadID=104 _ThreadName=Thread-11] [levelValue: 1000] [[
  Aborting, Failed to start container com.sun.enterprise.web.WebContainer]]

[2024-02-15T01:29:37.337472+03:00] [GF 7.0.13-SNAPSHOT] [SEVERE] [] [jakarta.enterprise.system.core] [tid: _ThreadID=104 _ThreadName=Thread-11] [levelValue: 1000] [[
  Exception while deploying the app [__admingui]]]

[2024-02-15T01:29:37.337774+03:00] [GF 7.0.13-SNAPSHOT] [SEVERE] [NCLS-CORE-00026] [jakarta.enterprise.system.core] [tid: _ThreadID=104 _ThreadName=Thread-11] [levelValue: 1000] [[
  Exception during lifecycle processing
java.lang.Exception: Aborting, Failed to start container com.sun.enterprise.web.WebContainer
	at com.sun.enterprise.v3.server.ApplicationLifecycle.setupContainerInfos(ApplicationLifecycle.java:785)
	at com.sun.enterprise.v3.server.ApplicationLifecycle.deploy(ApplicationLifecycle.java:413)
	at com.sun.enterprise.v3.server.ApplicationLoaderService.processApplication(ApplicationLoaderService.java:404)
	at com.sun.enterprise.v3.admin.adapter.InstallerThread.load(InstallerThread.java:211)
	at com.sun.enterprise.v3.admin.adapter.InstallerThread.run(InstallerThread.java:82)
]]

[2024-02-15T01:29:37.338515+03:00] [GF 7.0.13-SNAPSHOT] [INFO] [] [org.glassfish.web.loader.WebappClassLoader] [tid: _ThreadID=104 _ThreadName=Thread-11] [levelValue: 800] [[
  close(), this:
org.glassfish.web.loader.WebappClassLoader@2050cabf[name=unknown], urls=[
file:/home/avpinchuk/build/glassfish7/glassfish/lib/install/applications/__admingui/WEB-INF/classes/
file:/home/avpinchuk/build/glassfish7/glassfish/lib/install/applications/__admingui/WEB-INF/lib/console-core-7.0.13-SNAPSHOT.jar
file:/home/avpinchuk/build/glassfish7/glassfish/domains/domain1/generated/ejb/__admingui//
file:/home/avpinchuk/build/glassfish7/glassfish/lib/install/applications/__admingui/WEB-INF/extra/woodstock-webui-jsf-suntheme-6.0.1.jar
file:/home/avpinchuk/build/glassfish7/glassfish/lib/install/applications/__admingui/WEB-INF/extra/dojo-ajax-nodemo-1.12.4.jar
file:/home/avpinchuk/build/glassfish7/glassfish/lib/install/applications/__admingui/WEB-INF/extra/woodstock-webui-jsf-6.0.1.jar
file:/home/avpinchuk/build/glassfish7/glassfish/lib/install/applications/__admingui/WEB-INF/extra/json-2.0.jar
file:/home/avpinchuk/build/glassfish7/glassfish/lib/install/applications/__admingui/WEB-INF/extra/prototype-1.7.3.jar
file:/home/avpinchuk/build/glassfish7/glassfish/lib/install/applications/__admingui/WEB-INF/extra/commons-io-2.15.1.jar
][delegate=true, context=unknown, status=RUNNING, antiJARLocking=false, securityManager=false, packageDefinitionSecurityEnabled=false, repositories=RepositoryManager[WEB-INF/classes/], notFound.size=10, pathTimestamps.size=1, resourceEntryCache.size=0]]]

[2024-02-15T01:29:37.360875+03:00] [GF 7.0.13-SNAPSHOT] [SEVERE] [NCLS-CORE-00041] [jakarta.enterprise.system.core] [tid: _ThreadID=104 _ThreadName=Thread-11] [levelValue: 1000] [[
  Application deployment failed: Exception while deploying the app [__admingui]]]

@dmatej
Copy link
Contributor

dmatej commented Feb 14, 2024

Wow, that is really interesting and might be the cause ... hmmm

dmatej@dmatej-P7740:/app/appservers$ asadmin start-domain && asadmin stop-domain
Waiting for domain1 to start ...
Waiting finished after 2 597 ms.
Successfully started the domain : domain1
domain  Location: /media/data-samsung/data/dmatej/work/app/appservers/glassfish7/glassfish/domains/domain1
Log File: /media/data-samsung/data/dmatej/work/app/appservers/glassfish7/glassfish/domains/domain1/logs/server.log
Admin Port: 4 848
Command start-domain executed successfully.
Waiting for the domain to stop .....
Waiting finished after 4 090 ms.
Command stop-domain executed successfully.

dmatej@dmatej-P7740:/app/appservers$ asadmin start-domain
Waiting for domain1 to start ...
Waiting finished after 2 453 ms.
Successfully started the domain : domain1
domain  Location: /media/data-samsung/data/dmatej/work/app/appservers/glassfish7/glassfish/domains/domain1
Log File: /media/data-samsung/data/dmatej/work/app/appservers/glassfish7/glassfish/domains/domain1/logs/server.log
Admin Port: 4 848
Command start-domain executed successfully.

dmatej@dmatej-P7740:/app/appservers$ asadmin stop-domain
Waiting for the domain to stop .
Waiting finished after 147 ms.
Command stop-domain executed successfully.
dmatej@dmatej-P7740:/app/appservers$

dmatej
dmatej approved these changes Feb 14, 2024
View reviewed changes
Copy link
Contributor

@dmatej dmatej left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, lets do that. I see this as good news, at least now it is not a mystery any more :-)

@arjantijms arjantijms merged commit ff7388f into eclipse-ee4j:master Feb 15, 2024
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dmatej dmatej dmatej approved these changes

@arjantijms arjantijms arjantijms approved these changes

@pzygielo pzygielo pzygielo approved these changes

@hs536 hs536 Awaiting requested review from hs536

Assignees

@avpinchuk avpinchuk

Labels
build and test improvement
Projects
None yet
Milestone
7.0.13
Development

Successfully merging this pull request may close these issues.
4 participants
@avpinchuk @dmatej @arjantijms @pzygielo