Port already in use when jmx enabled

[psimms-r7]

Jenkins and plugins versions report

Environment

Paste the output here

What Operating System are you using (both controller, and any agents involved in the problem)?

Image - jenkins/jenkins:2.387.2-lts-jdk11
Helm chart -https://github.com/jenkinsci/helm-charts - v4.3.21
AWS EKS v1.25 - m7g.8xlarge - arm64

Reproduction steps

Enable JMX when installing jenkins

Example
https://github.com/jenkinsci/helm-charts/blob/main/charts/jenkins/values.yaml#L231-L235

  javaOpts: >
    -Dcom.sun.management.jmxremote.port=4000
    -Dcom.sun.management.jmxremote.authenticate=false
    -Dcom.sun.management.jmxremote.ssl=false
  jmxPort: 4000

Try to run jenkins-plugin-cli on jenkins instance

Error: JMX connector server communication error: service:jmx:rmi://jenkins-sandbox-0:4000
jdk.internal.agent.AgentConfigurationError: java.rmi.server.ExportException: Port already in use: 4000; nested exception is

Expected Results

Be able to run jenkins-plugin-cli without error when jmx is enabled on jenkins

Actual Results

jenkins@jenkins-sandbox-0:/$ jenkins-plugin-cli --version
Error: JMX connector server communication error: service:jmx:rmi://jenkins-sandbox-0:4000
jdk.internal.agent.AgentConfigurationError: java.rmi.server.ExportException: Port already in use: 4000; nested exception is: 
        java.net.BindException: Address already in use (Bind failed)
        at jdk.management.agent/sun.management.jmxremote.ConnectorBootstrap.exportMBeanServer(ConnectorBootstrap.java:820)
        at jdk.management.agent/sun.management.jmxremote.ConnectorBootstrap.startRemoteConnectorServer(ConnectorBootstrap.java:479)
        at jdk.management.agent/jdk.internal.agent.Agent.startAgent(Agent.java:447)
        at jdk.management.agent/jdk.internal.agent.Agent.startAgent(Agent.java:599)
Caused by: java.rmi.server.ExportException: Port already in use: 4000; nested exception is: 
        java.net.BindException: Address already in use (Bind failed)
        at java.rmi/sun.rmi.transport.tcp.TCPTransport.listen(TCPTransport.java:335)
        at java.rmi/sun.rmi.transport.tcp.TCPTransport.exportObject(TCPTransport.java:243)
        at java.rmi/sun.rmi.transport.tcp.TCPEndpoint.exportObject(TCPEndpoint.java:412)
        at java.rmi/sun.rmi.transport.LiveRef.exportObject(LiveRef.java:147)
        at java.rmi/sun.rmi.server.UnicastServerRef.exportObject(UnicastServerRef.java:234)
        at jdk.management.agent/sun.management.jmxremote.ConnectorBootstrap$PermanentExporter.exportObject(ConnectorBootstrap.java:203)
        at java.management.rmi/javax.management.remote.rmi.RMIJRMPServerImpl.export(RMIJRMPServerImpl.java:153)
        at java.management.rmi/javax.management.remote.rmi.RMIJRMPServerImpl.export(RMIJRMPServerImpl.java:138)
        at java.management.rmi/javax.management.remote.rmi.RMIConnectorServer.start(RMIConnectorServer.java:473)
        at jdk.management.agent/sun.management.jmxremote.ConnectorBootstrap.exportMBeanServer(ConnectorBootstrap.java:816)
        ... 3 more
Caused by: java.net.BindException: Address already in use (Bind failed)
        at java.base/java.net.PlainSocketImpl.socketBind(Native Method)
        at java.base/java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:452)
        at java.base/java.net.ServerSocket.bind(ServerSocket.java:395)
        at java.base/java.net.ServerSocket.<init>(ServerSocket.java:257)
        at java.base/java.net.ServerSocket.<init>(ServerSocket.java:149)
        at java.rmi/sun.rmi.transport.tcp.TCPDirectSocketFactory.createServerSocket(TCPDirectSocketFactory.java:45)
        at java.rmi/sun.rmi.transport.tcp.TCPEndpoint.newServerSocket(TCPEndpoint.java:670)
        at java.rmi/sun.rmi.transport.tcp.TCPTransport.listen(TCPTransport.java:324)
        ... 12 more

Anything else?

Changing the jmx port has the same outcome

[MarkEWaite]

Thanks for the issue report. I believe the issue is that the plugin installation manager tool is honoring the JAVA_OPTS value that is being passed to enable JMX, but the plugin installation manager tool is not the location where you intend to enable JMX. You want JMX enabled in Jenkins core while it is running.

If I use the following Dockerfile, the build of the container fails:

FROM jenkins/jenkins:2.387.2-lts-jdk11
ENV JAVA_OPTS=-Dcom.sun.management.jmxremote.port=4000 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false
COPY plugins.txt /usr/share/jenkins/
RUN jenkins-plugin-cli --plugin-file /usr/share/jenkins/plugins.txt

If I change that Dockerfile slightly, it works:

FROM jenkins/jenkins:2.387.2-lts-jdk11
COPY plugins.txt /usr/share/jenkins/
RUN jenkins-plugin-cli --plugin-file /usr/share/jenkins/plugins.txt
ENV JAVA_OPTS=-Dcom.sun.management.jmxremote.port=4000 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false

I don't know enough about the helm charts to know if there is a way for you to delay the configuration of the JAVA_OPTS value until after jenkins-plugin-cli has run, but that might be something worth investigating.

[psimms-r7]

Thank you for replying and looking at this for me

The problem is more for when there are deployment issues (and need to run the plugins-cli) or wanting to use the cli post deployment - I don't see these problems during the deployment itself but rather when using the cli on a live jenkins instance - ie. I ssh on and manually run it

But as you raised I see the JAVA_OPTS environment variable is there with the jmx args

So what java opts you start jenkins with the plugin-cli will also pickup? When ran manually on an already active jenkins instance the plugin-cli will get the same - is there a way to avoid this, or is there a need for java opts for the cli at all? Or can the plugin-cli point to a different env var so not to clash with the jenkins opts

As a workaround can do something like this I suppose which solves my problem here

JAVA_OPTS="" jenkins-plugin-cli --version

[MarkEWaite]

So what java opts you start jenkins with the plugin-cli will also pickup?

Yes, that's what I observed.

JAVA_OPTS might be helpful for the plugin installation manager tool when special arguments are needed in order to access the public internet. Otherwise, I don't know of a case where JAVA_OPTS would be needed. The shell script wrapper for the plugin installation manager tool definitely reads the JAVA_OPTS environment and passes it to the java command when running it.