Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ensure ClassLoaders have names for debugging #9277

Merged
merged 7 commits into from
May 18, 2024

Conversation

jtnord
Copy link
Member

@jtnord jtnord commented May 14, 2024

giving a ClassLoader a name can make it easier to debug class loading issues that occur on agents.

with jenkinsci/remoting#741 this gets propagated to the remote side so you can see what the RemoteClassLoader is actually loading from.

tested a build of this to try and debug an issue with an ssh-slaves based agent and added a breakpoint on the classloading code with I was having issues with and connected to the agent process.

Validated that the classloading issue I was attempting to diagnose has more diagnostics.

e.g.

image

Note: as the classloaders now have names, this will be shown in the default stack traces (similar to how we see java.base/java.net.URLClassLoader (due to this being present in the StackTraceElement

e.g.

		at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:44)
		at hudson.model.ResourceController.execute(ResourceController.java:101)
		at hudson.model.Executor.run(Executor.java:446)
Caused by: org.eclipse.jgit.api.errors.TransportException: ssh://172.17.32.1:2222/srv/git/your-repo.git/: remote hung up unexpectedly
	at PluginClassLoader for git-client//org.eclipse.jgit.api.FetchCommand.call(FetchCommand.java:249)
	at PluginClassLoader for git-client//org.jenkinsci.plugins.gitclient.JGitAPIImpl$3.execute(JGitAPIImpl.java:851)
	... 10 more
Caused by: org.eclipse.jgit.errors.TransportException: ssh://172.17.32.1:2222/srv/git/your-repo.git/: remote hung up unexpectedly
	at PluginClassLoader for git-client//org.eclipse.jgit.transport.TransportGitSsh$SshFetchConnection.<init>(TransportGitSsh.java:311)
	at PluginClassLoader for git-client//org.eclipse.jgit.transport.TransportGitSsh.openFetch(TransportGitSsh.java:152)
	at PluginClassLoader for git-client//org.eclipse.jgit.transport.FetchProcess.executeImp(FetchProcess.java:153)
	at PluginClassLoader for git-client//org.eclipse.jgit.transport.FetchProcess.execute(FetchProcess.java:105)
	at PluginClassLoader for git-client//org.eclipse.jgit.transport.Transport.fetch(Transport.java:1482)
	at PluginClassLoader for git-client//org.eclipse.jgit.api.FetchCommand.call(FetchCommand.java:238)
	... 11 more
Caused by: java.lang.NoClassDefFoundError: net/i2p/crypto/eddsa/spec/EdDSAParameterSpec
	at PluginClassLoader for mina-sshd-api-common//org.apache.sshd.common.util.security.SecurityUtils.getEDDSAPublicKeyEntryDecoder(SecurityUtils.java:580)
	at PluginClassLoader for mina-sshd-api-common//org.apache.sshd.common.config.keys.KeyUtils.<clinit>(KeyUtils.java:187)
	at PluginClassLoader for mina-sshd-api-common//org.apache.sshd.common.config.keys.loader.openssh.OpenSSHRSAPrivateKeyDecoder.<clinit>(OpenSSHRSAPrivateKeyDecoder.java:51)
	at PluginClassLoader for mina-sshd-api-common//org.apache.sshd.common.config.keys.loader.openssh.OpenSSHKeyPairResourceParser.<clinit>(OpenSSHKeyPairResourceParser.java:89)
	at PluginClassLoader for mina-sshd-api-common//org.apache.sshd.common.util.security.SecurityUtils.getKeyPairResourceParser(SecurityUtils.java:681)
	at PluginClassLoader for mina-sshd-api-common//org.apache.sshd.common.util.security.SecurityUtils.loadKeyPairIdentities(SecurityUtils.java:518)
	at PluginClassLoader for git-client//org.jenkinsci.plugins.gitclient.JGitAPIImpl$1.getDefaultKeys(JGitAPIImpl.java:307)
	at PluginClassLoader for git-client//org.eclipse.jgit.transport.sshd.SshdSessionFactory.lambda$0(SshdSessionFactory.java:212)
	at PluginClassLoader for git-client//org.eclipse.jgit.transport.sshd.SshdSession.<init>(SshdSession.java:93)
	at PluginClassLoader for git-client//org.eclipse.jgit.transport.sshd.SshdSessionFactory.getSession(SshdSessionFactory.java:196)
	at PluginClassLoader for git-client//org.eclipse.jgit.transport.sshd.SshdSessionFactory.getSession(SshdSessionFactory.java:1)
	at PluginClassLoader for git-client//org.eclipse.jgit.transport.SshTransport.getSession(SshTransport.java:107)
	at PluginClassLoader for git-client//org.eclipse.jgit.transport.TransportGitSsh$SshFetchConnection.<init>(TransportGitSsh.java:279)
	... 16 more
Caused by: java.lang.ClassNotFoundException: net.i2p.crypto.eddsa.spec.EdDSAParameterSpec
	at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:476)
	at jenkins.util.URLClassLoader2.findClass(URLClassLoader2.java:64)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:594)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:527)
	at hudson.remoting.RemoteClassLoader$ClassLoaderProxy.fetch4(RemoteClassLoader.java:1014)
	at hudson.remoting.RemoteClassLoader$ClassLoaderProxy.fetch3(RemoteClassLoader.java:1059)
	at jdk.internal.reflect.GeneratedMethodAccessor7.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:924)
	at hudson.remoting.Request$2.run(Request.java:377)
	at hudson.remoting.InterceptingExecutorService.lambda$wrap$0(InterceptingExecutorService.java:78)
	at org.jenkinsci.remoting.CallableDecorator.call(CallableDecorator.java:18)
	at hudson.remoting.CallableDecoratorList.lambda$applyDecorator$0(CallableDecoratorList.java:19)
	at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)
	at jenkins.security.ImpersonatingExecutorService$2.call(ImpersonatingExecutorService.java:80)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)
	Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call to channel
		at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1787)
		at hudson.remoting.Request.call(Request.java:199)
		at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:288)
		at com.sun.proxy.$Proxy7.fetch3(Unknown Source)
		at hudson.remoting.RemoteClassLoader.prefetchClassReference(RemoteClassLoader.java:362)
		at hudson.remoting.RemoteClassLoader.loadWithMultiClassLoader(RemoteClassLoader.java:267)
		at hudson.remoting.RemoteClassLoader.findClass(RemoteClassLoader.java:237)
		at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:594)
		at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:527)
		at PluginClassLoader for mina-sshd-api-common//org.apache.sshd.common.util.security.SecurityUtils.getEDDSAPublicKeyEntryDecoder(SecurityUtils.java:580)
		at PluginClassLoader for mina-sshd-api-common//org.apache.sshd.common.config.keys.KeyUtils.<clinit>(KeyUtils.java:187)

Testing done

interactive debugging with an agent from ssh-slaves plugin and a breakpoint on some code that loads classes.
inspected the classloader for several classes that are expected to come from different plugins.

Proposed changelog entries

  • Adds debugging information to remote classloaders.

Proposed upgrade guidelines

Agents that provide their own version of agent.jar will need to be updated to to use at least version 3244.vf7f977e04755 of remoting.
Generally this would impact inbound agents, ssh based agents should be updated automatically.

[tasklist]

Submitter checklist

  • The Jira issue, if it exists, is well-described.
  • The changelog entries and upgrade guidelines are appropriate for the audience affected by the change (users or developers, depending on the change) and are in the imperative mood (see examples). Fill in the Proposed upgrade guidelines section only if there are breaking changes or changes that may require extra steps from users during upgrade.
  • There is automated testing or an explanation as to why this change has no tests.
  • New public classes, fields, and methods are annotated with @Restricted or have @since TODO Javadocs, as appropriate.
  • New deprecations are annotated with @Deprecated(since = "TODO") or @Deprecated(forRemoval = true, since = "TODO"), if applicable.
  • New or substantially changed JavaScript is not defined inline and does not call eval to ease future introduction of Content Security Policy (CSP) directives (see documentation).
  • For dependency updates, there are links to external changelogs and, if possible, full differentials.
  • For new APIs and extension points, there is a link to at least one consumer.

Desired reviewers

@mention

Before the changes are marked as ready-for-merge:

[tasklist]

Maintainer checklist

  • There are at least two (2) approvals for the pull request and no outstanding requests for change.
  • Conversations in the pull request are over, or it is explicit that a reviewer is not blocking the change.
  • Changelog entries in the pull request title and/or Proposed changelog entries are accurate, human-readable, and in the imperative mood.
  • Proper changelog labels are set so that the changelog can be generated automatically.
  • If the change needs additional upgrade steps from users, the upgrade-guide-needed label is set and there is a Proposed upgrade guidelines section in the pull request title (see example).
  • If it would make sense to backport the change to LTS, a Jira issue must exist, be a Bug or Improvement, and be labeled as lts-candidate to be considered (see query).

pom.xml Outdated Show resolved Hide resolved
@jtnord jtnord added the developer Changes which impact plugin developers label May 14, 2024
public PluginFirstClassLoader2(@NonNull URL[] urls, @NonNull ClassLoader parent) {
super(Objects.requireNonNull(urls), Objects.requireNonNull(parent));

public PluginFirstClassLoader2(String name, @NonNull URL[] urls, @NonNull ClassLoader parent) {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

class is annotated with @Restricted(NoExternalUse.class) which is not visible in the collapsed diff.

@jtnord jtnord marked this pull request as ready for review May 15, 2024 09:50
@jtnord jtnord requested review from jglick and a team May 15, 2024 09:56
pom.xml Outdated Show resolved Hide resolved
pom.xml Outdated Show resolved Hide resolved
@jglick jglick changed the title Ensure Classloaders have names for debugging Ensure ClassLoaders have names for debugging May 15, 2024
Copy link
Member

@jglick jglick left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

core/src/main/java/hudson/ClassicPluginStrategy.java Outdated Show resolved Hide resolved
core/src/main/java/hudson/ClassicPluginStrategy.java Outdated Show resolved Hide resolved
core/src/main/java/hudson/ClassicPluginStrategy.java Outdated Show resolved Hide resolved
core/src/main/java/hudson/ClassicPluginStrategy.java Outdated Show resolved Hide resolved
core/src/main/java/hudson/PluginManager.java Outdated Show resolved Hide resolved
core/src/main/java/jenkins/util/URLClassLoader2.java Outdated Show resolved Hide resolved
core/src/main/java/jenkins/util/URLClassLoader2.java Outdated Show resolved Hide resolved
core/src/main/java/jenkins/util/URLClassLoader2.java Outdated Show resolved Hide resolved
core/src/main/java/jenkins/util/URLClassLoader2.java Outdated Show resolved Hide resolved
Co-authored-by: Jesse Glick <jglick@cloudbees.com>
pom.xml Show resolved Hide resolved
pom.xml Outdated Show resolved Hide resolved
@jtnord jtnord force-pushed the names-for-classloaders branch from e796d10 to e0c3211 Compare May 16, 2024 08:54
pom.xml Outdated Show resolved Hide resolved
@jtnord
Copy link
Member Author

jtnord commented May 16, 2024

/label ready-for-merge

@comment-ops-bot comment-ops-bot bot added the ready-for-merge The PR is ready to go, and it will be merged soon if there is no negative feedback label May 16, 2024
@MarkEWaite MarkEWaite merged commit 94f7154 into jenkinsci:master May 18, 2024
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
developer Changes which impact plugin developers ready-for-merge The PR is ready to go, and it will be merged soon if there is no negative feedback
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants