Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

StackOverflowError after enabling log collection #389

Open
m0n5t3r opened this issue Feb 1, 2024 · 3 comments · Fixed by #396
Open

StackOverflowError after enabling log collection #389

m0n5t3r opened this issue Feb 1, 2024 · 3 comments · Fixed by #396
Labels
kind/bug Bug related issue

Comments

@m0n5t3r
Copy link

m0n5t3r commented Feb 1, 2024

We've been getting stack overflows in a pipeline that uses stashedFile / the file parameter plugin; it appears to be the same issue as jenkinsci/file-parameters-plugin#182 (where they say the problem is in this plugin)

To Reproduce
Steps to reproduce the behavior:

  1. Enable datadog log collection in Manage Jenkins -> System
  2. Use something along the lines of stashedFile name: 'aFile', description: 'a file' as a parameter in a pipeline
  3. Try to run it
  4. The job fails with no console output, and the jenkins log shows a really long trace that starts like this, and just repeats the same list forever, bouncing between the file parameter plugin and the datadog plugin:
trace

Feb 01 00:53:27 jenkins jenkins[507501]: java.lang.StackOverflowError
Feb 01 00:53:27 jenkins jenkins[507501]:         at org.codehaus.groovy.reflection.ParameterTypes.coerceArgumentsToClasses(ParameterTypes.java:145)
Feb 01 00:53:27 jenkins jenkins[507501]:         at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:323)
Feb 01 00:53:27 jenkins jenkins[507501]:         at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:264)
Feb 01 00:53:27 jenkins jenkins[507501]:         at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1034)
Feb 01 00:53:27 jenkins jenkins[507501]:         at groovy.lang.Closure.call(Closure.java:420)
Feb 01 00:53:27 jenkins jenkins[507501]:         at groovy.lang.Closure$WritableClosure.writeTo(Closure.java:860)
Feb 01 00:53:27 jenkins jenkins[507501]:         at groovy.lang.Closure$WritableClosure.toString(Closure.java:986)
Feb 01 00:53:27 jenkins jenkins[507501]:         at io.jenkins.plugins.opentelemetry.backend.ObservabilityBackend.getTraceVisualisationUrl(ObservabilityBackend.java:114)
Feb 01 00:53:27 jenkins jenkins[507501]:         at io.jenkins.plugins.opentelemetry.job.MonitoringAction.lambda$getLinks$6(MonitoringAction.java:155)
Feb 01 00:53:27 jenkins jenkins[507501]:         at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
Feb 01 00:53:27 jenkins jenkins[507501]:         at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1625)
Feb 01 00:53:27 jenkins jenkins[507501]:         at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
Feb 01 00:53:27 jenkins jenkins[507501]:         at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
Feb 01 00:53:27 jenkins jenkins[507501]:         at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921)
Feb 01 00:53:27 jenkins jenkins[507501]:         at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
Feb 01 00:53:27 jenkins jenkins[507501]:         at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:682)
Feb 01 00:53:27 jenkins jenkins[507501]:         at io.jenkins.plugins.opentelemetry.job.MonitoringAction.getLinks(MonitoringAction.java:158)
Feb 01 00:53:27 jenkins jenkins[507501]:         at io.jenkins.plugins.opentelemetry.job.OtelEnvironmentContributorService.addEnvironmentVariables(OtelEnvironmentContributorService.java:58)
Feb 01 00:53:27 jenkins jenkins[507501]:         at io.jenkins.plugins.opentelemetry.job.OtelEnvironmentContributor.buildEnvironmentFor(OtelEnvironmentContributor.java:31)
Feb 01 00:53:27 jenkins jenkins[507501]:         at hudson.model.Run.getEnvironment(Run.java:2430)
Feb 01 00:53:27 jenkins jenkins[507501]:         at org.jenkinsci.plugins.workflow.job.WorkflowRun.getEnvironment(WorkflowRun.java:519)
Feb 01 00:53:27 jenkins jenkins[507501]:         at org.datadog.jenkins.plugins.datadog.model.BuildData.<init>(BuildData.java:150)
Feb 01 00:53:27 jenkins jenkins[507501]:         at org.datadog.jenkins.plugins.datadog.logs.DatadogTaskListenerDecorator.<init>(DatadogTaskListenerDecorator.java:49)
Feb 01 00:53:27 jenkins jenkins[507501]:         at org.datadog.jenkins.plugins.datadog.logs.DatadogTaskListenerDecorator$Factory.of(DatadogTaskListenerDecorator.java:82)
Feb 01 00:53:27 jenkins jenkins[507501]:         at org.jenkinsci.plugins.workflow.log.TaskListenerDecorator.lambda$apply$3(TaskListenerDecorator.java:164)
Feb 01 00:53:27 jenkins jenkins[507501]:         at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
Feb 01 00:53:27 jenkins jenkins[507501]:         at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:179)
Feb 01 00:53:27 jenkins jenkins[507501]:         at java.base/java.util.Iterator.forEachRemaining(Iterator.java:133)
Feb 01 00:53:27 jenkins jenkins[507501]:         at java.base/java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1845)
Feb 01 00:53:27 jenkins jenkins[507501]:         at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
Feb 01 00:53:27 jenkins jenkins[507501]:         at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
Feb 01 00:53:27 jenkins jenkins[507501]:         at java.base/java.util.stream.StreamSpliterators$WrappingSpliterator.forEachRemaining(StreamSpliterators.java:310)
Feb 01 00:53:27 jenkins jenkins[507501]:         at java.base/java.util.stream.Streams$ConcatSpliterator.forEachRemaining(Streams.java:735)
Feb 01 00:53:27 jenkins jenkins[507501]:         at java.base/java.util.stream.Streams$ConcatSpliterator.forEachRemaining(Streams.java:735)
Feb 01 00:53:27 jenkins jenkins[507501]:         at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
Feb 01 00:53:27 jenkins jenkins[507501]:         at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
Feb 01 00:53:27 jenkins jenkins[507501]:         at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921)
Feb 01 00:53:27 jenkins jenkins[507501]:         at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
Feb 01 00:53:27 jenkins jenkins[507501]:         at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:682)
Feb 01 00:53:27 jenkins jenkins[507501]:         at org.jenkinsci.plugins.workflow.log.TaskListenerDecorator.apply(TaskListenerDecorator.java:166)
Feb 01 00:53:27 jenkins jenkins[507501]:         at org.jenkinsci.plugins.workflow.job.WorkflowRun.getListener(WorkflowRun.java:236)
Feb 01 00:53:27 jenkins jenkins[507501]:         at org.jenkinsci.plugins.workflow.job.WorkflowRun$Owner.getListener(WorkflowRun.java:1024)
Feb 01 00:53:27 jenkins jenkins[507501]:         at io.jenkins.plugins.file_parameters.StashedFileParameterValue.buildEnvironment(StashedFileParameterValue.java:70)
Feb 01 00:53:27 jenkins jenkins[507501]:         at hudson.model.ParametersAction.buildEnvironment(ParametersAction.java:143)
Feb 01 00:53:27 jenkins jenkins[507501]:         at hudson.model.Run.getEnvironment(Run.java:2434)
Feb 01 00:53:27 jenkins jenkins[507501]:         at org.jenkinsci.plugins.workflow.job.WorkflowRun.getEnvironment(WorkflowRun.java:519)
Feb 01 00:53:27 jenkins jenkins[507501]:         at org.datadog.jenkins.plugins.datadog.model.BuildData.<init>(BuildData.java:150)
Feb 01 00:53:27 jenkins jenkins[507501]:         at org.datadog.jenkins.plugins.datadog.logs.DatadogTaskListenerDecorator.<init>(DatadogTaskListenerDecorator.java:49)
Feb 01 00:53:27 jenkins jenkins[507501]:         at org.datadog.jenkins.plugins.datadog.logs.DatadogTaskListenerDecorator$Factory.of(DatadogTaskListenerDecorator.java:82)
Feb 01 00:53:27 jenkins jenkins[507501]:         at org.jenkinsci.plugins.workflow.log.TaskListenerDecorator.lambda$apply$3(TaskListenerDecorator.java:164)
[...]

Expected behavior
Job runs

Environment and Versions (please complete the following information):
openjdk 17.0.9 2023-10-17
Jenkins 2.443
Datadog plugin 6.0.0
Datadog agent 6.50.3
File parameter plugin 316.va_83a_1221db_a_7

@m0n5t3r m0n5t3r added the kind/bug Bug related issue label Feb 1, 2024
@lemeurherve
Copy link
Member

lemeurherve commented Feb 7, 2024

We just had the same issue on ci.jenkins.io instance yesterday and today, here is the corresponding stack trace: https://gist.github.com/lemeurherve/eaf08053a133733a829c24ca0eec7117

A reboot of our instance fixed the stackoverflow error, we then updated Datadog version from 6.0.0 to 6.0.1

See also jenkins-infra/helpdesk#3934 (comment)

@nikita-tkachenko-datadog
Copy link
Collaborator

nikita-tkachenko-datadog commented Feb 8, 2024

Plugin v6.0.2 has been released.
@lemeurherve, it should fix the issue that you're experiencing.
It does not fix the originally reported issue with stashedFile, however, since the two are slightly different.

@lemeurherve
Copy link
Member

Updating ci.jenkins.io to 6.0.2 already fixed another issue, cf jenkins-infra/helpdesk#3944 (comment) 👏

I'll keep you updated if we encounter new stackoverflow errors or not in the following days.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug Bug related issue
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants