Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Odd logging behavior when parquet export is enabled #3070

Closed
lmsurpre opened this issue Dec 3, 2021 · 1 comment
Closed

Odd logging behavior when parquet export is enabled #3070

lmsurpre opened this issue Dec 3, 2021 · 1 comment
Labels
bug Something isn't working bulk-data

Comments

@lmsurpre
Copy link
Member

lmsurpre commented Dec 3, 2021 •
edited
Loading

Describe the bug
log messages start out like this:

[11/29/21, 20:21:36:712 UTC] 00000034 BulkDataClien 1   $export response (HTTP 201)
[11/29/21, 20:21:36:725 UTC] 00000034 FHIRRestServl I   Completed request[1.279 secs]: tenantId:[default] dsId:[default] user:[fhiruser] method:[POST] uri:[https://localhost:9443/fhir-server/api/v4/$export] status:[202]
[11/29/21, 20:21:37:010 UTC] 00000033 AuditLogServi I   Successfully initialized audit log service: com.ibm.fhir.audit.impl.NopService
[11/29/21, 20:21:37:086 UTC] 00000033 ResourcePaylo 1   Max resources Per Object: 200000
[11/29/21, 20:21:37:087 UTC] 00000033 ResourcePaylo 1   Part Upload Trigger Size: 104857600
[11/29/21, 20:21:37:087 UTC] 00000033 ResourcePaylo 1   Max Object Size (threshold): 209715200

when we get a bulkdata request (when stocator and spark are included in the bulkdata-webapp), log4j kicks in:

11/29/21, 20:21:37:869 UTC] 00000033 SystemErr     R   log4j:WARN No appenders could be found for logger (com.ibm.cloud.objectstorage.AmazonWebServiceClient).
[11/29/21, 20:21:37:870 UTC] 00000033 SystemErr     R   log4j:WARN Please initialize the log4j system properly.
[11/29/21, 20:21:37:870 UTC] 00000033 SystemErr     R   log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
[11/29/21, 20:21:38:146 UTC] 00000033 S3Provider    1   Succeed to get client!
[11/29/21, 20:21:38:147 UTC] 00000033 S3Provider    1   Succeed to get BucketName!
[11/29/21, 20:21:38:765 UTC] 00000033 SystemErr     R   Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
[11/29/21, 20:21:38:765 UTC] 00000033 SystemErr     R   21/11/29 20:21:38 WARN Utils: Your hostname, Lees-MacBook-Pro-2.local resolves to a loopback address: 127.0.0.1; using 192.168.0.108 instead (on interface en0)
[11/29/21, 20:21:38:766 UTC] 00000033 SystemErr     R   21/11/29 20:21:38 WARN Utils: Set SPARK_LOCAL_IP if you need to bind to another address
...
[11/29/21, 20:21:44:676 UTC] 00000033 SystemErr     R   21/11/29 20:21:44 INFO ResourcePayloadReader: bulkexportfastjob[1] processed 1690 resources in 2.82 seconds (rate=598.7 resources/second)
[11/29/21, 20:21:44:692 UTC] 00000039 SystemErr     R   21/11/29 20:21:44 INFO ExportJobListener: bulkexportfastjob[1]                     Practitioner       1690
[11/29/21, 20:21:44:692 UTC] 00000039 SystemErr     R   21/11/29 20:21:44 INFO ExportJobListener: bulkexportfastjob[1] -------------------------------- ----------
[11/29/21, 20:21:44:692 UTC] 00000039 SystemErr     R   21/11/29 20:21:44 INFO ExportJobListener: bulkexportfastjob[1]                            TOTAL       1690
[11/29/21, 20:21:58:267 UTC] 0000008a SystemErr     R   21/11/29 20:21:58 WARN GarbageCollectionMetrics: To enable non-built-in garbage collector(s) List(scavenge), users should configure it(them) to spark.eventLog.gcMetrics.youngGenerationGarbageCollectors or spark.eventLog.gcMetrics.oldGenerationGarbageCollectors
[11/29/21, 20:21:58:268 UTC] 0000008a SystemErr     R   21/11/29 20:21:58 WARN GarbageCollectionMetrics: To enable non-built-in garbage collector(s) List(global, scavenge), users should configure it(them) to spark.eventLog.gcMetrics.youngGenerationGarbageCollectors or spark.eventLog.gcMetrics.oldGenerationGarbageCollectors

and after that, we see the log4j-style logs for liberty messages (non-bulkdata) as well:

[11/29/21, 20:24:56:827 UTC] 00000035 SystemErr     R   21/11/29 20:24:56 INFO FHIRRestServletFilter: Received request: tenantId:[default] dsId:[default] user:[fhiruser] method:[GET] uri:[https://localhost:9443/fhir-server/api/v4/Patient]
[11/29/21, 20:24:56:896 UTC] 00000035 SystemErr     R   21/11/29 20:24:56 INFO FHIRRestServletFilter: Completed request[0.069 secs]: tenantId:[default] dsId:[default] user:[fhiruser] method:[GET] uri:[https://localhost:9443/fhir-server/api/v4/Patient] status:[200]

Environment
main

To Reproduce
Steps to reproduce the behavior:

  1. change the fhir-bulkdata-webapp dependencies on spark and stocator from provided to runtime (default)
  2. start the server
  3. issue a bulk export
  4. issue a normal fhir request

Expected behavior
bulkdata capability shouldn't affect server logging

Additional context
We've had a long history of issues with Spark's packaging of log4j and slf4j.
Liberty (and our FHIR Server impl) uses java.util.logging APIs.
But spark has a hard dependency on jul-to-slf4j and then a specific slf4j binding to log4j.
I think that is what is messing things up.
I tried to exclude the jul-to-slf4j dependency but I can't seem to make it work.
I think its another reason to remove this experimental feature from the server and simplify our lives.
You can always use spark to convert NDJSON to parquet after the export is complete...

My notes from the effort to try removing jul-to-slf4j:

WITH slf4j-jdk14
WITHOUT log4j

[11/29/21, 22:29:55:321 UTC] [step1 partition0] com.ibm.fhir.audit.AuditLogServiceFactory                      Successfully initialized audit log service: com.ibm.fhir.audit.impl.NopService
[11/29/21, 22:29:55:331 UTC] [step1 partition0] com.ibm.fhir.bulkdata.jbatch.export.fast.ResourcePayloadRead   Max resources Per Object: 200000
[11/29/21, 22:29:55:331 UTC] [step1 partition0] com.ibm.fhir.bulkdata.jbatch.export.fast.ResourcePayloadRead   Part Upload Trigger Size: 104857600
[11/29/21, 22:29:55:332 UTC] [step1 partition0] com.ibm.fhir.bulkdata.jbatch.export.fast.ResourcePayloadRead   Max Object Size (threshold): 209715200
[11/29/21, 22:29:55:421 UTC] [step1 partition0] com.ibm.fhir.bulkdata.provider.impl.S3Provider                 Connecting to S3: [https://s3.us-east.cloud-object-storage.appdomain.cloud] [us-east]
[11/29/21, 22:29:56:058 UTC] [step1 partition0] com.ibm.fhir.bulkdata.provider.impl.S3Provider                 getClient The Path Style access is 'true'
[11/29/21, 22:29:56:386 UTC] [step1 partition0] com.ibm.fhir.bulkdata.provider.impl.S3Provider                 Succeed to get client!
[11/29/21, 22:29:56:387 UTC] [step1 partition0] com.ibm.fhir.bulkdata.provider.impl.S3Provider                 Succeed to get BucketName!
[11/29/21, 22:29:56:568 UTC] [step1 partition0] com.ibm.ws.batch.JobLogger                                     CWWKY0024W: The current chunk was rolled back to the previous checkpoint for step step1 in job instance 1 and job execution 1. Step metrics = [(READ_SKIP_COUNT,0), (PROCESS_SKIP_COUNT,0), (WRITE_SKIP_COUNT,0), (FILTER_COUNT,0), (COMMIT_COUNT,0), (READ_COUNT,0), (WRITE_COUNT,0), (ROLLBACK_COUNT,0)]
[11/29/21, 22:29:56:607 UTC] [step1 partition0] com.ibm.ws.batch.JobLogger                                     CWWKY0030I: An exception occurred while running the step step1.
com.ibm.jbatch.container.exception.BatchContainerRuntimeException: Failure in Read-Process-Write Loop
	at com.ibm.jbatch.container.controller.impl.ChunkStepControllerImpl.invokeChunk(ChunkStepControllerImpl.java:704)
	at com.ibm.jbatch.container.controller.impl.ChunkStepControllerImpl.invokeCoreStep(ChunkStepControllerImpl.java:795)
	at com.ibm.jbatch.container.controller.impl.BaseStepControllerImpl.execute(BaseStepControllerImpl.java:295)
	at com.ibm.jbatch.container.controller.impl.ExecutionTransitioner.doExecutionLoop(ExecutionTransitioner.java:118)
	at com.ibm.jbatch.container.controller.impl.WorkUnitThreadControllerImpl.executeCoreTransitionLoop(WorkUnitThreadControllerImpl.java:96)
	at com.ibm.jbatch.container.controller.impl.WorkUnitThreadControllerImpl.executeWorkUnit(WorkUnitThreadControllerImpl.java:178)
	at com.ibm.jbatch.container.controller.impl.WorkUnitThreadControllerImpl$AbstractControllerHelper.runExecutionOnThread(WorkUnitThreadControllerImpl.java:503)
	at com.ibm.jbatch.container.controller.impl.WorkUnitThreadControllerImpl.runExecutionOnThread(WorkUnitThreadControllerImpl.java:92)
	at <unknown class>.run(BatchWorkUnit.java:113)
	at com.ibm.ws.context.service.serializable.ContextualRunnable.run(ContextualRunnable.java:79)
	at com.ibm.ws.threading.internal.ExecutorServiceImpl$RunnableWrapper.run(ExecutorServiceImpl.java:238)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:866)
Caused by: java.lang.NoClassDefFoundError: org.apache.log4j.spi.Filter
	at java.base/java.lang.Class.forNameImpl(Native Method)
	at java.base/java.lang.Class.forName(Class.java:338)
	at com.ibm.fhir.bulkdata.provider.impl.S3Provider.<init>(S3Provider.java:139)
	at com.ibm.fhir.bulkdata.jbatch.export.fast.ResourcePayloadReader.open(ResourcePayloadReader.java:278)
	at com.ibm.jbatch.container.artifact.proxy.ItemReaderProxy.open(ItemReaderProxy.java:62)
	at com.ibm.jbatch.container.controller.impl.ChunkStepControllerImpl.openReaderAndWriter(ChunkStepControllerImpl.java:953)
	at com.ibm.jbatch.container.controller.impl.ChunkStepControllerImpl.invokeChunk(ChunkStepControllerImpl.java:599)
	... 15 more
Caused by: java.lang.ClassNotFoundException: org.apache.log4j.spi.Filter
	at com.ibm.ws.classloading.internal.AppClassLoader.findClassCommonLibraryClassLoaders(AppClassLoader.java:654)
	at com.ibm.ws.classloading.internal.AppClassLoader.findClass(AppClassLoader.java:296)
	at java.base/java.lang.ClassLoader.loadClassHelper(ClassLoader.java:1188)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:1103)
	at com.ibm.ws.classloading.internal.AppClassLoader.findOrDelegateLoadClass(AppClassLoader.java:632)
	at com.ibm.ws.classloading.internal.AppClassLoader.loadClass(AppClassLoader.java:531)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:1086)
	... 22 more




WITH slf4j-jdk14
WITH log4j

[11/29/21, 22:45:39:647 UTC] [step1 partition0] com.ibm.ws.batch.JobLogger                                     CWWKY0024W: The current chunk was rolled back to the previous checkpoint for step step1 in job instance 1 and job execution 1. Step metrics = [(READ_SKIP_COUNT,0), (PROCESS_SKIP_COUNT,0), (WRITE_SKIP_COUNT,0), (FILTER_COUNT,0), (COMMIT_COUNT,0), (READ_COUNT,0), (WRITE_COUNT,0), (ROLLBACK_COUNT,0)]
[11/29/21, 22:45:39:702 UTC] [step1 partition0] com.ibm.ws.batch.JobLogger                                     CWWKY0030I: An exception occurred while running the step step1.
com.ibm.jbatch.container.exception.BatchContainerRuntimeException: Failure in Read-Process-Write Loop
	at com.ibm.jbatch.container.controller.impl.ChunkStepControllerImpl.invokeChunk(ChunkStepControllerImpl.java:704)
	at com.ibm.jbatch.container.controller.impl.ChunkStepControllerImpl.invokeCoreStep(ChunkStepControllerImpl.java:795)
	at com.ibm.jbatch.container.controller.impl.BaseStepControllerImpl.execute(BaseStepControllerImpl.java:295)
	at com.ibm.jbatch.container.controller.impl.ExecutionTransitioner.doExecutionLoop(ExecutionTransitioner.java:118)
	at com.ibm.jbatch.container.controller.impl.WorkUnitThreadControllerImpl.executeCoreTransitionLoop(WorkUnitThreadControllerImpl.java:96)
	at com.ibm.jbatch.container.controller.impl.WorkUnitThreadControllerImpl.executeWorkUnit(WorkUnitThreadControllerImpl.java:178)
	at com.ibm.jbatch.container.controller.impl.WorkUnitThreadControllerImpl$AbstractControllerHelper.runExecutionOnThread(WorkUnitThreadControllerImpl.java:503)
	at com.ibm.jbatch.container.controller.impl.WorkUnitThreadControllerImpl.runExecutionOnThread(WorkUnitThreadControllerImpl.java:92)
	at <unknown class>.run(BatchWorkUnit.java:113)
	at com.ibm.ws.context.service.serializable.ContextualRunnable.run(ContextualRunnable.java:79)
	at com.ibm.ws.threading.internal.ExecutorServiceImpl$RunnableWrapper.run(ExecutorServiceImpl.java:238)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:866)
Caused by: java.lang.NoClassDefFoundError: org.slf4j.impl.StaticLoggerBinder
	at org.apache.spark.internal.Logging$.org$apache$spark$internal$Logging$$isLog4j12(Logging.scala:222)
	at org.apache.spark.internal.Logging.initializeLogging(Logging.scala:127)
	at org.apache.spark.internal.Logging.initializeLogIfNecessary(Logging.scala:111)
	at org.apache.spark.internal.Logging.initializeLogIfNecessary$(Logging.scala:105)
	at org.apache.spark.util.Utils$.initializeLogIfNecessary(Utils.scala:89)
	at org.apache.spark.internal.Logging.initializeLogIfNecessary(Logging.scala:102)
	at org.apache.spark.internal.Logging.initializeLogIfNecessary$(Logging.scala:101)
	at org.apache.spark.util.Utils$.initializeLogIfNecessary(Utils.scala:89)
	at org.apache.spark.internal.Logging.log(Logging.scala:49)
	at org.apache.spark.internal.Logging.log$(Logging.scala:47)
	at org.apache.spark.util.Utils$.log(Utils.scala:89)
	at org.apache.spark.internal.Logging.logWarning(Logging.scala:69)
	at org.apache.spark.internal.Logging.logWarning$(Logging.scala:68)
	at org.apache.spark.util.Utils$.logWarning(Utils.scala:89)
	at org.apache.spark.util.Utils$.$anonfun$findLocalInetAddress$1(Utils.scala:1025)
	at org.apache.spark.util.Utils$.$anonfun$findLocalInetAddress$1$adapted(Utils.scala:1015)
	at <unknown class>.apply(Unknown Source)
	at scala.collection.immutable.Stream.foreach(Stream.scala:533)
	at org.apache.spark.util.Utils$.findLocalInetAddress(Utils.scala:1015)
	at org.apache.spark.util.Utils$.localIpAddress$lzycompute(Utils.scala:998)
	at org.apache.spark.util.Utils$.localIpAddress(Utils.scala:998)
	at org.apache.spark.util.Utils$.$anonfun$localCanonicalHostName$1(Utils.scala:1055)
	at <unknown class>.apply(Unknown Source)
	at scala.Option.getOrElse(Option.scala:189)
	at org.apache.spark.util.Utils$.localCanonicalHostName(Utils.scala:1055)
	at org.apache.spark.internal.config.package$.<init>(package.scala:990)
	at org.apache.spark.internal.config.package$.<clinit>(package.scala)
	at org.apache.spark.SparkConf$.<init>(SparkConf.scala:654)
	at org.apache.spark.SparkConf$.<clinit>(SparkConf.scala)
	at org.apache.spark.SparkConf.set(SparkConf.scala:94)
	at org.apache.spark.SparkConf.set(SparkConf.scala:83)
	at org.apache.spark.sql.SparkSession$Builder.$anonfun$getOrCreate$1(SparkSession.scala:920)
	at <unknown class>.apply(Unknown Source)
	at scala.collection.mutable.HashMap.$anonfun$foreach$1(HashMap.scala:149)
	at <unknown class>.apply(Unknown Source)
	at scala.collection.mutable.HashTable.foreachEntry(HashTable.scala:237)
	at scala.collection.mutable.HashTable.foreachEntry$(HashTable.scala:230)
	at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:44)
	at scala.collection.mutable.HashMap.foreach(HashMap.scala:149)
	at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:920)
	at com.ibm.fhir.bulkdata.export.writer.SparkParquetWriter.<init>(SparkParquetWriter.java:74)
	at com.ibm.fhir.bulkdata.provider.impl.S3Provider.<init>(S3Provider.java:140)
	at com.ibm.fhir.bulkdata.jbatch.export.fast.ResourcePayloadReader.open(ResourcePayloadReader.java:278)
	at com.ibm.jbatch.container.artifact.proxy.ItemReaderProxy.open(ItemReaderProxy.java:62)
	at com.ibm.jbatch.container.controller.impl.ChunkStepControllerImpl.openReaderAndWriter(ChunkStepControllerImpl.java:953)
	at com.ibm.jbatch.container.controller.impl.ChunkStepControllerImpl.invokeChunk(ChunkStepControllerImpl.java:599)
	... 15 more
Caused by: java.lang.ClassNotFoundException: org.slf4j.impl.StaticLoggerBinder
	at com.ibm.ws.classloading.internal.AppClassLoader.findClassCommonLibraryClassLoaders(AppClassLoader.java:654)
	at com.ibm.ws.classloading.internal.AppClassLoader.findClass(AppClassLoader.java:296)
	at java.base/java.lang.ClassLoader.loadClassHelper(ClassLoader.java:1188)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:1103)
	at com.ibm.ws.classloading.internal.AppClassLoader.findOrDelegateLoadClass(AppClassLoader.java:632)
	at com.ibm.ws.classloading.internal.AppClassLoader.loadClass(AppClassLoader.java:531)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:1086)
	... 61 more


WITH log4j
WITH slf4j-log4j12

11/30/21, 4:30:07:814 UTC] 00000039 StepChunkList E   StepChunkListener: job[bulkexportchunkjob/2/3] --- java.lang.reflect.InvocationTargetException
                                 com.ibm.jbatch.container.exception.BatchContainerRuntimeException: java.lang.reflect.InvocationTargetException
	at com.ibm.jbatch.container.controller.impl.ChunkStepControllerImpl.writeChunk(ChunkStepControllerImpl.java:503)
	at com.ibm.jbatch.container.controller.impl.ChunkStepControllerImpl.invokeChunk(ChunkStepControllerImpl.java:636)
	at com.ibm.jbatch.container.controller.impl.ChunkStepControllerImpl.invokeCoreStep(ChunkStepControllerImpl.java:795)
	at com.ibm.jbatch.container.controller.impl.BaseStepControllerImpl.execute(BaseStepControllerImpl.java:295)
	at com.ibm.jbatch.container.controller.impl.ExecutionTransitioner.doExecutionLoop(ExecutionTransitioner.java:118)
	at com.ibm.jbatch.container.controller.impl.WorkUnitThreadControllerImpl.executeCoreTransitionLoop(WorkUnitThreadControllerImpl.java:96)
	at com.ibm.jbatch.container.controller.impl.WorkUnitThreadControllerImpl.executeWorkUnit(WorkUnitThreadControllerImpl.java:178)
	at com.ibm.jbatch.container.controller.impl.WorkUnitThreadControllerImpl$AbstractControllerHelper.runExecutionOnThread(WorkUnitThreadControllerImpl.java:503)
	at com.ibm.jbatch.container.controller.impl.WorkUnitThreadControllerImpl.runExecutionOnThread(WorkUnitThreadControllerImpl.java:92)
	at <unknown class>.run(BatchWorkUnit.java:113)
	at com.ibm.ws.context.service.serializable.ContextualRunnable.run(ContextualRunnable.java:79)
	at com.ibm.ws.threading.internal.ExecutorServiceImpl$RunnableWrapper.run(ExecutorServiceImpl.java:238)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:866)
Caused by: java.lang.reflect.InvocationTargetException
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
	at org.apache.spark.sql.execution.datasources.DataSource.providingInstance(DataSource.scala:113)
	at org.apache.spark.sql.execution.datasources.DataSource.planForWriting(DataSource.scala:551)
	at org.apache.spark.sql.DataFrameWriter.saveToV1Source(DataFrameWriter.scala:382)
	at org.apache.spark.sql.DataFrameWriter.saveInternal(DataFrameWriter.scala:355)
	at org.apache.spark.sql.DataFrameWriter.save(DataFrameWriter.scala:239)
	at org.apache.spark.sql.DataFrameWriter.parquet(DataFrameWriter.scala:781)
	at com.ibm.fhir.bulkdata.export.writer.SparkParquetWriter.writeParquet(SparkParquetWriter.java:106)
	at com.ibm.fhir.bulkdata.provider.impl.S3Provider.pushFhirParquetToCos(S3Provider.java:459)
	at com.ibm.fhir.bulkdata.provider.impl.S3Provider.writeResources(S3Provider.java:369)
	at com.ibm.fhir.bulkdata.jbatch.export.system.ChunkWriter.writeItems(ChunkWriter.java:102)
	at com.ibm.jbatch.container.artifact.proxy.ItemWriterProxy.writeItems(ItemWriterProxy.java:80)
	at com.ibm.jbatch.container.controller.impl.ChunkStepControllerImpl.writeChunk(ChunkStepControllerImpl.java:481)
	... 16 more
Caused by: java.lang.NoClassDefFoundError: org.slf4j.bridge.SLF4JBridgeHandler
	at org.apache.spark.sql.execution.datasources.parquet.ParquetFileFormat.<init>(ParquetFileFormat.scala:64)
	... 32 more
Caused by: java.lang.ClassNotFoundException: org.slf4j.bridge.SLF4JBridgeHandler
	at com.ibm.ws.classloading.internal.AppClassLoader.findClassCommonLibraryClassLoaders(AppClassLoader.java:654)
	at com.ibm.ws.classloading.internal.AppClassLoader.findClass(AppClassLoader.java:296)
	at java.base/java.lang.ClassLoader.loadClassHelper(ClassLoader.java:1188)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:1103)
	at com.ibm.ws.classloading.internal.AppClassLoader.findOrDelegateLoadClass(AppClassLoader.java:632)
	at com.ibm.ws.classloading.internal.AppClassLoader.loadClass(AppClassLoader.java:531)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:1086)
	... 33 more

I think the last one is the most damning because it shows that the spark ParquetFileFormat has a hard dependency on the SLF4JBridgeHandler that comes from the jul-to-slf4j jar.
@lmsurpre lmsurpre added the bug Something isn't working label Dec 3, 2021
@lmsurpre lmsurpre mentioned this issue Dec 3, 2021
Merged
@lmsurpre lmsurpre mentioned this issue Jan 5, 2022
Closed
@lmsurpre
Copy link
Member Author

the export parquet feature was removed which invalidates this one

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working bulk-data
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@prb112 @lmsurpre