You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
There many failed workloads in Mllib test.
Such as glm-regression. The following is the error log. I use spark 1.3.0 Can anyone help to find the
reason?
16/07/23 10:50:43 INFO SparkDeploySchedulerBackend: SchedulerBackend is ready for scheduling beginning after reached minRegisteredResourcesRatio: 0.0
Exception in thread "main" java.lang.NoClassDefFoundError: org.apache.spark.ml.attribute.Attribute
at mllib.perf.GLMRegressionTest.createInputData(MLAlgorithmTests.scala:119)
at mllib.perf.TestRunner$$anonfun$2.apply(TestRunner.scala:67)
at mllib.perf.TestRunner$$anonfun$2.apply(TestRunner.scala:66)
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
at scala.collection.immutable.Range.foreach(Range.scala:141)
at scala.collection.TraversableLike$class.map(TraversableLike.scala:244)
at scala.collection.AbstractTraversable.map(Traversable.scala:105)
at mllib.perf.TestRunner$.main(TestRunner.scala:66)
at mllib.perf.TestRunner.main(TestRunner.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:95)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:56)
at java.lang.reflect.Method.invoke(Method.java:620)
at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:569)
at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:166)
at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:189)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:110)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: java.lang.ClassNotFoundException: org.apache.spark.ml.attribute.Attribute
at java.net.URLClassLoader.findClass(URLClassLoader.java:600)
at java.lang.ClassLoader.loadClassHelper(ClassLoader.java:786)
at java.lang.ClassLoader.loadClass(ClassLoader.java:764)
at java.lang.ClassLoader.loadClass(ClassLoader.java:741)
... 19 more
Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
There many failed workloads in Mllib test.
Such as glm-regression. The following is the error log. I use spark 1.3.0 Can anyone help to find the
reason?
16/07/23 10:50:43 INFO SparkDeploySchedulerBackend: SchedulerBackend is ready for scheduling beginning after reached minRegisteredResourcesRatio: 0.0
Exception in thread "main" java.lang.NoClassDefFoundError: org.apache.spark.ml.attribute.Attribute
at mllib.perf.GLMRegressionTest.createInputData(MLAlgorithmTests.scala:119)
at mllib.perf.TestRunner$$anonfun$2.apply(TestRunner.scala:67)
at mllib.perf.TestRunner$$anonfun$2.apply(TestRunner.scala:66)
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
at scala.collection.immutable.Range.foreach(Range.scala:141)
at scala.collection.TraversableLike$class.map(TraversableLike.scala:244)
at scala.collection.AbstractTraversable.map(Traversable.scala:105)
at mllib.perf.TestRunner$.main(TestRunner.scala:66)
at mllib.perf.TestRunner.main(TestRunner.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:95)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:56)
at java.lang.reflect.Method.invoke(Method.java:620)
at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:569)
at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:166)
at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:189)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:110)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: java.lang.ClassNotFoundException: org.apache.spark.ml.attribute.Attribute
at java.net.URLClassLoader.findClass(URLClassLoader.java:600)
at java.lang.ClassLoader.loadClassHelper(ClassLoader.java:786)
at java.lang.ClassLoader.loadClass(ClassLoader.java:764)
at java.lang.ClassLoader.loadClass(ClassLoader.java:741)
... 19 more
Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
Java options: -Dspark.storage.memoryFraction=0.66 -Dspark.serializer=org.apache.spark.serializer.JavaSerializer -Dspark.locality.wait=60000000 -Dspark.shuffle.manager=SORT
Options: glm-regression --num-trials=10 --inter-trial-wait=3 --num-partitions=64 --random-seed=5 --num-examples=500000 --num-iterations=20 --optimizer=auto --reg-type=elastic-net --elastic-net-param=0.0 --reg-param=0.01 --feature-noise=1.0 --step-size=0.0 --label-noise=0.1 --intercept=0.2 --loss=l2 --num-features=10000
The text was updated successfully, but these errors were encountered: