RabitTracker:179 - FAULT: failed to start tracker process

[Ayyan-droid]

hi i am getting this error can any guide me
spark version 3.1
xgboost4j_2.12-1.6.1.jar
xgboost4j-spark_2.12-1.6.1.jar

ERROR RabitTracker:179 - FAULT: failed to start tracker process
ERROR XGBoostSpark:643 - the job was aborted due to
java.lang.IllegalArgumentException: requirement failed: FAULT: Failed to start tracker
at scala.Predef$.require(Predef.scala:281)
at ml.dmlc.xgboost4j.scala.spark.XGBoost$.startTracker(XGBoost.scala:436)
at ml.dmlc.xgboost4j.scala.spark.XGBoost$.trainDistributed(XGBoost.scala:593)
at ml.dmlc.xgboost4j.scala.spark.XGBoostClassifier.train(XGBoostClassifier.scala:199)
at ml.dmlc.xgboost4j.scala.spark.XGBoostClassifier.train(XGBoostClassifier.scala:40)
at org.apache.spark.ml.Predictor.fit(Predictor.scala:151)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
at py4j.Gateway.invoke(Gateway.java:282)
at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
at py4j.commands.CallCommand.execute(CallCommand.java:79)
at py4j.GatewayConnection.run(GatewayConnection.java:238)
at java.lang.Thread.run(Thread.java:750)

[trivialfis]

Could you please share the specification of the environment where we can try to reproduce it?

[Ayyan-droid]

hi i am running it on docker with following versions
spark version 3.1
xgboost4j_2.12-1.6.1.jar
xgboost4j-spark_2.12-1.6.1.jar

[Ayyan-droid]

code works fine on local environment

[Ayyan-droid]

hii kindly help me on this issue i am completely stuck i am using a docker image with python version 3.7.11 spark 3.1

…

On Mon, Aug 22, 2022 at 8:35 PM Jiaming Yuan ***@***.***> wrote: Could you please share the specification of the environment where we can try to reproduce it? — Reply to this email directly, view it on GitHub <#8191 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A2UNYQHTPKEYWD7DTVI5P7DV2OM3RANCNFSM57GOPSEA> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[trivialfis]

Hi, we need more information on how to reproduce it. We run XGBoost jobs almost daily and haven't seen it.

[Ayyan-droid]

hi Dmlc/Xgboost i am using redhat/ubi8:8.4 linux for docker image and i am using code from this repo https://github.com/sllynn/spark-xgboost spark 3.1.1 python 3.7.11 and xgboost4j and xgboost4j spark 2.12-1.6.1 kindly help regards

[Ayyan-droid]

How can i specify tracker propertise

…

On Wed, Aug 31, 2022 at 8:59 AM Muhammad Bin Azeem < ***@***.***> wrote: hi Dmlc/Xgboost i am using redhat/ubi8:8.4 linux for docker image and i am using code from this repo https://github.com/sllynn/spark-xgboost spark 3.1.1 python 3.7.11 and xgboost4j and xgboost4j spark 2.12-1.6.1 kindly help regards

[trivialfis]

@Ayyan-droid Consider using the pyspark in xgboost with the nightly build.

Nightly build:
https://xgboost.readthedocs.io/en/latest/install.html#id1

Pyspark tutorial:
https://xgboost.readthedocs.io/en/latest/tutorials/spark_estimator.html

[wbo4958]

@Ayyan-droid if you're using the XGBoost jvm packages, you can specify the Tracker configuration by

Map("tracker_conf" -> TrackerConf(0l, "python", YOUR_IP, PYTHON_EXEC))

See

case class TrackerConf(workerConnectionTimeout: Long, trackerImpl: String,
  hostIp: String = "", pythonExec: String = "")