You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I am trying to use sparkxgb package in a yarn session, the problem is that when I try to create a spark session it fails with the following message:
spark <- spark_connect(master = "yarn-client",version = "2.3.0",app_name = "test", config=config) [main] INFO com.unraveldata.agent.ResourceCollector - Unravel Sensor 4.5.0.8_dev+origin_fix_hive_on_spark_4.5.0.8+103/1.3.11.3 initializing. Ivy Default Cache set to: /home/cdsw/.ivy2/cache The jars for the packages stored in: /home/cdsw/.ivy2/jars :: loading settings :: url = jar:file:/opt/cloudera/parcels/SPARK2-2.3.0.cloudera3-1.cdh5.13.3.p0.458809/lib/spark2/jars/ivy-2.4.0.jar!/org/apache/ivy/core/settings/ivysettings.xml ml.dmlc#xgboost4j-spark added as a dependency :: resolving dependencies :: org.apache.spark#spark-submit-parent-0e80d9c5-936d-4e4a-b31b-6b950c29d1f9;1.0 confs: [default]
`Error in force(code) :
Failed while connecting to sparklyr to port (8880) for sessionid (96811): Gateway in localhost:8880 did not respond.
Path: /opt/cloudera/parcels/SPARK2-2.3.0.cloudera3-1.cdh5.13.3.p0.458809/lib/spark2/bin/spark-submit
Parameters: --class, sparklyr.Shell, --jars, '/home/cdsw/R/sparkxgb/java/sparkxgb-2.3-2.11.jar', --packages, 'ml.dmlc:xgboost4j-spark:0.81', '/home/cdsw/R/sparklyr/java/sparklyr-2.3-2.11.jar', 8880, 96811
Log: /tmp/RtmpesTPnW/file13d1275ad76_spark.log
---- Output Log ----
---- Error Log ----`
The text was updated successfully, but these errors were encountered:
Hello,
I am trying to use sparkxgb package in a yarn session, the problem is that when I try to create a spark session it fails with the following message:
spark <- spark_connect(master = "yarn-client",version = "2.3.0",app_name = "test", config=config) [main] INFO com.unraveldata.agent.ResourceCollector - Unravel Sensor 4.5.0.8_dev+origin_fix_hive_on_spark_4.5.0.8+103/1.3.11.3 initializing. Ivy Default Cache set to: /home/cdsw/.ivy2/cache The jars for the packages stored in: /home/cdsw/.ivy2/jars :: loading settings :: url = jar:file:/opt/cloudera/parcels/SPARK2-2.3.0.cloudera3-1.cdh5.13.3.p0.458809/lib/spark2/jars/ivy-2.4.0.jar!/org/apache/ivy/core/settings/ivysettings.xml ml.dmlc#xgboost4j-spark added as a dependency :: resolving dependencies :: org.apache.spark#spark-submit-parent-0e80d9c5-936d-4e4a-b31b-6b950c29d1f9;1.0 confs: [default]
`Error in force(code) :
Failed while connecting to sparklyr to port (8880) for sessionid (96811): Gateway in localhost:8880 did not respond.
Path: /opt/cloudera/parcels/SPARK2-2.3.0.cloudera3-1.cdh5.13.3.p0.458809/lib/spark2/bin/spark-submit
Parameters: --class, sparklyr.Shell, --jars, '/home/cdsw/R/sparkxgb/java/sparkxgb-2.3-2.11.jar', --packages, 'ml.dmlc:xgboost4j-spark:0.81', '/home/cdsw/R/sparklyr/java/sparklyr-2.3-2.11.jar', 8880, 96811
Log: /tmp/RtmpesTPnW/file13d1275ad76_spark.log
---- Output Log ----
---- Error Log ----`
The text was updated successfully, but these errors were encountered: