Skip to content

A demo to show how a Python script can share a SparkContext in a Scala written Spark driver.

Notifications You must be signed in to change notification settings

jason-ni/gemini-spark

Repository files navigation

This is a simple demo to show how to reuse the SparkContext of a Scala Spark driver in a remote Python script.

Run following commands to see the demo:

  1. sbt assembly
  2. Run "spark-submit target/scala-2.11/GeminiSpark-assembly-1.0.jar" in another console.
  3. ./runpy.sh test.py

About

A demo to show how a Python script can share a SparkContext in a Scala written Spark driver.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published