Spark DataFrames Introduction

This is an introduction for Apache Spark DataFrames. You can practically learn Apache Spark DataFrame APIs with the real data sets from Github Archive.

Apache Spark DataFrames are very useful data analytics tool for data scientists. It allow us to analize data more easily and more fast.

Reynold Xin, Michael Armbrust and Davies Liu wrote wrote the great blog article . And Reynold Xin gave a nice presentation Spark DataFrames for Large-Scale Data Science in Bay Area Spark Meetup at Feb, 17, 2015.

Set up Environments for This Introduction

You can select an environment to run Spark on your local machine or Amazon EC2.

• Set up Local Machine
• Set up Spark Cluster on EC2

Spark DataFrames Introduction

Spark DataFrame Introduction

TODO

I will try to add the introduction for Python and R. Because Spark provides the APIs for Python and R (via SparkR), we should add the explanations for not only Scala but also Python and R.