Skip to content

Link Prediction is about predicting the future connections in a graph. In this project, Link Prediction is about predicting whether two authors will be collaborating for their future paper or not given the graph of authors who collaborated for atleast one paper together.

Notifications You must be signed in to change notification settings

sahith/Link-Prediction-for-Citation-Networks-using-Apache-Spark

Repository files navigation

Link-Prediction-for-Citation-Networks-using-Apache-Spark

Link Prediction is about predicting the future connections in a graph. In this project, Link Prediction is about predicting whether two authors will be collaborating for their future paper or not given the graph of authors who collaborated for atleast one paper together.

Implemented this project as a part of Big Data Management and Analytics course (CS 6350.002) at The University of Texas at Dallas.

Instructions to compile and run the code

  1. Create AWS EMR cluster with spark selected in advanced options. More at AWS EMR Instructions
  2. Upload the jar file of this project in s3 bucket.
  3. Add a step with the following configurations
    Step Type: spark application
    Name: Link Prediction
    Deploy Mode: Cluster
    Spark-submit options: --class "LinkPrediction"
    Application Location: s3 path of the uploaded jar file
    Arguments: // Specify the arguments in the given order
    s3://sxa180065/LinkPrediction1/com-dblp.ungraph.txt
    s3://{your bucket name}/CN
    s3://{your bucket name}/JC
    s3://{your bucket name}/PA
    s3://{your bucket name}/AA
    s3://{your bucket name}/RA
    s3://{your bucket name}/ND
    s3://{your bucket name}/Logging
    s3://{your bucket name}/Predictions
    Action on failure: Continue
  4. Output of link prediction according to each measure can be seen in the above specified paths
    Common Neighbors count at */CN
    Jaccard Coefficient at */JC
    Preferential Attachment at */PA
    Adamic Score at */AA
    Resource Allocation at */RA
    Neighborhood distance at */ND
    Log Statements to */Logging
    Predictions of the supervised algorithm at */Predictions

Dataset Link

Citation Dataset

Databricks Link

https://databricks-prod-cloudfront.cloud.databricks.com/public/4027ec902e239c93eaaa8714f173bcfc/517545003229769/2213049753017927/912825778976380/latest.html

About

Link Prediction is about predicting the future connections in a graph. In this project, Link Prediction is about predicting whether two authors will be collaborating for their future paper or not given the graph of authors who collaborated for atleast one paper together.

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages