The Death and Life of Great Italian Cities: A Mobile Phone Data Perspective

This repository shows the code to apply the concept of "The Death and Life of Great Italian Cities: A Mobile Phone Data Perspective" into the Italian scenario. Through the code it is possible to reproduce some results, and to see how vitality can be described through the Jane Jacobs' conditions and, thus, the urban built environment.

See the paper and the slides for more details.

Note about the repository: this repository was created long time after the publication due to numerous people that asked more information about the code. We shared the code along with new script to easy the data processing. For this reason, OSM data is from 2019 (not as the paper).

Note about the data: We are not able to share the Foursquare and raw mobile phone data, thus these files could here be replaced with OSM data + other mobile phone data. The saved database does not contain the Foursquare data.

Overview

• data/ contains all the data the scripts and the models use.
• figures/ is the output directory of the images generated by the scripts.
• generated_files/ contains the files generated from the scripts.
• install/ contains some code to build the initial repository.
• load_data.bash script to load the data in the database. You need this only if you want to repeat all the data processing.
• step1_compute_features.ipynb this script loads the data from the database and computes the features. You need this only if you want to repeat all the data processing.
• step2_exploration.ipynb some plots for data exploration. You need this only if you want to repeat all the data processing.
• step3_Regression.ipynb this script gets the data from the CSV files in generated_files and outputs the results of the paper.

Please consider citing our paper if you use our model or code (see below for citation). We live thanks to this small action you can take!

Installation

We assume that you're using Python 3.6 with pip installed.

Once that's done you need to run the following inside the root directory to install the remaining dependencies:

pip3 install -r install/requirements.txt

This will install the following dependencies:

• numpy
• pandas
• scipy
• matplotlib
• seaborn
• scikit-learn
• SQLAlchemy
• psycopg
• geoalchemy2
• jupyter
• psycopg2
• shapely
• geoalchemy

Database installation

If you want to run the scripts step1_compute_features.ipynb and step2_exploration.ipynb, you have to download these files: Harvard Dataverse and then: Then, you have to place them in the right directories and run:

createdb WWW
gunzip < install/db_without_POIs.sql.gz | psql WWW

Whenever you are ready, and you placed all the files in the right directories, you can run configure the loader refresh_views.bash (variables on the top of the file), the run:

bash refresh_views.bash

This script will refresh the materialized views of the database. Then you can run the python scripts :)

NOTE: the data does not contain the POIs. If you want to load them through another datasource, check "Custom data". Moreover, pay attention to the step3, as it uses the POIs features by default.

Custom data

Further instructions

Disclaimer

This code has been published after the peer-reviewed publication (1 year after it), to publish the code for new developers and researchers. Thus, I am sorry for the small differences you can find, or for the lack of code to reproduce some images. I will refactor it in the future. Although I improved the original code A LOT with new software and scripts that have been released in this year, it has not been optimized for efficiency, but should be fast enough for most purposes. We do not give any guarantees that there are no bugs - use the code on your own responsibility!

License

This code is licensed under the MIT license.

Included datasets

The datasets are uploaded to this repository for convenience purposes only. They were not released by us and we do not claim any rights on them. If you want to use any of the datasets please consider citing the original authors. Sadly, we can't share the mobile phone dataset. However, there are similar dataset released in Open Data license.

Citation

@inproceedings{DeNadai:2016,
     author = {De Nadai, Marco and Staiano, Jacopo and Larcher, Roberto and Sebe, Nicu and Quercia, Daniele and Lepri, Bruno},
     title = {The Death and Life of Great Italian Cities: A Mobile Phone Data Perspective},
     booktitle = {Proceedings of the 25th International Conference on World Wide Web},
     series = {WWW '16},
     year = {2016},
     isbn = {978-1-4503-4143-1},
     location = {Montr\&\#233;al, Qu\&\#233;bec, Canada},
     pages = {413--423},
     numpages = {11},
     url = {https://doi.org/10.1145/2872427.2883084},
     doi = {10.1145/2872427.2883084},
     acmid = {2883084},
     publisher = {International World Wide Web Conferences Steering Committee},
     address = {Republic and Canton of Geneva, Switzerland},
     keywords = {cities, mobile phone data, open data, urban informatics},
}