CIMetricsAggregator

Tool for aggregating and caching metrics regarding Openstack CI testing results built in Python using Django 1.6.5

Dependencies (*nix)

MariaDB (or MySQL) server
PHP5
Django 1.6.5
nginx
mysql-python
ijson
Python 2.7
uwsgi

Dependencies (Windows)

WAMP Server
mod_wsgi (Look here: http://www.lfd.uci.edu/~gohlke/pythonlibs/#mod_wsgi)
Django 1.6.5
mysql-python (Get binary here: http://www.lfd.uci.edu/~gohlke/pythonlibs/#mysql-python)
ijson
Python 2.7
pyWin32

Deployment Instructions (*nix)

Step 1:
Install the dependencies (listed above) and create a database named 'Metrics'

Step 2:
Clone the github repo into /CIMetricsTool

Step 3:
You must enter the correct username/password into openstack-stats/settings.py for the MariaDB/MySQL database.

Step 4:
Generate the SQL structure using the following command:
python manage.py syncdb

Step 5:
Start both nginx, and uwsgi services using the configurations found in the repo. Note: nginx.conf needs to be copied to /etc/nginx/sites-enabled for the configuration to be loaded properly

Step 6:
Navigate to <your_server>/admin and at least add upstream Jenkins to the source model:
project: openstack/nova worker: Jenkins
And,
project: openstack/neutron worker: Jenkins
Step 7:
The service that actually queries Gerrit and saves the resultant data into the database is found in aggregatorService.py. This service must be started manually, or set to autorun on boot. You can use the following upstart config, or write your own config for your init daemon of choice.

description "CIMetrics Aggregator Service"
author "Gabriel Loewen <gabloe@microsoft.com>"

start on runlevel [234]
stop on runlevel [0156]

exec <SOURCE ROOT>/openstack_stats/aggregatorService.py
respawn

Deployment Instructions (Windows)

Step 1:
Install the dependencies (listed above) and create a database named 'Metrics'

Step 2:
Clone the github repo into C:/CIMetricsTool and change the static root location in settings.py to C:\CIMetricsTool\statsproj\openstack_stats\static

Step 3:
You must enter the correct username/password into openstack-stats/settings.py for the MySQL database.

Step 4:
Generate the SQL structure using the following command:
python manage.py syncdb

Step 5: Add the following to your httpd.conf file:

    WSGIScriptAlias / C:/CIMetricsTool/statsproj/openstack_stats/openstack_stats/wsgi.py
    WSGIPythonPath C:/CIMetricsTool/statsproj/openstack_stats
    <Directory C:/CIMetricsTool/statsproj/openstack_stats/openstack_stats>
    <Files wsgi.py>
    Require all granted
    </Files>
    </Directory>
    Alias /static/ C:/CIMetricsTool/statsproj/openstack_stats/static/
    <Directory C:/CIMetricsTool/statsproj/openstack_stats/static>
    Require all granted
    </Directory>

Step 6:
Navigate to <your_server>/admin and at least add upstream Jenkins to the source model:
project: openstack/nova worker: Jenkins
And,
project: openstack/neutron worker: Jenkins

Step 7:
Restart apache

Step 8:
The service that actually queries Gerrit and saves the resultant data into the database is found in aggregatorService.py. This service must be started manually, or set to autorun on boot.

Docker Instructions (Ubuntu)

A docker file has been created FROM ubuntu which can be used to set this up pretty easily. Once the docker container is started ... you presently need to 'exec' a script to start up the website and begin collecting data from the public stores. If you change the sources or reports, presently, you will need to stop the aggregator and restart it.

To Start the Container and see it up on port 80 of your docker host ...

docker pull msopenstack/ci_metrics_aggregator



docker run -p 80:80 -P -d --name cim msopenstack/ci_metrics_aggregator

docker exec cim /CIMetricsTool/bin/start-everything.sh