sp19-281-spartans

Team Project - Device Monitoring System

Team Name

Spartans

Team Members

Project Name

Device Monitoring System

Project Description

Device Monitoring System is a SAAS app where one can monitor the usage of the 1000's of desktops in an organization. The UI shows the the following 3 statistics of a desktop(producer):

Device Temperature
Device CPU Usage
Device RAM Memory

To keep track of so much amount of data which is being pushed regularly by a producer, we need a reliable and stable architecture.

Architecture

Design

Bottom in the architecture is our Producers which are the desktops. These keep on producing the CPU Stats very frequently. Producer Code
Then we have load balanced RabbitMQ HA clusters of mirrored queues. These queues serve as Message queues in which the producers produce the data and consumers consume the data. These are mirrored queues(where one is master and the other is a mirrored queue). Both the queues will have the same data, hence forming a Highly Available cluster. RabbitMQ Cluster setup
Consuming the messages is an auto scaled group of consumers, which consume the data from the RabbitMQ cluster and push it to the NoSQL Riak TS database cluster. Consumer code
The auto scaling event of the consumers should be triggered when there is an overwhelming incoming of messages from the queue or any consumer goes down. The AS event is triggered when the RabbitMQ queue length exceeds a certain threshold. To do this, we created an alarm based on a custom metric of queue length on the Cloud Watch console. The RabbitMQ hosted instances keeps on pushing the queue length data to the CloudWatch. Whenever the queue length exceeds a threshold, alarm is triggered and the consumers are scaled out. CloudWatch Custom Alerts and Consumer ASG
The Riak TS is a timeseries NoSQL database which is a perfect match for storing Timeseries data like sensor data. Plus its query language is like SQL and we created 3 tables each for the device stats. Riak TS setup
In front of the Riak TS, we hosted our microservices which read the individual stats from the RiakTS. There is another microservice for maintaining all the producers on MongoDB.

Microservice for CPU Usage stats

Microservice for Memory stats

Microservice for temperature stats

Microservice for maintaining the producers
The SAAS app front end is deployed on Heroku which queries the above microservices to fetch the different statistics. Front End Code
SAAS App on Heroku

https://spartan-up-monitor.herokuapp.com/