Aim of this project

Presently, a lot of massive and groundbreaking initiatives are being undertaken to analyse microbiome data from diverse environments around the world in order to discover interesting secrets present in the world of microbiome. Microbiome Diversity Inspector can be used to highlight about the diversity of microbiome community in various materials found over different regions. The inclusion of other new algorithms in this platform can lead to interesting discoveries which may prove very beneficial to mankind. For more details, visit this link. Visit this link for our project presentation.

What is Microbiome Diversity Inspector?

Microbiome Diversity Inspector is a one-stop scalable platform for visual analysis of large microbiome data running completely from the comfort of your browser(although a desktop-standalone version is also available!). The server application runs on all major OS like - Windows, macOS and Linux systems and the client runs in every modern browser. This project is being done as a part of "Major Project" course in 7th Semester and 8th Semester under Prof. Malay Bhattacharyya.

Who is this for?

Although tools/servers like MG-Rast or One Codex do an extremely impressive job of analyzing the microbiome data, they do not provide some crucial out-of-the-box features like - performing real-time analysis of entropy of huge microbiome data, conversion between various standardised file formats used in metagenomic analysis, e.g.- converting big FASTQ files to their FASTA counterpart and vice versa, computing alpha-diversity of samples. Often, it is seen that these tools become very limited in use by restricting the users to upload up to a certain limit of data only. This platform is for all microbiome-enthusiasts who want the convenience and scalability of the aforementioned services at real-time speed at one place without any such restrictions!

How to install the platform?

The two-step installation process of the platform is a breeze. Since the server runs on Node.js runtime environment, the first step to be taken in order to install the platform is to download Node.js and npm.

Step 1: Installing Node.js and npm

Installation on Windows

Node.js and npm can be installed from a download link. Go to the Node installation page, and download the Node installer and then follow the setup wizard to install Node.js on your computer.

Installation on macOS or Linux

In order to install everything on a Mac or Linux, we’ll be running commands in Terminal. Also if you're using Linux, then you might have to prefix sudo in all/some of the commands mentioned below onwards in order to run them with elevated privileges.

We’re going to use Node Version Manager(nvm) to install Node.js and npm. Type the below command -
curl -o- https://raw.githubusercontent.com/creationix/nvm/v0.33.1/install.sh | bash

Open the ~/.bash_profile file, and make sure source ~/.bashrc is written in there somewhere. Restart the terminal. Then install Node.js by typing - nvm install node and then run Node by typing - nvm use node.

We are now all set to install the actual platform. The platform can be installed and used as a web application as well as a standalone desktop application. The installation process for both of them is almost same but to install the desktop-standalone application, few additional instructions should be followed.

Step 2: Common instructions for installing the platform

Download the repository - Microbiome-Diversity-Inspector. This would download a Zipped folder - Microbiome-Diversity-Inspector-master.zip. Unzip and extract this Zipped folder to get the unzipped directory - Microbiome-Diversity-Inspector-master. Using the shell, navigate inside this unzipped directory (Use - $ cd /path/to/Microbiome-Diversity-Inspector-master, where /path/to/ is the path of Microbiome-Diversity-Inspector-master). The repository can also be downloaded by cloning the git repository into the directory where you want the platform to be installed (/path/to/Microbiome-Diversity-Inspector-master). Then type - npm install in the shell which will automatically install all the dependencies of the platform, or otherwise, manually install each and every dependency mentioned in package.json(Make sure that you are not connected to a proxy server while following these steps. If you must be connected to a proxy server, then follow the instructions mentioned here). Bring up the server by typing - $ npm run start* in the shell (Command Prompt in Windows and Terminal in Linux and macOS).

* This command is used for starting the application/platform. This command can also be used to re-run the application, if the application crashes or does not behave properly(you might need to abort the already running application first by typing the Ctrl+C command). If you are facing an issue after typing this command, more specifically - if you are getting an EADDRINUSE error in your command line interface, then there is a high chance that there is another process which is using the port :8080. Make sure that no other program is conflicting Node such as IIS, SQL, Skype, Zonealarm, firewall/antivirus, NOD32, Eset, any web related program including Remote Desktop, Teamviewer etc and that the port :8080 is available before bringing up the server.

We now lay out instructions separately for installing the web version and the desktop-standalone version below -

Installing the Web Version

After following the above instructions, just open - http://localhost:8080/ in a browser.

Installing the Desktop-Standalone Version

Following the aforementioned common instructions, install Nativefier(essentially a wrapper over Electron) by typing - $ npm install nativefier -g and then type - $ nativefier --name "Microbiome-Diversity-Inspector" --icon "src/client/assets/images/favicon.ico" "http://localhost:8080" in the shell. This will install the desktop application inside the directory from where the aforementioned commands where executed (The name of the directory is generally of the format- Microbiome-Diversity-Inspector-FOOBAR, where FOOBAR is generally the OS configuration details). Then launch the desktop application named - Microbiome-Diversity-Inspector present inside the newly installed directory.

Note- If facing any difficulty in installing the platform, then raise an issue here and we will try to resolve it as soon as possible. You can also directly contact - gbelwariar@gmail.com or mkhan31995@gmail.com for assistance.

Notes -

• The platform provides various file-processing options like - converting a FASTQ file to a FASTA file and vice versa, analyze entropy of a file containing metagenomic data in real-time. Since such files could be of huge size, thus there is always a possibilty of the server being choked to death. Hence in order to process the files in real-time, the platform requires all such input files to be present inside the directory - Microbiome-Diversity-Inspector. Keeping this in mind, the platform automatically downloads all the output files inside the directory - Microbiome-Diversity-Inspector, so that the user need not bother about migrating the output files from a different directory to the Microbiome-Diversity-Inspector directory for further analysis of those files.

• Some features of this platform, like- computation of alpha-diversity and downloading the MetaSub data requires an active internet connection for proper functioning.

Technologies (Languages/Frameworks/Packages) Used -

• JavaScript - The core programming language used to develop the platform.
• AngularJS (1.x) - The framework on which the front-end is built on.
• Node.js - The runtime on which the platform's back-end server runs on.
• Express - The framework used to develop the back-end.
• Mocha - The test framework to integration test server-side code.
• Jasmine - The test framework to unit test client-side code.
• Karma - The test runner used for running client-side tests.
• SuperTest - The library used with Mocha to test the back-end server.
• nodemon - For development purpose, used as an "auto-watch-reload" utility for back-end server.
• Browsersync - For development purpose, used as an "auto-watch-reload" utility for front-end.
• ESLint - For development purpose, used as a linter to follow best programming practices.
• Nativefier - Built on the top of Electron, used for deploying the desktop-standalone application.
• CanvasJS - The framework that renders the result of the real-time entropy analysis on a dynamic chart.
• HTML - The markup language to create the UI of the platform.
• CSS - The style sheet language to style the UI of the platform.
• and a bunch of npm packages...

External Services Used -

• One Codex RESTful APIs - Powers the computation of alpha-diversity.

Screenshots Of The Platform -

1. Utility for downloading the MetaSUB data :

2. Showing the real-time entropy analysis :

3. Conversion of a FASTQ file to FASTA in action :

4. Conversion of a FASTA file to FASTQ in action :

5. Computation of alpha-diversity and NormAlpha (Normalised alpha-diversity) :

Observations of our analysis for NormAlpha -

s : Denotes the number of species in our sample
x-axis : Represents the instances of the combinations of probabilities of all the 's' species with a step value of 0.1
y-axis : NormAlpha(Red) and Alpha-diversity(Blue)

Known Issues/Limitations -

• The server will not send AJAX requests to One Codex's exposed URL endpoints if there is a proxy connection enabled in the browser. Since the desktop application is essentially an Electron wrapper over the web application, hence this limitation also holds for the desktop application as well.

Contributors