Skip to content
/ graph-app-kit Public template

Go from graph data to a secure and interactive visual graph app in 15 minutes. Batteries-included self-hosting of graph data apps with Streamlit, Graphistry, RAPIDS, and more!

License

Notifications You must be signed in to change notification settings

graphistry/graph-app-kit

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Version CI Publish Docker Cloud Build Status ✔️ Linux ✔️ OS X ❌ Windows (#39)

Uptime Robot status Twitter Follow

Welcome to graph-app-kit

Turn your graph data into a secure and interactive visual graph app in 15 minutes!

Screenshot

Why

This open source effort puts together patterns the Graphistry team has reused across many graph projects as teams go from code-heavy Jupyter notebook experiments to deploying streamlined analyst tools. Whether building your first graph app, trying an idea, or wanting to check a reference, this project aims to simplify that process. It covers pieces like: Easy code editing and deployment, a project stucture ready for teams, built-in authentication, no need for custom JS/CSS at the start, batteries-included data + library dependencies, and fast loading & visualization of large graphs.

What

  • Minimal core: The barebones dashboard server. In provides a StreamLit docker-compose container with PyData ecosystem libraries and examples of visualizing data from various systems. Install it, plug in credentials to various web services like cloud databases and a free Graphistry Hub visualization account, and launch. It does not have GPU ETL and GPU AI libraries.

  • Full core: Initially for AWS, the full core bundles adds to the docker-compose system: Accounts, Jupyter notebooks for authoring, serves StreamLit dashboards with both public + private zones, and runs Graphistry/RAPIDS locally on the same server. Launch with on click via the Cloud Formation template.

  • Full core + DB: DB-specific variants are the same as minimal/full, and add simpler DB-specific quick launching/connecting.

Get started

Quick (Local code) - full GPU core + third-party connectors

Note: Base image includes Nvidia RAPIDS and AI dependencies so is quite large, see CPU alternative for a lightweight alternativve

Note: Use sudo for docker-compose commands if your configuration requires it and is giving permission error

# Minimal core
git clone https://github.com/graphistry/graph-app-kit.git
cd graph-app-kit/src/docker

# Enable docker buildkit
# ... or run docker-compose via provided alias script `./dc`
export DOCKER_BUILDKIT=1
export COMPOSE_DOCKER_CLI_BUILD=1

# Build
docker-compose build

# Optional: Edit src/docker/.env (API accounts), docker-compose.yml: Auth, ports, ...

# Launch
docker-compose up -d
docker-compose logs -f -t --tail=100

=> http://localhost:8501/

To add views and relaunch:

# Add dashboards @ src/python/views/<your_custom_view>/__init__.py

docker-compose up -d --force-recreate

Quick (Local code) - minimal CPU core + third-party connectors

Same commands as above, but use ./dc.cpu, which aliases docker-compose -f docker-compose.yml -f override/cpu.override.yml:

git clone https://github.com/graphistry/graph-app-kit.git
cd graph-app-kit/src/docker
./dc.cpu build
...
./dc.cpu up

Quick Launchers - minimal/full core

  1. Quick launch options:

Full: Launch Stack

  • Public + protected Streamlit dashboards, Jupyter notebooks + editing, Graphistry, RAPIDS
  • Login to web UI as admin / i-instanceid -> file uploader, notebooks, ...
  • Dashboards: /public/dash and /private/dash
  • More info

Admin:

# launch logs
tail -f /var/log/cloud-init-output.log -n 1000

# app logs
sudo docker ps
sudo docker logs -f -t --tail=1 MY_CONTAINER

# restart a graphistry container
cd graphistry && sudo docker-compose restart MY_CONTAINER

# restart caddy (Caddy 1 override)
cd graphistry && sudo docker-compose -f docker-compose.gak.graphistry.yml up -d caddy

# run streamlit
cd graph-app-kit/public/graph-app-kit && docker-compose -p pub run -d --name streamlit-pub streamlit
cd graph-app-kit/private/graph-app-kit && docker-compose -p priv run -d --name streamlit-priv streamlit

Minimal: Open Streamlit, ssh to connect/add free Graphistry Hub username/pass:

Database-specific: Amazon Neptune, TigerGraph

  1. Add views

  2. Main configurations and extensions: Database connectors, authentication, notebook-based editing, and more

Native (Experimental)

Install dependencies, pick location of views folder, and run:

cd src/python
pip3 install -r requirements-system.txt
pip3 install -r requirements-app.txt
VIEW_PATH="`pwd`/views" streamlit run entrypoint.py 

The pieces

Core

  • Prebuilt Python project structure ready for prototyping

  • Streamlit quick self-serve dashboarding

  • Graphistry point-and-click GPU-accelerated visual graph analytics

  • Data frames: Data wrangling via Pandasand Apache Arrow, including handling formats such as CSV, XLS, JSON, Parquet, and more

  • Standard Docker and docker-compose cross-platform deployment

GPU acceleration (optional) - Full

For non-minimal installs, if GPUs are present, graph-app-kit leverages GPU cloud acceleration:

  • GPU Analytics: RAPIDS and CUDA already setup for use if run with an Nvidia docker runtime - cudf GPU dataframes, BlazingSQL GPU SQL, cuGraph GPU graph algorithms, cuML libraries, and more

  • GPU Visualization: Connect to an external Graphistry server or, faster, run on the same GPU server

Prebuilt integrations & recipes

graph-app-kit works well with the Python data ecosystem (pandas, cudf, PySpark, SQL, ...) and we're growing the set of builtins and recipes:

Contribute

We welcome all sorts of help!

  • Deployment: Docker, cloud runners, ...
  • Dependencies: Common graph packages
  • Connectors: Examples for common databases and how to get a lot of data out
  • Demos!

See DEVELOP.md for more contributor information

About

Go from graph data to a secure and interactive visual graph app in 15 minutes. Batteries-included self-hosting of graph data apps with Streamlit, Graphistry, RAPIDS, and more!

Resources

License

Stars

Watchers

Forks

Languages