OpenShift AI / ML GitOps Catalog

This project is a catalog of configurations used to provision infrastructure, on OpenShift, that supports machine learning (ML) and artificial intelligence (AI) workloads.

The intention of this repository is to help support practical use of OpenShift for AI / ML workloads.

Please look at the GitOps Catalog if you only need to automate an operator install.

In this repo, look at various kustomized configs and argo apps for ideas.

This repo is currently subject to frequent, breaking changes!

Prerequisites

• OpenShift 4.8+
  • role: cluster-admin - for all demo or cluster configs
  • role: self-provisioner - for namespaced components

Red Hat Demo Platform Options (Tested)

• AWS with OpenShift Open Environment
  • 1 x Control Plane - m6a.2xlarge
  • 0 x Workers - m6a.2xlarge
• One Node OpenShift
  • 1 x Control Plane - m6a.2xlarge
• MLOps Demo: Data Science & Edge Practice

Tools

The following cli tools are required:

• bash, git
• oc - Download mac, linux, windows
• kubectl (optional) - Included in oc bundle
• kustomize (optional) - Download mac, linux

NOTE: bash, git, and oc are available in the OpenShift Web Terminal

The following are used to encrypt secrets and are optional:

• age - Info
• sops - Info

Bootstrapping a Cluster

1. Verify you are logged into your cluster using oc.
2. Clone this repository

To a local environment

oc whoami
git clone < repo url >

Use an OpenShift Web Terminal

NOTE: Due to a bug you may need to install the web terminal operator at version 1.7 and upgrade in order for it to work correctly.

YOLO_URL=https://raw.githubusercontent.com/redhat-na-ssa/demo-ai-gitops-catalog/main/scripts/library/term.sh
. <(curl -s "${YOLO_URL}")
term_init

# make custom web terminal persistent
apply_firmly bootstrap/web-terminal

NOTE: open a new terminal to activate new configuration

Cluster Quick Start for OpenShift

Basic cluster config

# load functions
. scripts/functions.sh

# setup an enhanced web terminal on a default cluster
# alt cmd: until oc apply -k bootstrap/web-terminal; do : ; done
apply_firmly bootstrap/web-terminal

# setup a default cluster w/o argocd managing it
apply_firmly clusters/default

Setup a demo

# setup a dev spaces demo /w gpu
apply_firmly demos/devspaces-nvidia-gpu-autoscale

# setup a rhoai demo /w gpu
apply_firmly demos/rhoai-nvidia-gpu-autoscale

Setup an ArgoCD managed cluster

# setup a basic instance of argocd managing a default cluster
apply_firmly bootstrap/argo-managed

Many common operational tasks are provided in the scripts library. You can run individual functions in a bash shell:

These functions are available in an enhanced web terminal (see above)

# load functions
. scripts/functions.sh

get_functions

The bootstrap.sh script will:

• Install the OpenShift GitOps Operator
• Create an ArgoCD instance in the openshift-gitops namespace

Workshops

This is currently under development

# load functions
. scripts/wip/workshop_functions.sh

# setup workshop with 25 users
workshop_setup 25

Additional Configurations

Sandbox Namespace

The sandbox namespace is useable by all authenticated users. All objects in the sandbox are cleaned out weekly.

Contributions

Please run the following before submitting a PR / commit

scripts/lint.sh

Additional Info

• Local Docs
• Notes Dump

External Links

• GitOps Catalog
• AI Pilot Gitops
• ArgoCD - Example
• ArgoCD - Patterns