🧪 Simple data science experimentation & tracking with jupyter, papermill, and mlflow.

Open Source Feature Flagging and A/B Testing Platform

GO Feature Flag is a simple, complete and lightweight self-hosted feature flag solution 100% Open Source. 🎛️

Feature flags, experiments, and remote config management with GitOps

This Chrome Extension automatically performs SRM checks and flags potential data quality issues on supported experimentation platforms.

A lightweight Franklin plugin for experimentation and segmentation.

A feature flags service written in .NET

Create your own study by cloning and editing configs; or check out the code behind the study components.

Shadow is a discrete-event network simulator that directly executes real application code, enabling you to simulate distributed systems with thousands of network-connected processes in realistic and scalable private network experiments using your laptop, desktop, or server running Linux.

LLMOps with Prompt Flow is a "LLMOps template and guidance" to help you build LLM-infused apps using Prompt Flow. It offers a range of features including Centralized Code Hosting, Lifecycle Management, Variant and Hyperparameter Experimentation, A/B Deployment, reporting for all runs and experiments and so on.

Official Omnitool repository

Auditory experiment framework with a focus on rapid experiment design

A “build-your-own” Feature Flagging/Toggling/Experimentation/etc system!

Apache Spark based framework for analysis A/B experiments

moai is a PyTorch-based AI Model Development Kit (MDK) created to improve data-driven model workflows, design and reproducibility.

Unleash SDK for Next.js

Open Source version of SigOpt API, performing hyperparameter optimization and visualization

UpTrain is an open-source unified platform to evaluate and improve Generative AI applications. We provide grades for 20+ preconfigured checks (covering language, code, embedding use-cases), perform root cause analysis on failure cases and give insights on how to resolve them.

Implementation of metaheuristic optimization methods in Python for scientific, industrial, and educational scenarios. Experiments can be executed in parallel or in a distributed fashion. Experimental results can be evaluated in various ways, including diagrams, tables, and export to Excel.

Library for multi-armed bandit selection strategies, including efficient deterministic implementations of Thompson sampling and epsilon-greedy.