The blog, read report and code example for AGI/LLM related knowledge.
-
Updated
Jun 7, 2024 - Python
The blog, read report and code example for AGI/LLM related knowledge.
High-efficiency floating-point neural network inference operators for mobile, server, and Web
OnnxRT based Inference Optimization of Roberta model trained for Sentiment Analysis On Twitter Dataset
BladeDISC is an end-to-end DynamIc Shape Compiler project for machine learning workloads.
Faster inference YOLOv8: Optimize and export YOLOv8 models for faster inference using OpenVINO and Numpy 🔢
[WIP] A template for getting started writing code using GGML
MLP-Rank: A graph theoretical approach to structured pruning of deep neural networks based on weighted Page Rank centrality as introduced by the related thesis.
Learn the ins and outs of efficiently serving Large Language Models (LLMs). Dive into optimization techniques, including KV caching and Low Rank Adapters (LoRA), and gain hands-on experience with Predibase’s LoRAX framework inference server.
YOLOV8 - Object detection
A compilation of various ML and DL models and ways to optimize the their inferences.
Improving Natural Language Processing tasks using BERT-based models
cross-platform modular neural network inference library, small and efficient
The Tensor Algebra SuperOptimizer for Deep Learning
Batch Partitioning for Multi-PE Inference with TVM (2020)
Modified inference engine for quantized convolution using product quantization
Interface for TensorRT engines inference along with an example of YOLOv4 engine being used.
[MLSys 2021] IOS: Inter-Operator Scheduler for CNN Acceleration
🤖️ Optimized CUDA Kernels for Fast MobileNetV2 Inference
Batch estimation on Lie groups
A constrained expectation-maximization algorithm for feasible graph inference.
Add a description, image, and links to the inference-optimization topic page so that developers can more easily learn about it.
To associate your repository with the inference-optimization topic, visit your repo's landing page and select "manage topics."