#

kv-cache

Here are 11 public repositories matching this topic...

HDT3213 / godis

A Golang implemented Redis Server and Cluster. Go 语言实现的 Redis 服务器和分布式集群

go redis golang cluster redis-server redis-cluster godis kv-cache

Updated May 6, 2024
Go

FMInference / H2O

[NeurIPS'23] H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models.

sparsity high-throughput heavy-hitters kv-cache gpt-3 large-language-models

Updated Apr 17, 2024
Python

kddubey / cappr

Completion After Prompt Probability. Make your LLM make a choice

text-classification probability zero-shot huggingface kv-cache prompt-engineering llamacpp llm-inference

Updated Mar 5, 2024
Python

DRSY / EasyKV

Easy control for Key-Value Constrained Generative LLM Inference(https://arxiv.org/abs/2402.06262)

cache-management kv-cache llm cache-eviction

Updated Feb 13, 2024
Python

hkproj / pytorch-llama-notes

Notes about LLaMA 2 model

study-notes rmsprop attention-is-all-you-need kv-cache rotary-position-encoding llama2

Updated Aug 30, 2023
Python

aju22 / LLaMA2

This repository contains an implementation of the LLaMA 2 (Large Language Model Meta AI) model, a Generative Pretrained Transformer (GPT) variant. The implementation focuses on the model architecture and the inference process. The code is restructured and heavily commented to facilitate easy understanding of the key parts of the architecture.

natural-language-processing transformer attention llama gpt rope kv-cache llm llama2 rms-norm

Updated Oct 1, 2023
Python

mehdihosseinimoghadam / AVA-Mistral-7B

Fine-Tuned Mistral 7B Persian Large Language Model LLM / Persian Mistral 7B

nlp deep-learning ava mistral kv-cache large-language-models llm mistral-7b ava-mistral ava-mistral-7b persian-mistral-7b persian-mistral

Updated Apr 2, 2024
Jupyter Notebook

glisses / Efficient-Effective-KV-Cache-Replacement-Policy-for-LLMs

SCAC strategy for efficient and effective KV cache eviction in LLMs

Updated Apr 24, 2024
Python

phkhanhtrinh23 / milliGPT

This a minimal implementation of a GPT model but it has some advanced features such as temperature/ top-k/ top-p sampling, and KV Cache.

generative-model gpt sampling-methods kv-cache

Updated Oct 13, 2023
Python

reshalfahsi / image-captioning-mobilenet-llama3

Image Captioning With MobileNet-LLaMA 3

nlp cnn pytorch transformer image-captioning image-text flickr8k-dataset mobilenetv3 pytorch-lightning kv-cache rotary-position-embedding grouped-query-attention rms-norm llama3

Updated May 5, 2024
Jupyter Notebook

lamaparbat / EXPRESS_REDIS_CACHING_RATE_LIMIT

EXPRESS REST API CACHING + RATE LIMITING + KV-STORE

redis express rate-limiting restapi ioredis kv-cache redis-stack

Updated Apr 16, 2024
JavaScript

Improve this page

Add a description, image, and links to the kv-cache topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the kv-cache topic, visit your repo's landing page and select "manage topics."