RAFT and Berkeley Function Calling Leaderboard Updates

😍 v0.2 release 🚀

Highlights

🎯 Berkeley Function Calling Leaderboard (BFCL): How do models stack up for function calling?

• Now includes latency and cost
• More open-source and closed-source models
• Bug fixes in dataset.

RAFT: Fine-tuning technique to improve LLMs for in-domain RAG!

What's Changed

• Adding APIs of 9 Google Service to API Zoo by @meenakshi-mittal in #204
• Github Actions to Maintain API Zoo Index by @ramanv0 in #188
• Adding Zoom API to API Zoo by @meenakshi-mittal in #221
• API Zoo Index Github Actions Fix by @ramanv0 in #261
• Added Google Forms API by @elva01 in #185
• RAFT + readme + small sample dataset by @kaiwen129 in #218
• Sample data for RAFT by @ShishirPatil in #264
• Docusign Additions by @dangeo773 in #194
• [Bug Fix] Fix Executable Exact Match Condition Did not Meet by @Fanjia-Yan in #251
• [Bug Fix] Fix Error in Parallel Function Possible Answer by @Fanjia-Yan in #252
• [Bug Fix] Restrict AST checker on Boolean Variable by @Fanjia-Yan in #256
• Adding 7 Oracle APIs to API Zoo by @meenakshi-mittal in #205
• Adding Datadog API to API Zoo by @meenakshi-mittal in #206
• Added Notion APIs (Block, Page, and Database) to APIZoo by @jennifer818 in #195
• removed testing code by @kaiwen129 in #281
• feat: more type annotations for the functions by @UponTheSky in #283
• [Fix] java, javascript parsers in openfunctions-v2 by @CharlieJCJ in #284
• Leaderboard Update April 1 by @HuanzhiMao in #299
• Remove Large File from ./inference by @CharlieJCJ in #297
• Typo in raft.py by @danielfleischer in #311
• Leaderboard April 3 release by @HuanzhiMao in #309
• Support OSS Evaluation for Leaderboard by @HuanzhiMao in #318
• Update README.md by @HuanzhiMao in #320
• Fix typos by @viniciuslazzari in #323
• Correction in BFCL README instruction, fixed path in instructions by @CharlieJCJ in #329

New Contributors

• @elva01 made their first contribution in #185
• @kaiwen129 made their first contribution in #218
• @jennifer818 made their first contribution in #195
• @UponTheSky made their first contribution in #283
• @danielfleischer made their first contribution in #311

Full Changelog: v0.1...v0.2

Gorilla v0.1: OpenFunctions-v2, Berkeley Function Calling Leaderboard, and more.

😍 v0.1 release 🚀

Highlights

• 🎯 Berkeley Function Calling Leaderboard (BFCL): How do models stack up for function calling? Evaluation code for the Berkeley Function Calling Leaderboard.
• 🏆 Gorilla OpenFunctions v2: Inference examples for OpenFunctions-v2 - SoTA open-source LLM for function calling. On-par with GPT-4 🙌 Supports more languages 👌.
• API Zoo Index: An accessible collection of API documentation for humans to search through, and for LLMs to use as tools 👀

We are excited about our long due v0.1 release! Here's more:

What's Changed

• Adding BM25 and GPT retrievers by @ShishirPatil in #61
• update(anthropic): #63 to (0.3.x) by @AmirAflak in #64
• Add inference support for Macbook silicon chip by @benjaminhuo in #76
• Update README.md by @eltociear in #80
• PR for Gradio WebUI Feature ([feature] Gradio webui - #102) by @TanmayDoesAI in #105
• Update README.md by @abhi-databricks in #109
• Adds wandb to eval files by @morganmcg1 in #114
• Fix use_wandb in ast eval, responses file deletion, wandb artifacts renaming by @morganmcg1 in #115
• sentence optimization in docstring and examples by @rajveer43 in #117
• Gorilla OpenFunctions by @ShishirPatil in #142
• Example on running it locally with Hugging Face 🤗 Transformers by @Danielskry in #148
• Added Gmail api to api zoo by @saikolasani in #163
• Add Google Maps API (python client) by @felixzhu555 in #164
• Add support for the OpenWeatherMap API by @aryanvichare in #159
• Stripe Additions by @dangeo773 in #169
• Added Kubernetes Pod API and Pod Template API by @saikolasani in #170
• Quantized Gorilla by @CharlieJCJ in #160
• Add a guide on how to self-host the OpenFunctions model by @ramanv0 in #157
• Private Inference using Gorilla hosted endpoint on Replicate by @ramanv0 in #162
• added yfinance api to api zoo by @raywanb in #161
• Gorilla OpenFunctions run locally in Google Colab by @meenakshi-mittal in #166
• Fixed issue with Kubernetes Pod/Pod Template filename by @saikolasani in #198
• Create openfunctions-v2 issue template by @ShishirPatil in #203
• Add support for the ServiceNow REST API by @aryanvichare in #176
• Berkeley Function Calling Leaderboard evaluation scripts and OpenFunctions v2 inference by @ShishirPatil in #215
• [Berkeley-Function-Calling-Leaderboard] Refactor leaderboard result generation and checking by @Fanjia-Yan in #223
• Update openfunctions-v2 chatting format in README.md by @tianjunz in #239
• Update BFCL README.md by @CharlieJCJ in #241
• Local Inference script for openfunctions v2 by @ShishirPatil in #242
• [Update Gemini-1.0-Pro result checker] by @Fanjia-Yan in #245
• Update project roadmap and repository structure by @ShishirPatil in #257

New Contributors

• @AmirAflak made their first contribution in #64
• @benjaminhuo made their first contribution in #76
• @TanmayDoesAI made their first contribution in #105
• @abhi-databricks made their first contribution in #109
• @morganmcg1 made their first contribution in #114
• @rajveer43 made their first contribution in #117
• @Danielskry made their first contribution in #148
• @saikolasani made their first contribution in #163
• @felixzhu555 made their first contribution in #164
• @aryanvichare made their first contribution in #159
• @dangeo773 made their first contribution in #169
• @raywanb made their first contribution in #161
• @meenakshi-mittal made their first contribution in #166

Full Changelog: v0.0.1...v0.1

Gorilla release v0.0.1

🦍 Gorilla: An API store for LLMs 🚀

🚀 After 50,000 user requests through our hosted APIs, we are happy to tear the first release for Gorilla 💪

🤩 In this release:

💻 gorilla-cli, LLMs for your CLI!
🟢 Commercially usable, Apache 2.0 licensed Gorilla models
🚀 CLI interface to chat with Gorilla!
🚀 Torch Hub and TensorFlow Hub Models!
🚀 The first Gorilla model! Colab or 🤗!
🔥 APIZoo contribution guide for community API contributions!
🔥 APIBench dataset and the evaluation code of Gorilla!