[bug] Hosted Gorilla: <Issue>

[Marcseb]

Hi, I am trying to use Gorilla with the snippets below (extrapolated from the example in Gorilla colab, after migration for Openai Version: 1.16.2):

Import Chat completion template and set-up variables

import os
import openai
from openai import OpenAI
import urllib.parse

client = OpenAI()
openai.api_key = "EMPTY" # Key is ignored and does not matter
openai.api_base = "http://zanino.millennium.berkeley.edu:8000/v1"

Report issues

def raise_issue(e, model, prompt):
issue_title = urllib.parse.quote("[bug] Hosted Gorilla: ")
issue_body = urllib.parse.quote(f"Exception: {e}\nFailed model: {model}, for prompt: {prompt}")
issue_url = f"https://github.com/ShishirPatil/gorilla/issues/new?assignees=&labels=hosted-gorilla&projects=&template=hosted-gorilla-.md&title={issue_title}&body={issue_body}"
print(f"An exception has occurred: {e} \nPlease raise an issue here: {issue_url}")

Query Gorilla server

def get_gorilla_response(prompt="I would like to translate from English to French.", model="gorilla-7b-hf-v1"):
try:
completion = client.chat.completions.create(
model=model,
messages=[{"role": "user", "content": prompt}]
)
return completion.choices[0].message.content
except Exception as e:
raise_issue(e, model, prompt)

prompt = "I would like to translate 'I feel very good today.' from English to French."
print(get_gorilla_response(prompt=prompt, model="gorilla-7b-hf-v1"))

I am getting the following exception: Error code: 404 - {'error': {'message': 'The model gorilla-7b-hf-v1 does not exist or you do not have access to it.', 'type': 'invalid_request_error', 'param': None, 'code': 'model_not_found'}}
Failed model: gorilla-7b-hf-v1, for prompt: I would like to translate 'I feel very good today.' from English to French

Could you please help me on this issue? Thanks in advance.

[ShishirPatil]

Hey @Marcseb I just tried it out, and looks like I can hit our endpoints for the given model. Can you try with openai==0.28.xx ?

Here's a colab notebook if that helps: https://colab.research.google.com/drive/11HJWR3ylG1HSE2v78W1gRK-dKkSA0pHe?usp=sharing

[Marcseb]

Thank you very much Shishir for your prompt response. I was indeed anticipating too much the migration to a newer openai version. Thanks also for the colab with the example of integration with Langchain.

…

________________________________ Da: Shishir Patil ***@***.***> Inviato: venerdì 5 aprile 2024 08:31 A: ShishirPatil/gorilla ***@***.***> Cc: Marcseb ***@***.***>; Mention ***@***.***> Oggetto: Re: [ShishirPatil/gorilla] [bug] Hosted Gorilla: <Issue> (Issue #312) Hey @Marcseb<https://github.com/Marcseb> I just tried it out, and looks like I can hit our endpoints for the given model. Can you try with openai==0.28.xx ? Here's a colab notebook if that helps: https://colab.research.google.com/drive/11HJWR3ylG1HSE2v78W1gRK-dKkSA0pHe?usp=sharing — Reply to this email directly, view it on GitHub<#312 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/APZ4ELD6XCXMJV3NJKSNZODY3ZOODAVCNFSM6AAAAABFX3MOPSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMZZGI2DCNJQGM>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[sepiatone]

Hey @Marcseb I just tried it out, and looks like I can hit our endpoints for the given model. Can you try with openai==0.28.xx ?

Here's a colab notebook if that helps: https://colab.research.google.com/drive/11HJWR3ylG1HSE2v78W1gRK-dKkSA0pHe?usp=sharing

@ShishirPatil

There has been some refactoring of the LangChain code earlier this year - the code should be as follows:

from langchain_openai import ChatOpenAI
chat_model = ChatOpenAI(
    openai_api_base="http://zanino.millennium.berkeley.edu:8000/v1",
    openai_api_key="EMPTY",
    model="gorilla-7b-hf-v1",
    verbose=True
)

and the prompt

example = chat_model.invoke("I want to translate from English to Chinese")
print(example.content)

I checked with the latest version of openai 1.23.6 at it works fine.

I've opened a PR #400 to update the blog post.