Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Hermes-2-Pro function calling example with JSON schema #390

Merged
merged 2 commits into from May 14, 2024

Conversation

rickzx
Copy link
Contributor

@rickzx rickzx commented May 8, 2024

This PR adds a function calling example with JSON schema, using Hermes-2-Pro-Mistral-7B model.

The system prompt is adapted from: https://github.com/NousResearch/Hermes-Function-Calling/blob/main/prompt_assets/sys_prompt.yml#L29

Query:

What is the current weather in celsius in Pittsburgh and Tokyo?

Output JSON schema:

const T = Type.Object({
  tool_calls: Type.Array(
    Type.Object({
      arguments: Type.Any(),
      name: Type.String(),
    })
  )
});

Generated output:

{"tool_calls": [{"arguments": {"location": "Pittsburgh, PA", "unit": "celsius"}, "name": "get_current_weather"}, {"arguments": {"location": "Tokyo, Japan", "unit": "celsius"}, "name": "get_current_weather"}]}

Copy link
Contributor

@CharlieFRuan CharlieFRuan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thank you so much!

Let's follow up with:

  • Update on function calling usage / example
  • Either support / remove gorilla support
  • q4f32 Hermes 2; update npm (for the prebuilt model), hence make ../.. back to 0.2.36

@CharlieFRuan CharlieFRuan merged commit a27b837 into mlc-ai:main May 14, 2024
CharlieFRuan added a commit that referenced this pull request May 21, 2024
### Changes
Main changes include:
- New model `Hermes-2-Pro-Mistral-7B` in `prebuiltAppConfig` via:
  - #390
- Various `index.js` and `index.js.map` post-processings to resolve
frontend compatibility issues with `require()` and `perf_hoooks`
  - #397
  - #406
- Catch WebGPU OOM error upon `reload()` and `CreateEngine()`:
  - #402
- Service Worker support (in addition to Extension Service Worker):
  - #395
  - #400
  - #401

### WASM Version
v0_2_34 as no change is required.

### TVMjs
TVMjs compiled at
apache/tvm@a5862a5,
with only one change in `tvm/web`:
apache/tvm#17005
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants