feat(forge/llm): Add GroqProvider

[Pwuts]

Background

This has issues with the profile generator, so maybe merge #7121 first

Changes 🏗️

• Add GroqProvider in forge.llm.providers.groq

  • Add to llm.providers.multi
  • Add groq dependency (v0.8.0)

• Update AutoGPT docs & config template

  • Update .env.template
  • Update docs

PR Quality Scorecard ✨

• Have you used the PR description template?   +2 pts
• Is your pull request atomic, focusing on a single change?   +5 pts
• Have you linked the GitHub issue(s) that this PR addresses?   +5 pts
• Have you documented your changes clearly and comprehensively?   +5 pts
• Have you changed or added a feature?   -4 pts
  • Have you added/updated corresponding documentation?   +4 pts
  • Have you added/updated corresponding integration tests?   +5 pts
• Have you changed the behavior of AutoGPT?   -5 pts
  • Have you also run agbenchmark to verify that these changes do not regress performance?   +10 pts

---

PR Type

Enhancement

---

Description

• Introduced GroqProvider to manage interactions with the Groq API, including methods for chat completions and API retries.
• Updated multi.py to integrate GroqProvider alongside other model providers.
• Extended ModelProviderName enum to include GROQ.
• Added the groq package to project dependencies to ensure API interaction capabilities.

---

Changes walkthrough 📝

	Relevant files
Enhancement	groq.py Implement GroqProvider for Groq API Interaction                    autogpts/autogpt/autogpt/core/resource/model_providers/groq.py Introduced a new GroqProvider class for handling interactions with the Groq API. Defined model configurations, credentials, and settings for Groq. Implemented methods for creating chat completions and handling API retries. +417/-0  multi.py Integrate GroqProvider into MultiProvider                                autogpts/autogpt/autogpt/core/resource/model_providers/multi.py Added GroqProvider to the list of model providers. Updated the ModelName type to include GroqModelName. Included GROQ_CHAT_MODELS in the combined chat models dictionary. +6/-4      schema.py Update ModelProviderName Enum with GROQ                                    autogpts/autogpt/autogpt/core/resource/model_providers/schema.py Added GROQ to the ModelProviderName enum. +1/-0
Dependencies	pyproject.toml Add Groq Dependency in pyproject.toml                                        autogpts/autogpt/pyproject.toml Added groq package as a dependency. +1/-0

---

💡 PR-Agent usage:
Comment /help on the PR to get a list of all available PR-Agent tools and their descriptions

[netlify]

✅ Deploy Preview for auto-gpt-docs canceled.

Name	Link
🔨 Latest commit	bfc8c0b
🔍 Latest deploy log	https://app.netlify.com/sites/auto-gpt-docs/deploys/6650a302576163000833eba3

[codiumai-pr-agent-pro]

PR Description updated to latest commit (976396c)

• Copy walkthrough table to "Files Changed" Tab

[codiumai-pr-agent-pro]

PR Review 🔍

⏱️ Estimated effort to review [1-5]	4, because the PR introduces a significant amount of new code (418 lines) related to the integration of a new model provider (GroqProvider). The complexity of the code, which includes handling API connections, error parsing, retries, and token management, requires a thorough review to ensure functionality and maintainability.
🧪 Relevant tests	No
⚡ Possible issues	Possible Bug: The method count_tokens and count_message_tokens in GroqProvider return 0, which seems incorrect and could lead to issues with token counting.
	Error Handling: The error handling in create_chat_completion might not properly manage all exceptions, especially with nested try-except blocks.
	Performance Concern: The recursive appending of messages in create_chat_completion during parsing failures might lead to excessive memory use or infinite loops if not controlled properly.
🔒 Security concerns	No

[codiumai-pr-agent-pro]

PR Code Suggestions ✨

Category	Suggestions
Enhancement	Replace hardcoded token costs with dynamic fetching to enhance maintainability. Replace the hardcoded token costs with a more dynamic approach, such as fetching them from an external configuration or API. This will make the system more flexible and easier to maintain as token costs change. autogpts/autogpt/autogpt/core/resource/model_providers/groq.py [56-57] -prompt_token_cost=0.05 / 1e6, -completion_token_cost=0.10 / 1e6, +prompt_token_cost=self.get_token_cost('prompt'), +completion_token_cost=self.get_token_cost('completion'),
	Add logging for token limits to assist in debugging and monitoring. Add logging for the token costs in the get_token_limit method to aid in debugging and monitoring the token usage. autogpts/autogpt/autogpt/core/resource/model_providers/groq.py [154-156] def get_token_limit(self, model_name: str) -> int: - return GROQ_CHAT_MODELS[model_name].max_tokens + max_tokens = GROQ_CHAT_MODELS[model_name].max_tokens + self._logger.debug(f"Token limit for model {model_name}: {max_tokens}") + return max_tokens
Robustness	Add error handling for API key retrieval to prevent runtime errors. Implement error handling for the API key retrieval in GroqCredentials to ensure robustness in scenarios where the environment variables are not set or are invalid. autogpts/autogpt/autogpt/core/resource/model_providers/groq.py [96] -api_key: SecretStr = UserConfigurable(from_env="ANTHROPIC_API_KEY") +api_key: SecretStr = UserConfigurable(from_env="ANTHROPIC_API_KEY", default=SecretStr('default_api_key'))
	Improve error handling in _parse_assistant_tool_calls for better fault tolerance. Use a more robust error handling strategy in the _parse_assistant_tool_calls method to ensure that all errors are logged and appropriately handled, rather than just continuing on failure. autogpts/autogpt/autogpt/core/resource/model_providers/groq.py [370-383] if assistant_message.tool_calls: for _tc in assistant_message.tool_calls: try: parsed_arguments = json_loads(_tc.function.arguments) except Exception as e: err_message = ( f"Decoding arguments for {_tc.function.name} failed: " + str(e.args[0]) ) - parse_errors.append( - type(e)(err_message, *e.args[1:]).with_traceback( - e.__traceback__ - ) - ) + self._logger.error(err_message) + sentry_sdk.capture_exception(e) continue
Maintainability	Refactor create_chat_completion to improve code readability and maintainability. Refactor the create_chat_completion method to separate the concerns of API calling and response parsing into distinct methods. This will improve the readability and maintainability of the code. autogpts/autogpt/autogpt/core/resource/model_providers/groq.py [175-184] async def create_chat_completion( self, model_prompt: list[ChatMessage], model_name: GroqModelName, completion_parser: Callable[[AssistantChatMessage], _T] = lambda _: None, functions: Optional[list[CompletionModelFunction]] = None, max_output_tokens: Optional[int] = None, prefill_response: str = "", **kwargs, ) -> ChatModelResponse[_T]: + groq_messages, completion_kwargs = self._prepare_completion_kwargs( + model_prompt, model_name, functions, max_output_tokens, **kwargs + ) + response = await self._call_api_for_completion(completion_kwargs) + return self._parse_api_response(response, completion_parser)

[codiumai-pr-agent-pro]

Changelog updates: 🔄

2024-05-07

Added

• Introduced GroqProvider to manage interactions with Groq's API, expanding the available model providers.

to commit the new content to the CHANGELOG.md file, please type:
'/update_changelog --pr_update_changelog.push_changelog_changes=true'

[codiumai-pr-agent-pro]

PR Analysis 🔬

• This screen contains a list of code components that were changed in this PR.
• You can initiate specific actions for each component, by checking the relevant boxes.
• After you check a box, the action will be performed automatically by PR-Agent.
• Results will appear as a comment on the PR, typically after 30-60 seconds.

file	Changed components
groq.py	Test Docs Improve Similar   GroqModelName (class)   +5/-0   Test Docs Improve Similar   GroqConfiguration (class)   +2/-0   Test Docs Improve Similar   GroqCredentials (class)   +17/-0   Test Docs Improve Similar   GroqSettings (class)   +4/-0   Test Docs Improve Similar   GroqProvider (class)   +300/-0
multi.py	Test Docs Improve Similar   _get_model_provider_class (method of MultiProvider)   +2/-2   Test Docs Improve Similar   _get_provider_class (method of MultiProvider)   +3/-2
schema.py	Test Docs Improve Similar   ModelProviderName (class)   +1/-0

---

💡 Usage guide:

---

Using static code analysis capabilities, the analyze tool scans the PR code changes and find the code components (methods, functions, classes) that changed in the PR.

The tool can be triggered automatically every time a new PR is opened, or can be invoked manually by commenting on any PR:

/analyze

Language that are currently supported: Python, Java, C++, JavaScript, TypeScript, C#.
See more information about the tool in the docs.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 36.05%. Comparing base (5292736) to head (bfc8c0b).
Report is 1 commits behind head on master.

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #7130   +/-   ##
=======================================
  Coverage   36.05%   36.05%           
=======================================
  Files          19       19           
  Lines        1273     1273           
  Branches      182      182           
=======================================
  Hits          459      459           
  Misses        786      786           
  Partials       28       28           

Flag	Coverage Δ
Linux	36.05% <ø> (ø)
Windows	35.91% <ø> (ø)
autogpt-agent	36.05% <ø> (ø)
macOS	36.05% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[ntindle]

GROQ_API_BASE

[Pwuts]

Technically available, but relevant?

[ntindle]

Imo we should treat this as up to date as possible

[Swiftyos]

Why do we need a new provider, when Groq is compatible with OpenAI and just needs the base_url changing?

[ntindle]

Why do we need a new provider, when Groq is compatible with OpenAI and just needs the base_url changing?

We would want a provider to support additional models with different features, costs, and requirements than OpenAI. Currently, we support anthropic and OpenAI, and both require different things like the above. In the future, this is where things like a rate limit to prevent overages would live as well.

[github-actions]

This pull request has conflicts with the base branch, please resolve those so we can evaluate the pull request.

[github-actions]

Conflicts have been resolved! 🎉 A maintainer will review the pull request shortly.

[ntindle]

is there a way to do this relative??????????

[Pwuts]

In pyproject.toml it is relative, but poetry.lock resolves it to an absolute path. Luckily this doesn't break it on other systems (afaik)