Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(agent): Remove unused autogpt code #7112

Merged

Conversation

kcze
Copy link
Contributor

@kcze kcze commented Apr 29, 2024

User description

Background

Should be merged after:

Follow-up after Component-based Agents

Changes 🏗️

Remove unused autogpt code and reorganize its file structure.

  • Moved:
    • autogpt/agent_manager/agent_manager.py to autogpt/agents/agent_manager.py, so the dir agent_manager was removed
    • dump_prompt from autogpt.core.runner.client_lib.logging.helpers to forge/llm/prompting/utils.py
    • coroutine decorator from autogpt.core.runner.client_lib.utils to autogpt/app/utils.py
  • Removed within autogpt:
    • Memory-related code from multiple files (not used), including memory/*
    • Memory-related config entries/env vars: memory_backend, memory_index, redis_host, redis_port, redis_password, wipe_redis_on_start
    • core files, from failed re-arch:
      • *.md docs
      • core/ability/*
      • core/agent/*
      • core/memory/*
      • core/planning/*
      • core/plugin/*
      • core/workspace/*
      • core/runner/* (dump_prompt and coroutine were moved)
    • Related tests
  • Updated relevant docs

PR Quality Scorecard ✨

  • Have you used the PR description template?   +2 pts
  • Is your pull request atomic, focusing on a single change?   +5 pts
  • Have you linked the GitHub issue(s) that this PR addresses?   +5 pts
  • Have you documented your changes clearly and comprehensively?   +5 pts
  • Have you changed or added a feature?   -4 pts
    • Have you added/updated corresponding documentation?   +4 pts
    • Have you added/updated corresponding integration tests?   +5 pts
  • Have you changed the behavior of AutoGPT?   -5 pts
    • Have you also run agbenchmark to verify that these changes do not regress performance?   +10 pts

PR Type

enhancement, bug fix, miscellaneous


Description

  • Replaced custom logging with standard logging module across multiple files.
  • Updated import paths for models, exceptions, and other modules.
  • Removed unused imports, parameters, and functions.
  • Added new classes and methods for better functionality and error handling.
  • Simplified and cleaned up code structure.
  • Updated documentation to reflect code changes.

Changes walkthrough 📝

Relevant files
Enhancement
34 files
db.py
Standardize logging and update model imports.                       

autogpts/forge/forge/agent_protocol/database/db.py

  • Replaced custom logger with standard logging module.
  • Updated import paths for models.
  • Changed Status to StepStatus.
  • +57/-47 
    api_router.py
    Standardize logging and update model imports.                       

    autogpts/forge/forge/agent_protocol/api_router.py

  • Replaced custom logger with standard logging module.
  • Updated import paths for exceptions and models.
  • +25/-25 
    main.py
    Clean up imports and remove unused parameters.                     

    autogpts/autogpt/autogpt/app/main.py

  • Removed unused imports and parameters.
  • Updated import paths for AgentManager.
  • Removed references to ai_settings and prompt_settings.
  • +5/-18   
    configurator.py
    Clean up unused imports and functions.                                     

    autogpts/autogpt/autogpt/app/configurator.py

    • Removed unused imports and functions.
    +1/-51   
    task.py
    Simplify task models and update status enum.                         

    autogpts/forge/forge/agent_protocol/models/task.py

    • Removed redundant classes.
    • Updated Status to StepStatus.
    +4/-60   
    text.py
    Add future annotations and update type hints.                       

    autogpts/forge/forge/content_processing/text.py

  • Added from __future__ import annotations.
  • Updated type hints for Config.
  • +7/-6     
    db_test.py
    Update imports and status enum in tests.                                 

    autogpts/forge/forge/agent_protocol/database/db_test.py

  • Updated import paths for database and models.
  • Changed Status to StepStatus.
  • +11/-5   
    agent_protocol_server.py
    Update import paths for agent protocol server.                     

    autogpts/autogpt/autogpt/app/agent_protocol_server.py

    • Updated import paths for AgentManager and other modules.
    +10/-10 
    config.py
    Add custom formatter for colored logging output.                 

    autogpts/forge/forge/logging/config.py

    • Added FancyConsoleFormatter for colored logging output.
    +45/-0   
    agent.py
    Standardize logging and update model imports.                       

    autogpts/forge/forge/agent/agent.py

  • Replaced custom logger with standard logging module.
  • Updated import paths for models and database.
  • +11/-12 
    system.py
    Add new constraints, resources, and best practices methods.

    autogpts/forge/forge/components/system/system.py

    • Added new constraints, resources, and best practices methods.
    +36/-0   
    agent_manager.py
    Add new AgentManager class.                                                           

    autogpts/autogpt/autogpt/agents/agent_manager.py

    • Added new AgentManager class for managing agent states.
    +47/-1   
    base.py
    Move default triggering prompt inside the file.                   

    autogpts/forge/forge/agent/base.py

    • Moved DEFAULT_TRIGGERING_PROMPT inside the file.
    +8/-12   
    artifact.py
    Add new artifact models.                                                                 

    autogpts/forge/forge/agent_protocol/models/artifact.py

    • Added new artifact models.
    +47/-0   
    ai_directives.py
    Remove file-based loading of AI directives.                           

    autogpts/forge/forge/config/ai_directives.py

    • Removed file-based loading of AI directives.
    +3/-24   
    utils.py
    Add coroutine decorator and update print_motd.                     

    autogpts/autogpt/autogpt/app/utils.py

  • Added coroutine decorator.
  • Removed config parameter from print_motd.
  • +14/-4   
    agent.py
    Add dump_prompt function.                                                               

    autogpts/autogpt/autogpt/agents/agent.py

    • Added dump_prompt function.
    +20/-1   
    generators.py
    Remove file-based loading of AI directives.                           

    autogpts/autogpt/autogpt/agent_factory/generators.py

    • Removed file-based loading of AI directives.
    +1/-3     
    benchmarks.py
    Update import paths for AgentManager.                                       

    autogpts/autogpt/agbenchmark_config/benchmarks.py

    • Updated import paths for AgentManager.
    +1/-2     
    selenium.py
    Add TooMuchOutputError class.                                                       

    autogpts/forge/forge/components/web/selenium.py

    • Added TooMuchOutputError class.
    +5/-1     
    __main__.py
    Standardize logging.                                                                         

    autogpts/forge/forge/main.py

    • Replaced custom logger with standard logging module.
    +3/-5     
    agent_test.py
    Update imports and workspace in tests.                                     

    autogpts/forge/forge/agent/agent_test.py

  • Updated import paths for database and models.
  • Changed workspace to use LocalFileStorage.
  • +14/-4   
    configurators.py
    Remove file-based loading of AI directives.                           

    autogpts/autogpt/autogpt/agent_factory/configurators.py

    • Removed file-based loading of AI directives.
    +1/-4     
    action_history.py
    Add future annotations.                                                                   

    autogpts/forge/forge/components/action_history/action_history.py

    • Added from __future__ import annotations.
    +3/-1     
    code_executor.py
    Add CodeExecutionError class.                                                       

    autogpts/forge/forge/components/code_executor/code_executor.py

    • Added CodeExecutionError class.
    +4/-1     
    pagination.py
    Add new pagination model.                                                               

    autogpts/forge/forge/agent_protocol/models/pagination.py

    • Added new pagination model.
    +8/-0     
    middlewares.py
    Update type hint for app parameter.                                           

    autogpts/forge/forge/agent_protocol/middlewares.py

    • Updated type hint for app parameter.
    +2/-7     
    __init__.py
    Add new models for artifact, pagination, and task.             

    autogpts/forge/forge/agent_protocol/models/init.py

    • Added new models for artifact, pagination, and task.
    +12/-0   
    git_log_to_release_notes.py
    Update import path for coroutine.                                               

    autogpts/autogpt/scripts/git_log_to_release_notes.py

    • Updated import path for coroutine.
    +1/-1     
    __init__.py
    Add AgentManager to __all__.                                                         

    autogpts/autogpt/autogpt/agents/init.py

    • Added AgentManager to __all__.
    +2/-0     
    parameter.py
    Change CommandParameter to inherit from BaseModel.             

    autogpts/forge/forge/command/parameter.py

    • Changed CommandParameter to inherit from BaseModel.
    +2/-3     
    __init__.py
    Add CodeExecutionError to imports.                                             

    autogpts/forge/forge/components/code_executor/init.py

    • Added CodeExecutionError to imports.
    +1/-0     
    __init__.py
    Add TooMuchOutputError to imports.                                             

    autogpts/forge/forge/components/web/init.py

    • Added TooMuchOutputError to imports.
    +1/-1     
    __init__.py
    Add AgentDB to imports.                                                                   

    autogpts/forge/forge/agent_protocol/database/init.py

    • Added AgentDB to imports.
    +1/-0     
    Documentation
    1 files
    building_challenges.md
    Update example code in documentation.                                       

    docs/content/challenges/building_challenges.md

    • Updated example code to remove memory_json_file.
    +1/-2     

    💡 PR-Agent usage:
    Comment /help on the PR to get a list of all available PR-Agent tools and their descriptions

    @github-actions github-actions bot added documentation Improvements or additions to documentation AutoGPT Agent Forge size/xl labels Apr 29, 2024
    Copy link

    netlify bot commented Apr 29, 2024

    Deploy Preview for auto-gpt-docs canceled.

    Name Link
    🔨 Latest commit 913b3e0
    🔍 Latest deploy log https://app.netlify.com/sites/auto-gpt-docs/deploys/664daec18df59b000873edc2

    @github-actions github-actions bot added the conflicts Automatically applied to PRs with merge conflicts label Apr 29, 2024
    Copy link

    This pull request has conflicts with the base branch, please resolve those so we can evaluate the pull request.

    @kcze kcze changed the title clean(autogpt): Remove unused autogpt code refactor(autogpt): Remove unused autogpt code May 1, 2024
    @kcze kcze changed the title refactor(autogpt): Remove unused autogpt code refactor(agent): Remove unused autogpt code May 2, 2024
    Copy link

    Conflicts have been resolved! 🎉 A maintainer will review the pull request shortly.

    @github-actions github-actions bot removed the conflicts Automatically applied to PRs with merge conflicts label May 21, 2024
    @kcze kcze marked this pull request as ready for review May 21, 2024 10:38
    @kcze kcze requested a review from a team as a code owner May 21, 2024 10:38
    @kcze kcze requested review from Bentlybro and majdyz and removed request for a team May 21, 2024 10:38
    @codiumai-pr-agent-pro codiumai-pr-agent-pro bot added enhancement New feature or request Bug fix miscellaneous Review effort [1-5]: 4 and removed documentation Improvements or additions to documentation labels May 21, 2024
    Copy link

    PR-Agent was enabled for this repository. To continue using it, please link your git user with your CodiumAI identity here.

    PR Review 🔍

    ⏱️ Estimated effort to review [1-5]

    4, because the PR involves a significant amount of changes across multiple files, including refactoring, renaming, and reorganization of code. The changes affect logging, database interactions, and model definitions, which are critical areas that require careful review to ensure functionality and integration are maintained.

    🧪 Relevant tests

    Yes

    ⚡ Possible issues

    Possible Bug: The renaming of logging instances from LOG to logger and the changes in import paths for models and database interactions could lead to issues if not all references were correctly updated. This needs thorough testing to ensure that all parts of the application are still communicating correctly.

    Performance Concern: The restructuring of model imports and database access patterns could potentially affect performance. Benchmarking or profiling might be necessary to ensure that performance has not degraded.

    🔒 Security concerns

    No

    Copy link

    PR-Agent was enabled for this repository. To continue using it, please link your git user with your CodiumAI identity here.

    PR Code Suggestions ✨

    CategorySuggestion                                                                                                                                    Score
    Best practice
    Use logger.exception to include stack traces in error logs for better debugging

    Consider using logger.exception instead of logger.error in the except Exception as e
    blocks to automatically include the stack trace in the log message. This can provide more
    context for debugging.

    autogpts/forge/forge/agent_protocol/database/db.py [178]

    -logger.error(f"Unexpected error while creating task: {e}")
    +logger.exception(f"Unexpected error while creating task: {e}")
     
    Suggestion importance[1-10]: 9

    Why: Using logger.exception in exception blocks is a best practice as it includes the stack trace in the log, providing more context for debugging. This is a significant improvement for error handling.

    9
    Possible issue
    Ensure database sessions are closed in a finally block to avoid resource leaks

    Consider adding a check to ensure that session.close() is called in a finally block to
    guarantee that the session is closed even if an exception occurs.

    autogpts/forge/forge/agent_protocol/database/db.py [237]

    -session.close()
    +finally:
    +    session.close()
     
    Suggestion importance[1-10]: 8

    Why: Ensuring that database sessions are closed even in the event of an error is crucial for resource management and preventing leaks. This is a valid and important improvement.

    8
    Add exception handling to the coroutine decorator

    The coroutine decorator should handle exceptions that might occur during the execution of
    the coroutine to prevent the application from crashing unexpectedly.

    autogpts/autogpt/autogpt/app/utils.py [244-245]

     def wrapper(*args: P.args, **kwargs: P.kwargs):
    -    return asyncio.run(f(*args, **kwargs))
    +    try:
    +        return asyncio.run(f(*args, **kwargs))
    +    except Exception as e:
    +        logger.error(f"Exception occurred in coroutine: {e}")
    +        raise
     
    Suggestion importance[1-10]: 8

    Why: Exception handling in the coroutine decorator is crucial for robustness and preventing crashes, making this a high-value suggestion.

    8
    Ensure the coroutine decorator does not call asyncio.run if the event loop is already running

    The coroutine decorator should use asyncio.run only if the event loop is not already
    running to avoid runtime errors.

    autogpts/autogpt/autogpt/app/utils.py [244-245]

     def wrapper(*args: P.args, **kwargs: P.kwargs):
    -    return asyncio.run(f(*args, **kwargs))
    +    if not asyncio.get_event_loop().is_running():
    +        return asyncio.run(f(*args, **kwargs))
    +    else:
    +        return f(*args, **kwargs)
     
    Suggestion importance[1-10]: 8

    Why: Ensuring asyncio.run is called only when the event loop isn't running is crucial to avoid runtime errors, thus a high-value suggestion.

    8
    Maintainability
    Remove the unused config parameter from the split_text function

    The split_text function's config parameter is not used within the function. Consider
    removing it if it is not necessary.

    autogpts/forge/forge/content_processing/text.py [222-227]

     def split_text(
         text: str,
    -    config: Config,
         max_chunk_length: int,
         tokenizer: ModelTokenizer,
         with_overlap: bool = True,
     ) -> list[str]:
     
    Suggestion importance[1-10]: 8

    Why: The suggestion is valid and improves the function by removing an unused parameter, thus enhancing code clarity and maintainability.

    8

    Copy link

    codecov bot commented May 21, 2024

    Codecov Report

    Attention: Patch coverage is 86.66667% with 2 lines in your changes are missing coverage. Please review.

    Project coverage is 36.05%. Comparing base (bcc5282) to head (913b3e0).

    Files Patch % Lines
    ...ogpts/autogpt/autogpt/app/agent_protocol_server.py 0.00% 1 Missing ⚠️
    autogpts/autogpt/autogpt/app/utils.py 90.00% 1 Missing ⚠️
    Additional details and impacted files
    @@             Coverage Diff             @@
    ##           master    #7112       +/-   ##
    ===========================================
    + Coverage   22.63%   36.05%   +13.42%     
    ===========================================
      Files          66       19       -47     
      Lines        2669     1273     -1396     
      Branches      299      182      -117     
    ===========================================
    - Hits          604      459      -145     
    + Misses       2035      786     -1249     
    + Partials       30       28        -2     
    Flag Coverage Δ
    Linux 36.05% <86.66%> (+13.42%) ⬆️
    Windows 35.91% <86.66%> (+13.39%) ⬆️
    autogpt-agent 36.05% <86.66%> (+13.42%) ⬆️
    macOS 36.05% <86.66%> (+13.42%) ⬆️

    Flags with carried forward coverage won't be shown. Click here to find out more.

    ☔ View full report in Codecov by Sentry.
    📢 Have feedback on the report? Share it here.

    @kcze kcze requested review from Pwuts and removed request for Bentlybro May 21, 2024 10:42
    @github-actions github-actions bot added the documentation Improvements or additions to documentation label May 21, 2024
    @kcze kcze force-pushed the kpczerwinski/open-428-clean-autogpt branch from 351408a to 7832e32 Compare May 21, 2024 15:51
    @github-actions github-actions bot added the conflicts Automatically applied to PRs with merge conflicts label May 21, 2024
    Copy link

    This pull request has conflicts with the base branch, please resolve those so we can evaluate the pull request.

    @github-actions github-actions bot removed the conflicts Automatically applied to PRs with merge conflicts label May 21, 2024
    Copy link

    Conflicts have been resolved! 🎉 A maintainer will review the pull request shortly.

    Copy link
    Member

    @Pwuts Pwuts left a comment

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    Looks good overall!

    autogpts/autogpt/autogpt/agents/agent.py Outdated Show resolved Hide resolved
    @kcze kcze merged commit 3475aaf into Significant-Gravitas:master May 22, 2024
    14 of 19 checks passed
    @kcze kcze deleted the kpczerwinski/open-428-clean-autogpt branch May 22, 2024 08:43
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Projects
    Status: Done
    Development

    Successfully merging this pull request may close these issues.

    None yet

    2 participants