Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

clean: Remove Depreciated Challenges #7144

Merged
merged 2 commits into from May 20, 2024

Conversation

Swiftyos
Copy link
Contributor

@Swiftyos Swiftyos commented May 11, 2024

User description

Deleting the Depreciated Challenges as why are they here anyway!?

Also there are code errors in the depreciated challenge's which make the unparseable by linters and formatters

like:

import from

PR Type

Enhancement, Bug fix


Description

  • Removed all deprecated challenge files and directories from the benchmark project, cleaning up the repository and improving maintainability.
  • Updated the authors field in pyproject.toml to reflect the current team, "AutoGPT Team".
  • Updated the copyright year in the LICENSE file to 2024, ensuring legal accuracy.

Changes walkthrough 📝

Relevant files
Configuration changes
pyproject.toml
Update Project Author in pyproject.toml                                   

benchmark/pyproject.toml

  • Updated the authors field from "Silen Naihin" to "AutoGPT Team".
+1/-1     
Documentation
LICENSE
Update Copyright Year in LICENSE                                                 

benchmark/LICENSE

  • Updated the copyright year from 2023 to 2024.
+1/-1     

💡 PR-Agent usage:
Comment /help on the PR to get a list of all available PR-Agent tools and their descriptions

@Swiftyos Swiftyos requested review from Pwuts and kcze May 11, 2024 07:41
@Swiftyos Swiftyos requested a review from ntindle May 11, 2024 07:41
Copy link

netlify bot commented May 11, 2024

Deploy Preview for auto-gpt-docs canceled.

Name Link
🔨 Latest commit 7230e13
🔍 Latest deploy log https://app.netlify.com/sites/auto-gpt-docs/deploys/663f2105e989420008abc8d1

@codiumai-pr-agent-pro codiumai-pr-agent-pro bot added enhancement New feature or request Bug fix labels May 11, 2024
Copy link

PR Description updated to latest commit (7230e13)

Copy link

PR Review 🔍

⏱️ Estimated effort to review [1-5]

2, because the changes are straightforward, involving simple updates to metadata files and the removal of deprecated files. The scope is limited and does not involve complex logic changes.

🧪 Relevant tests

No

⚡ Possible issues

No

🔒 Security concerns

No

Copy link

PR Code Suggestions ✨

CategorySuggestions                                                                                                                                                       
Legal
Verify the legality of the copyright change.

Ensure that the copyright change from "Silen Naihin" to "AutoGPT" is legally verified and
approved. This change might involve legal implications that need to be addressed to avoid
copyright infringement or disputes.

benchmark/LICENSE [3]

-Copyright (c) 2024 AutoGPT
+Copyright (c) 2024 AutoGPT (pending legal verification)
 
Maintainability
Include all contributors in the authors field.

Update the authors field to include both the new and previous contributors, unless there
is a specific reason to remove the original author. This helps in maintaining a clear
record of contributions.

benchmark/pyproject.toml [5]

-authors = ["AutoGPT Team"]
+authors = ["AutoGPT Team", "Silen Naihin <silen.naihin@gmail.com>"]
 

Copy link

Changelog updates: 🔄

2024-05-11

Removed

  • Removed all deprecated challenge files and directories, enhancing project cleanliness and maintainability.

Changed

  • Updated the authors field in pyproject.toml to "AutoGPT Team".
  • Updated the copyright year in the LICENSE file to 2024.

to commit the new content to the CHANGELOG.md file, please type:
'/update_changelog --pr_update_changelog.push_changelog_changes=true'

Copy link

PR Analysis 🔬

  • This screen contains a list of code components that were changed in this PR.
  • You can initiate specific actions for each component, by checking the relevant boxes.
  • After you check a box, the action will be performed automatically by PR-Agent.
  • Results will appear as a comment on the PR, typically after 30-60 seconds.
fileChanged components

💡 Usage guide:

Using static code analysis capabilities, the analyze tool scans the PR code changes and find the code components (methods, functions, classes) that changed in the PR.

The tool can be triggered automatically every time a new PR is opened, or can be invoked manually by commenting on any PR:

/analyze

Language that are currently supported: Python, Java, C++, JavaScript, TypeScript, C#.
See more information about the tool in the docs.

Copy link

CI Failure Feedback 🧐

Action: tests-agbenchmark (forge)

Failed stage: Run regression tests [❌]

Failure summary:

The action failed due to a ValueError raised during the pytest collection phase. The error was
triggered by the validate_skill_tree function, which found multiple roots in the skill tree. This
indicates that the skill tree structure was not correctly defined, as it should have a single root.
The presence of multiple roots in the skill tree led to the failure of the test collection process,
resulting in no tests being run.

Relevant error logs:
1:  ##[group]Operating System
2:  Ubuntu
...

232:  �[36;1m�[0m
233:  �[36;1mecho "Running the following command: poetry run agbenchmark --mock --category=coding"�[0m
234:  �[36;1mpoetry run agbenchmark --mock --category=coding�[0m
235:  �[36;1m�[0m
236:  �[36;1mecho "Running the following command: poetry run agbenchmark --test=WriteFile"�[0m
237:  �[36;1mpoetry run agbenchmark --test=WriteFile�[0m
238:  �[36;1mcd ../../benchmark�[0m
239:  �[36;1mpoetry install�[0m
240:  �[36;1mecho "Adding the BUILD_SKILL_TREE environment variable. This will attempt to add new elements in the skill tree. If new elements are added, the CI fails because they should have been pushed"�[0m
...

491:  888      "Y88P"  888     "Y88888  "Y8888              
492:  888                      
493:  Y8b d88P                      
494:  "Y88P"                v0.1.0
495:  [2024-05-11 07:42:12,180] [�[1m�[90mwatchfiles.main�[0m�[0m] [�[94mINFO�[0m]	�[94m📝  16 changes detected�[0m
496:  ✅ Agent application started and available on port 8000
497:  Running the following command: poetry run agbenchmark --maintain --mock
498:  [2024-05-11 07:42:12,532] [�[1m�[90mwatchfiles.main�[0m�[0m] [�[94mINFO�[0m]	�[94m📝  3 changes detected�[0m
499:  [2024-05-11 07:42:14,462] �[34mINFO �[0m Failed to extract font properties from /usr/share/fonts/truetype/noto/NotoColorEmoji.ttf: In FT2Font: Can not load face (unknown file format; error code 0x2)
...

768:  [2024-05-11 07:42:20,418] [�[1m�[90mforge.agent    �[0m�[0m] [�[94mINFO�[0m]	�[94m📝  📦 Task created: a7935256-afaa-4599-9d0a-8ea56bd906d1 input: Create a random password generator. The ...�[0m
769:  [2024-05-11 07:42:20,425] [�[1m�[90mforge.agent    �[0m�[0m] [�[94mINFO�[0m]	�[94m📝  	✅ Final Step completed: f53f3664-9477-4e4b-996d-ab682c94a5de. 
770:  Output should be placeholder text Washington D.C. You'll need to 
771:  modify execute_step to include LLM behavior. Follow the tutorial if confused. �[0m
772:  [2024-05-11 07:42:20,496] [�[1m�[90mwatchfiles.main�[0m�[0m] [�[94mINFO�[0m]	�[94m📝  65 changes detected�[0m
773:  ../../../../../.cache/pypoetry/virtualenvs/autogpt-forge-91r9A6WA-py3.10/lib/python3.10/site-packages/agbenchmark/generate_test.py::TestPasswordGenerator::test_method[0] 
774:  �[35m�[1m======================== Starting PasswordGenerator challenge ========================�[0m
775:  �[36mTimeout:�[39m 90 seconds
776:  �[36mTask:�[39m Create a random password generator. The password should have between 8 and 16 characters and should contain at least one letter, number and symbol. The password should be printed to the console. The entry point will be a python file that can be run this way: python password_generator.py [--length x] where x is the length of the password. If no length is specified, the password should be 8 characters long. The password_generator can also be imported as a module and called as password = password_generator.generate_password(length=x). Any invalid input should raise a ValueError.
...

1087:  class ShipPlacement(BaseModel):
1088:  ship_type: str
1089:  start: dict  # {"row": int, "column": str}
1090:  direction: str
1091:  @validator("start")
1092:  def validate_start(cls, start):
1093:  row, column = start.get("row"), start.get("column")
1094:  if not (1 <= row <= 10):
1095:  raise ValueError("Row must be between 1 and 10 inclusive.")
1096:  if column not in list("ABCDEFGHIJ"):
1097:  raise ValueError("Column must be one of A, B, C, D, E, F, G, H, I, J.")
...

1410:  PASSED
1411:  [2024-05-11 07:42:29,421] [�[1m�[90mforge.agent    �[0m�[0m] [�[94mINFO�[0m]	�[94m📝  📦 Task created: d52bf206-ec7b-4993-9fc3-25e9cab2ba88 input: Create a random password generator. The ...�[0m
1412:  [2024-05-11 07:42:29,428] [�[1m�[90mforge.agent    �[0m�[0m] [�[94mINFO�[0m]	�[94m📝  	✅ Final Step completed: c1b49493-d695-4804-80f3-633adfbbccc0. 
1413:  Output should be placeholder text Washington D.C. You'll need to 
1414:  modify execute_step to include LLM behavior. Follow the tutorial if confused. �[0m
1415:  ../../../../../.cache/pypoetry/virtualenvs/autogpt-forge-91r9A6WA-py3.10/lib/python3.10/site-packages/agbenchmark/generate_test.py::TestPasswordGenerator::test_method[0] 
1416:  �[35m�[1m======================== Starting PasswordGenerator challenge ========================�[0m
1417:  �[36mTimeout:�[39m 90 seconds
1418:  �[36mTask:�[39m Create a random password generator. The password should have between 8 and 16 characters and should contain at least one letter, number and symbol. The password should be printed to the console. The entry point will be a python file that can be run this way: python password_generator.py [--length x] where x is the length of the password. If no length is specified, the password should be 8 characters long. The password_generator can also be imported as a module and called as password = password_generator.generate_password(length=x). Any invalid input should raise a ValueError.
...

1541:  class ShipPlacement(BaseModel):
1542:  ship_type: str
1543:  start: dict  # {"row": int, "column": str}
1544:  direction: str
1545:  @validator("start")
1546:  def validate_start(cls, start):
1547:  row, column = start.get("row"), start.get("column")
1548:  if not (1 <= row <= 10):
1549:  raise ValueError("Row must be between 1 and 10 inclusive.")
1550:  if column not in list("ABCDEFGHIJ"):
1551:  raise ValueError("Column must be one of A, B, C, D, E, F, G, H, I, J.")
...

1785:  - Installing python-multipart (0.0.7)
1786:  - Installing pyvis (0.3.2)
1787:  - Installing selenium (4.16.0)
1788:  - Installing tabulate (0.9.0)
1789:  - Installing toml (0.10.2)
1790:  - Installing types-requests (2.31.0.6)
1791:  - Installing uvicorn (0.23.2)
1792:  Installing the current project: agbenchmark (0.0.10)
1793:  Adding the BUILD_SKILL_TREE environment variable. This will attempt to add new elements in the skill tree. If new elements are added, the CI fails because they should have been pushed
...

1800:  ============================= test session starts ==============================
1801:  platform linux -- Python 3.10.14, pytest-7.4.4, pluggy-1.3.0
1802:  rootdir: /home/runner/work/AutoGPT/AutoGPT/benchmark
1803:  configfile: pyproject.toml
1804:  plugins: anyio-4.2.0, asyncio-0.21.1
1805:  asyncio: mode=auto
1806:  Warning: When  cdn_resources is 'local' jupyter notebook has issues displaying graphics on chrome/safari. Use cdn_resources='in_line' or cdn_resources='remote' if you have issues viewing graphics in a notebook.
1807:  collected 33 items
1808:  INTERNALERROR> Traceback (most recent call last):
1809:  INTERNALERROR>   File "/home/runner/.cache/pypoetry/virtualenvs/agbenchmark-uSLv8lPB-py3.10/lib/python3.10/site-packages/_pytest/main.py", line 271, in wrap_session
1810:  INTERNALERROR>     session.exitstatus = doit(config, session) or 0
1811:  INTERNALERROR>   File "/home/runner/.cache/pypoetry/virtualenvs/agbenchmark-uSLv8lPB-py3.10/lib/python3.10/site-packages/_pytest/main.py", line 324, in _main
1812:  INTERNALERROR>     config.hook.pytest_collection(session=session)
1813:  INTERNALERROR>   File "/home/runner/.cache/pypoetry/virtualenvs/agbenchmark-uSLv8lPB-py3.10/lib/python3.10/site-packages/pluggy/_hooks.py", line 493, in __call__
1814:  INTERNALERROR>     return self._hookexec(self.name, self._hookimpls, kwargs, firstresult)
1815:  INTERNALERROR>   File "/home/runner/.cache/pypoetry/virtualenvs/agbenchmark-uSLv8lPB-py3.10/lib/python3.10/site-packages/pluggy/_manager.py", line 115, in _hookexec
1816:  INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
1817:  INTERNALERROR>   File "/home/runner/.cache/pypoetry/virtualenvs/agbenchmark-uSLv8lPB-py3.10/lib/python3.10/site-packages/pluggy/_callers.py", line 152, in _multicall
1818:  INTERNALERROR>     return outcome.get_result()
1819:  INTERNALERROR>   File "/home/runner/.cache/pypoetry/virtualenvs/agbenchmark-uSLv8lPB-py3.10/lib/python3.10/site-packages/pluggy/_result.py", line 114, in get_result
1820:  INTERNALERROR>     raise exc.with_traceback(exc.__traceback__)
1821:  INTERNALERROR>   File "/home/runner/.cache/pypoetry/virtualenvs/agbenchmark-uSLv8lPB-py3.10/lib/python3.10/site-packages/pluggy/_callers.py", line 77, in _multicall
1822:  INTERNALERROR>     res = hook_impl.function(*args)
1823:  INTERNALERROR>   File "/home/runner/.cache/pypoetry/virtualenvs/agbenchmark-uSLv8lPB-py3.10/lib/python3.10/site-packages/_pytest/main.py", line 335, in pytest_collection
1824:  INTERNALERROR>     session.perform_collect()
1825:  INTERNALERROR>   File "/home/runner/.cache/pypoetry/virtualenvs/agbenchmark-uSLv8lPB-py3.10/lib/python3.10/site-packages/_pytest/main.py", line 678, in perform_collect
1826:  INTERNALERROR>     hook.pytest_collection_modifyitems(
1827:  INTERNALERROR>   File "/home/runner/.cache/pypoetry/virtualenvs/agbenchmark-uSLv8lPB-py3.10/lib/python3.10/site-packages/pluggy/_hooks.py", line 493, in __call__
1828:  INTERNALERROR>     return self._hookexec(self.name, self._hookimpls, kwargs, firstresult)
1829:  INTERNALERROR>   File "/home/runner/.cache/pypoetry/virtualenvs/agbenchmark-uSLv8lPB-py3.10/lib/python3.10/site-packages/pluggy/_manager.py", line 115, in _hookexec
1830:  INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
1831:  INTERNALERROR>   File "/home/runner/.cache/pypoetry/virtualenvs/agbenchmark-uSLv8lPB-py3.10/lib/python3.10/site-packages/pluggy/_callers.py", line 152, in _multicall
1832:  INTERNALERROR>     return outcome.get_result()
1833:  INTERNALERROR>   File "/home/runner/.cache/pypoetry/virtualenvs/agbenchmark-uSLv8lPB-py3.10/lib/python3.10/site-packages/pluggy/_result.py", line 114, in get_result
1834:  INTERNALERROR>     raise exc.with_traceback(exc.__traceback__)
1835:  INTERNALERROR>   File "/home/runner/.cache/pypoetry/virtualenvs/agbenchmark-uSLv8lPB-py3.10/lib/python3.10/site-packages/pluggy/_callers.py", line 77, in _multicall
1836:  INTERNALERROR>     res = hook_impl.function(*args)
1837:  INTERNALERROR>   File "/home/runner/work/AutoGPT/AutoGPT/benchmark/agbenchmark/utils/dependencies/__init__.py", line 157, in pytest_collection_modifyitems
1838:  INTERNALERROR>     items[:] = manager.sorted_items
1839:  INTERNALERROR>   File "/home/runner/work/AutoGPT/AutoGPT/benchmark/agbenchmark/utils/dependencies/main.py", line 230, in sorted_items
1840:  INTERNALERROR>     graph_interactive_network(dag, labels, html_graph_path="")
1841:  INTERNALERROR>   File "/home/runner/work/AutoGPT/AutoGPT/benchmark/agbenchmark/utils/dependencies/graphs.py", line 293, in graph_interactive_network
1842:  INTERNALERROR>     validate_skill_tree(graph_data, "")
1843:  INTERNALERROR>   File "/home/runner/work/AutoGPT/AutoGPT/benchmark/agbenchmark/utils/dependencies/graphs.py", line 443, in validate_skill_tree
1844:  INTERNALERROR>     raise ValueError(f"{skill_tree_name} skill tree has multiple roots: {roots}.")
1845:  INTERNALERROR> ValueError:  skill tree has multiple roots: ['agbenchmark/generate_test.py::TestWriteFile::test_method[0]', 'agbenchmark/generate_test.py::TestWebArenaTask_27::test_method[0]', 'agbenchmark/generate_test.py::TestWebArenaTask_4::test_method[0]', 'agbenchmark/generate_test.py::TestWebArenaTask_112::test_method[0]', 'agbenchmark/generate_test.py::TestWebArenaTask_6::test_method[0]', 'agbenchmark/generate_test.py::TestWebArenaTask_124::test_method[0]', 'agbenchmark/generate_test.py::TestWebArenaTask_11::test_method[0]', 'agbenchmark/generate_test.py::TestWebArenaTask_0::test_method[0]', 'agbenchmark/generate_test.py::TestWebArenaTask_77::test_method[0]', 'agbenchmark/generate_test.py::TestWebArenaTask_163::test_method[0]', 'agbenchmark/generate_test.py::TestWebArenaTask_21::test_method[0]', 'agbenchmark/generate_test.py::TestWebArenaTask_43::test_method[0]', 'agbenchmark/generate_test.py::TestWebArenaTask_107::test_method[0]', 'agbenchmark/generate_test.py::TestWebArenaTask_95::test_method[0]'].
1846:  ============================ no tests ran in 0.05s =============================
1847:  ##[error]Process completed with exit code 3.

✨ CI feedback usage guide:

The CI feedback tool (/checks) automatically triggers when a PR has a failed check.
The tool analyzes the failed checks and provides several feedbacks:

  • Failed stage
  • Failed test name
  • Failure summary
  • Relevant error logs

In addition to being automatically triggered, the tool can also be invoked manually by commenting on a PR:

/checks "https://github.com/{repo_name}/actions/runs/{run_number}/job/{job_number}"

where {repo_name} is the name of the repository, {run_number} is the run number of the failed check, and {job_number} is the job number of the failed check.

Configuration options

  • enable_auto_checks_feedback - if set to true, the tool will automatically provide feedback when a check is failed. Default is true.
  • excluded_checks_list - a list of checks to exclude from the feedback, for example: ["check1", "check2"]. Default is an empty list.
  • enable_help_text - if set to true, the tool will provide a help message with the feedback. Default is true.
  • persistent_comment - if set to true, the tool will overwrite a previous checks comment with the new feedback. Default is true.
  • final_update_message - if persistent_comment is true and updating a previous checks message, the tool will also create a new message: "Persistent checks updated to latest commit". Default is true.

See more information about the checks tool in the docs.

Copy link
Contributor

@kcze kcze left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good

@Swiftyos Swiftyos merged commit 2cca4fa into master May 20, 2024
13 of 14 checks passed
@Swiftyos Swiftyos deleted the swiftyos/remove-depreciated-benchmarks branch May 20, 2024 13:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

None yet

2 participants