Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix:Introduce _is_python_code check into _extract_code #1140

Merged
merged 1 commit into from May 14, 2024

Conversation

adamingas
Copy link
Contributor

@adamingas adamingas commented May 1, 2024

  • Add tests to cover more cases for _extract_code and _is_python_code

Motivation:
Using a dataframe with a large number of columns, the LLM()._execute_code method would fail when asking how many rows there are in the dataframe. The failure would be because the LLM wasn't using triple back ticks to surround the code.

If the LLM produces correct python code, but for some reason does not
preface it with ```, the LLM()._extract_code method fails with
NoCodeFoundError.

In this change we first check if the separator is included in the response and we take only the part that is in-between. Then we clean the code from python/py statements.
Then we check if it's valid code. This check should pass otherwise executing the code will fail, and it's better to raise an error earlier than later.

With this change, if the LLM produced perfect code without the ``` separator, it would pass and return the code, whereas previously it would fail.

* Add tests to cover more cases for `_extract_code` and `_is_python_code`
@gventuri gventuri requested a review from ArslanSaleem May 1, 2024 21:23
@tushar-31093
Copy link

Guys, this is a genuine concern, could you confirm if this fix might solve this problem?

@adamingas
Copy link
Contributor Author

@gventuri Can someone take a look at the PR? Happy to change stuff if need be.

@gventuri
Copy link
Collaborator

@adamingas merged, thanks a lot for the PR :D

@gventuri gventuri merged commit 74125bf into Sinaptik-AI:main May 14, 2024
3 of 9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants