Prompt exceeding OpenAI's rate limit

[webermatias]

I asked Skyvern to perform a search in google and do an exploration. One of the results was a YouTube page with many comments. In this step, it hit the rate limit several times, resulting in OpenAI marking the key as unavailable.

2024-03-10T15:25:54.087388_a_233651180269098532_llm_request.json
2024-03-10T15:25:54.039585_a_233651180269098518_llm_prompt.txt

The errors:

OpenAI rate limit exceeded, marking key as unavailable. error_code=rate_limit_exceeded error_message=Error code: 429 - {'error': {'message': 'Request too large for gpt-4-vision-preview in organization org-KvD1ZUhg9B7cNvz3mxtyAZBX on tokens per min (TPM): Limit 40000, Requested 53355. The input or output tokens must be reduced in order to run successfully. Visit https://platform.openai.com/account/rate-limits to learn more.', 'type': 'tokens', 'param': None, 'code': 'rate_limit_exceeded'}}

openai.RateLimitError: Error code: 429 - {'error': {'message': 'Request too large for gpt-4-vision-preview in organization org-KvD1ZUhg9B7cNvz3mxtyAZBX on tokens per min (TPM): Limit 40000, Requested 53355. The input or output tokens must be reduced in order to run successfully. Visit https://platform.openai.com/account/rate-limits to learn more.', 'type': 'tokens', 'param': None, 'code': 'rate_limit_exceeded'}}

[suchintan]

It's very likely that this is happening because every url within the comments is interactable, and we generate 800 tokens tokens per comment 😱

{'id': 392, 'tagName': 'a', 'attributes': {'href': '/@TimKitchens7'}, 'children': [{'id': 393, 'tagName': 'img', 'attributes': {'alt': '', 'src': 'https://yt3.ggpht.com/e1atGPiP24JyimquGvvBoeKxiclrfjatbAIMMepSfSYEiX8fzpF-0bvYQ843xvW96Rbac1T21g=s88-c-k-c0x00ffffff-no-rj'}}]}, {'id': 394, 'tagName': 'a', 'attributes': {'href': '/@TimKitchens7'}, 'text': '@TimKitchens7', 'children': [{'id': 395, 'tagName': 'span', 'text': '@TimKitchens7'}]}, {'id': 396, 'tagName': 'a', 'attributes': {'href': '/watch?v=Jl6BuoXcZPE&lc=UgxAY3LFLkZDxT4l_ZB4AaABAg'}, 'text': '10 days ago'}, {'id': 397, 'tagName': 'a', 'attributes': {'aria-label': 'Like this comment along with 0 other people', 'title': '', 'href': 'https://accounts.google.com/ServiceLogin?service=youtube&uilel=3&passive=true&continue=https%3A%2F%2Fwww.youtube.com%2Fsignin%3Faction_handle_signin%3Dtrue%26app%3Ddesktop%26hl%3Den%26next%3D%252Fwatch%253Fv%253DJl6BuoXcZPE&hl=en'}, 'children': [{'id': 398, 'tagName': 'div', 'children': [{'id': 399, 'tagName': 'div'}]}, {'id': 400, 'tagName': 'div', 'children': [{'id': 401, 'tagName': 'div'}, {'id': 402, 'tagName': 'div'}]}]}, {'id': 403, 'tagName': 'a', 'attributes': {'aria-label': '', 'title': '', 'href': 'https://accounts.google.com/ServiceLogin?service=youtube&uilel=3&passive=true&continue=https%3A%2F%2Fwww.youtube.com%2Fsignin%3Faction_handle_signin%3Dtrue%26app%3Ddesktop%26hl%3Den%26next%3D%252Fwatch%253Fv%253DJl6BuoXcZPE&hl=en'}, 'children': [{'id': 404, 'tagName': 'div', 'children': [{'id': 405, 'tagName': 'div'}]}, {'id': 406, 'tagName': 'div', 'children': [{'id': 407, 'tagName': 'div'}, {'id': 408, 'tagName': 'div'}]}]}, {'id': 409, 'tagName': 'a', 'attributes': {'aria-label': 'Reply', 'title': '', 'href': 'https://accounts.google.com/ServiceLogin?service=youtube&uilel=3&passive=true&continue=https%3A%2F%2Fwww.youtube.com%2Fsignin%3Faction_handle_signin%3Dtrue%26app%3Ddesktop%26hl%3Den%26next%3D%252Fwatch%253Fv%253DJl6BuoXcZPE&hl=en'}, 'text': 'Reply', 'children': [{'id': 410, 'tagName': 'div', 'text': 'Reply', 'children': [{'id': 411, 'tagName': 'span', 'text': 'Reply'}]}, {'id': 412, 'tagName': 'div', 'children': [{'id': 413, 'tagName': 'div'}, {'id': 414, 'tagName': 'div'}]}]}, {'id': 415, 'tagName': 'a', 'attributes': {'href': '/@ahassan7270'}

https://platform.openai.com/tokenizer

[theRealWardo]

I bumped into this one as well. For sites with a large number of links, the project really just doesn't work due to number of tokens required and even for sites with a moderate number of links it can be pretty expensive to run due to this prompting technique.

When some improvements are made here, I'll try the project again. Until then, I can't really automate my workflows with this project.

[suchintan]

@theRealWardo we will investigate and get back to you when it's fixed!

[github-actions]

This issue is stale because it has been open for 30 days with no activity.