[Roadmap] Full Support for OPENAPI Specs to Integrate More Tools

[yiyiyi0817]

Required prerequisites

• I have searched the Issue Tracker and Discussions that this hasn't already been reported. (+1 or comment there if it has.)
• Consider asking first in a Discussion.

Motivation

Using the openapi specs integration tools allows CAMEL developers to develop and integrate a large number of tools more quickly.

Solution

1. Support All Kinds of Sepcs Files or Urls

• Support Swagger/OpenAPI 2.0 and 3.0 (.yaml) without the need for security authentication Supports automatic calling of APIs conforming to the OpenAPI specification. #548
• Support Swagger/OpenAPI 2.0 and 3.0 (.json) without the need for security authentication
• Support Swagger/OpenAPI 2.0 and 3.0 (URLs) without the need for security authentication

Reference: https://github.com/RonnyPfannschmidt/prance

2. Support All Kinds of Security Scheme

• Support Security Scheme: apiKey
• Support Security Scheme: http
• Support Security Scheme: oauth2
• Support Security Scheme: openIdConnect

Reference: https://swagger.io/specification/#security-scheme-object

I think we can start by integrating apiKey, which is a more common security authentication method. It may not be necessary to integrate all of the above methods, and sometimes just one of them will succeed in ping API.

3. Add more APIs

• Collect more OPENAPI specs files or URLs of different APIs
  • klarna
  • speak
  • coursera
  • More... #580

Reference: xlang-ai/OpenAgents#132

According to the authors of OpenAgentshttps://github.com/xlang-ai/OpenAgents. while API Hubs like Rapid have integrated a large number of traditional APIs with openapi specs, their experiments revealed:

"Initially, we tried to import some tools from Rapid API to expand our collection, but we found (I'm not sure if any research is currently addressing this) that the input parameters and returned information from these traditional APIs are overly complex and structured data. The current models struggle with planning and accurately parsing natural language into such inputs, and understanding the responses is also limited, especially with noisy data. Not to mention the API providers are generally not willing to provide APIs to developers instead of companies. We cannot afford the efforts on wrapping on these APIs."

Therefore, their approach was

"So, we actually reverse-engineered the functionality of OpenAI Plugins, obtained the endpoints and openapi. yaml from the providers offering Plugins API to them, and then automated pings to see if they were free or if some required additional verification or had IP restrictions. Then, we retrieved the APIs that were accessible. At that time, we were able to retrieve over a hundred APIs using this method."

I think perhaps we could consider their approach, even though OpenAI's plugin functionality has now become the GPT Store.

Alternatives

No response

Additional context

This issue may not apply to APIs that do not have openapi specs, as well as APIs that return overly complex and structured data.