You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
A lot of money and performance is waste answering the same questions again and again
Proposed solution
Be able to save all the queries together with llm responses in some database and try to fetch it first from there, if not then call the LLM, and have some parameters to invalidate or update cache from time to time.
Importance
would be great to have
The text was updated successfully, but these errors were encountered:
Context / Scenario
That's a feature that already exists in langchain and will be beneficial to save costs. The idea will be to ported from phyton to c#
https://github.com/zilliztech/GPTCache
https://python.langchain.com/docs/integrations/llms/llm_caching/
https://www.mongodb.com/developer/products/atlas/advanced-rag-langchain-mongodb/
The problem
A lot of money and performance is waste answering the same questions again and again
Proposed solution
Be able to save all the queries together with llm responses in some database and try to fetch it first from there, if not then call the LLM, and have some parameters to invalidate or update cache from time to time.
Importance
would be great to have
The text was updated successfully, but these errors were encountered: