You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I am looking at running Kernel Memory as a service in Asp.Net API.
Question
I took a look at the Service.AspNetCore code and saw that IKernelMemory is registered as a singleton. With an API i was wondering why we should register IKernelMemory as a singleton instead of as a scoped service?
The text was updated successfully, but these errors were encountered:
The recommended deployment uses asynchronous queues to execute tasks in the background, to avoid blocking user requests and to deal with long running jobs.
For example, when uploading a file it might take seconds/minutes to process, depending on the pipeline in use, depending on external services bandwidth, and in case of transient errors requiring the service to retry.
To avoid blocking TCP/HTTP requests while documents are processed, the ingestion is designed to run as a task outside the scope of a HTTP request. Using a singleton allows KM components to work. It's also perfectly safe to use KM as a singleton.
On the other hand, if for some reason you cannot use singletons, then I would recommend configuring KM in Serverless mode. Everything will run synchronously, blocking HTTP requests while ingestion is running, and once a request is complete the KM instance can be disposed.
Context / Scenario
I am looking at running Kernel Memory as a service in Asp.Net API.
Question
I took a look at the Service.AspNetCore code and saw that IKernelMemory is registered as a singleton. With an API i was wondering why we should register IKernelMemory as a singleton instead of as a scoped service?
The text was updated successfully, but these errors were encountered: