Memory Issues

[suchoss]

• U Kindex objektů opakujeme stejné texty => Memory waste (KoncentraceDodavateluCalculator):

• Úřad práce - KIndex memory cache - "KIndexData72496991_False" - 117 MB paměti! Ten objekt je dost velký na memory cache (a takových je víc). Potřebujeme mít v té paměti jednotlivá Iča a k nim poznámky? Tady by to chtělo ideálně předělat KIndex objekty...

• KIndex jako takový má na starosti v paměti cca 90 % zabraného místa

U Kindexu bych navrhoval uložit data do databáze a zbavit se memory cache. Nepřepočítáváme to tak často a jsou to pevné objekty po letech.

[suchoss]

"Unhadled exception >500 "OnHTTPErrorMidddleware" "/Error/500"?"?q=osobaid%3Aandrej-babis"\n Ex Type: Enyim.Caching.Memcached.MemcachedClientException
LogMessage: Empty response received.
In Method: System.String ReadResponse(Enyim.Caching.Memcached.PooledSocket)
Stack Trace:
at Enyim.Caching.Memcached.Protocol.Text.TextSocketHelper.ReadResponse(PooledSocket socket)
at Enyim.Caching.Memcached.Protocol.Text.GetHelper.ReadItem(PooledSocket socket)
at Enyim.Caching.Memcached.Protocol.Text.GetOperation.ReadResponse(PooledSocket socket)
at Enyim.Caching.Memcached.MemcachedNode.ExecuteOperation(IOperation op)
at Enyim.Caching.MemcachedClient.PerformTryGet(String key, UInt64& cas, Object& value)
at Enyim.Caching.MemcachedClient.TryGet(String key, Object& value)
at Devmasters.Cache.Memcached.MemcachedCacheProvider1.Exists(String key) at Devmasters.Cache.BaseCache1.Get()
at Devmasters.Cache.ManagerBase3.Get(Key key, TimeSpan itemSpecificExpiration) at Devmasters.Cache.ManagerBase3.Get(Key key)
at HlidacStatu.Repositories.Osoby.OsobyMCMByNameId.Get(String key) in P:\CCD\hs.team\Hlidac-Statu\Repositories\Temp\Osoby.cs:line 129
at HlidacStatu.Repositories.Searching.Rules.OsobaId.processQueryPart(Part part) in P:\CCD\hs.team\Hlidac-Statu\Repositories\Searching\Rules\OsobaId.cs:line 36
at HlidacStatu.Repositories.Searching.Rules.RuleBase.Process(Part queryPart) in P:\CCD\hs.team\Hlidac-Statu\Repositories\Searching\Rules\RuleBase.cs:line 43
at HlidacStatu.Repositories.Searching.SimpleQueryCreator.GetSimpleQuery(SplittingQuery sq, IRule[] rules) in P:\CCD\hs.team\Hlidac-Statu\Repositories\Searching\SimpleQueryCreator.cs:line 20
at HlidacStatu.Repositories.Searching.SimpleQueryCreator.GetSimpleQuery[T](String query, IRule[] rules) in P:\CCD\hs.team\Hlidac-Statu\Repositories\Searching\SimpleQueryCreator.cs:line 89
at HlidacStatu.Repositories.SmlouvaRepo.Searching.SimpleSearchAsync(String query, Int32 page, Int32 pageSize, OrderResult order, AggregationContainerDescriptor1 anyAggregation, Nullable1 platnyZaznam, Boolean includeNeplatne, Boolean logError, Boolean fixQuery, Boolean withHighlighting, Boolean exactNumOfResults, CancellationToken cancellationToken) in P:\CCD\hs.team\Hlidac-Statu\Repositories\SmlouvaRepo.Search.cs:line 236
at HlidacStatu.Repositories.ES.QueryGrouped.SmlouvyPerYearAsync(String query, Int32[] interestedInYearsOnly) in P:\CCD\hs.team\Hlidac-Statu\Repositories\Temp\QueryGrouped.cs:line 112
at HlidacStatu.Repositories.Statistics.SmlouvyStatistics.CalculateAsync(String query) in P:\CCD\hs.team\Hlidac-Statu\Repositories\Statistics\SmlouvyStatistics.cs:line 34
at AspNetCoreGeneratedDocument.Views_Home_Partials__analyzaPage.ExecuteAsync() in P:\CCD\hs.team\Hlidac-Statu\Web\Views\Home\Partials\_analyzaPage.cshtml:line 18
at Microsoft.AspNetCore.Mvc.Razor.RazorView.RenderPageCoreAsync(IRazorPage page, ViewContext context)
at Microsoft.AspNetCore.Mvc.Razor.RazorView.RenderPageAsync(IRazorPage page, ViewContext context, Boolean invokeViewStarts)
at Microsoft.AspNetCore.Mvc.Razor.RazorView.RenderAsync(ViewContext context)
at Microsoft.AspNetCore.Mvc.TagHelpers.PartialTagHelper.RenderPartialViewAsync(TextWriter writer, Object model, IView view)
at Microsoft.AspNetCore.Mvc.TagHelpers.PartialTagHelper.ProcessAsync(TagHelperContext context, TagHelperOutput output)
at Microsoft.AspNetCore.Razor.Runtime.TagHelpers.TagHelperRunner.g__Awaited|0_0(Task task, TagHelperExecutionContext executionContext, Int32 i, Int32 count)
at AspNetCoreGeneratedDocument.Views_Home_Analyza.b__17_2() in P:\CCD\hs.team\Hlidac-Statu\Web\Views\Home\Analyza.cshtml:line 22
at Microsoft.AspNetCore.Razor.Runtime.TagHelpers.TagHelperExecutionContext.GetChildContentAsync(Boolean useCachedResult, HtmlEncoder encoder)
at Microsoft.AspNetCore.Mvc.TagHelpers.CacheTagHelper.ProcessContentAsync(TagHelperOutput output)
at Microsoft.AspNetCore.Mvc.TagHelpers.CacheTagHelper.CreateCacheEntry(CacheTagKey cacheKey, TagHelperOutput output)
at Microsoft.AspNetCore.Mvc.TagHelpers.CacheTagHelper.ProcessAsync(TagHelperContext context, TagHelperOutput output)
at Microsoft.AspNetCore.Razor.Runtime.TagHelpers.TagHelperRunner.g__Awaited|0_0(Task task, TagHelperExecutionContext executionContext, Int32 i, Int32 count)
at AspNetCoreGeneratedDocument.Views_Home_Analyza.ExecuteAsync() in P:\CCD\hs.team\Hlidac-Statu\Web\Views\Home\Analyza.cshtml:line 21
at Microsoft.AspNetCore.Mvc.Razor.RazorView.RenderPageCoreAsync(IRazorPage page, ViewContext context)
at Microsoft.AspNetCore.Mvc.Razor.RazorView.RenderPageAsync(IRazorPage page, ViewContext context, Boolean invokeViewStarts)
at Microsoft.AspNetCore.Mvc.Razor.RazorView.RenderAsync(ViewContext context)
at Microsoft.AspNetCore.Mvc.ViewFeatures.ViewExecutor.ExecuteAsync(ViewContext viewContext, String contentType, Nullable1 statusCode) at Microsoft.AspNetCore.Mvc.ViewFeatures.ViewExecutor.ExecuteAsync(ViewContext viewContext, String contentType, Nullable1 statusCode)
at Microsoft.AspNetCore.Mvc.ViewFeatures.ViewExecutor.ExecuteAsync(ActionContext actionContext, IView view, ViewDataDictionary viewData, ITempDataDictionary tempData, String contentType, Nullable`1 statusCode)
at Microsoft.AspNetCore.Mvc.ViewFeatures.ViewResultExecutor.ExecuteAsync(ActionContext context, ViewResult result)
at Microsoft.AspNetCore.Mvc.ViewResult.ExecuteResultAsync(ActionContext context)
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.g__Awaited|30_0[TFilter,TFilterAsync](ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Rethrow(ResultExecutedContextSealed context)
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.ResultNext[TFilter,TFilterAsync](State& next, Scope& scope, Object& state, Boolean& isCompleted)
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.g__Awaited|28_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.g__Awaited|25_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Rethrow(ResourceExecutedContextSealed context)
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.g__Awaited|20_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
at Microsoft.AspNetCore.Routing.EndpointMiddleware.g__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger)
at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context)
at HlidacStatu.LibCore.MiddleWares.ApiAuthenticationMiddleware.Invoke(HttpContext context) in P:\CCD\hs.team\Hlidac-Statu\LibCore\MiddleWares\ApiAuthenticationMiddleware.cs:line 74
at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.Invoke(HttpContext context)
at Microsoft.AspNetCore.ResponseCaching.ResponseCachingMiddleware.Invoke(HttpContext httpContext)
at HlidacStatu.Web.Startup.<>c.<b__8_1>d.MoveNext() in P:\CCD\hs.team\Hlidac-Statu\Web\Startup.cs:line 268
--- End of stack trace from previous location ---

[suchoss]

Častý garbage collecting, který jde z infofactů:

Tady mi přijde, že na S3 cache dochází k velké deserializaci (škoda, že nejde načíst hotové objekty z cache) - napadá mě možná optimalizace => přechod z newtonsoft json na dotnet json. Snížil by se nejspíš pressure na GC.

[suchoss]

VZ z produkce - hit je docela velký