forked from grafana/loki
-
Notifications
You must be signed in to change notification settings - Fork 0
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Parallel pipeline for selctor expr #2
Open
splitice
wants to merge
1,854
commits into
X4BNet:temp/work-3
Choose a base branch
from
liguozhong:parallel_pipeline_for_selctorExpr
base: temp/work-3
Could not load branches
Branch not found: {{ refName }}
Could not load tags
Nothing to show
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Parallel pipeline for selctor expr #2
splitice
wants to merge
1,854
commits into
X4BNet:temp/work-3
from
liguozhong:parallel_pipeline_for_selctorExpr
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
splitice
force-pushed
the
temp/work-3
branch
13 times, most recently
from
May 21, 2022 12:22
8812237
to
573fe20
Compare
… time range. (grafana#9096) **What this PR does / why we need it**: When computing index stats for a query, we sum up the stats from all the chunks the query would read, even if some chunks will only be read partially. Moreover, if the query spans across two different TSDB tables, and there are chunks duplicated on both tables (at the end of the first one and at the beginning of the next one), the chunks' stats will be counted twice. This PR improves how we compute stats by taking care of partially matching chunks. Assuming entries and bytes are evenly distributed in the chunk, we will take the proportional number of entries and the number of bytes that fit on the span. For example, a chunk spans from t1 to t10 and has 10 entries and 10 KBs. A query from t3 through t7 would hit 50% of the chunk time span. Therefore, we estimate that the query would load 5 KBs and 5 entries from the chunk. **Special notes for your reviewer**: **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [ ] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`
**What this PR does / why we need it**: Bumps our go dependency from 1.20.1 to 1.20.3 to address [security fixes](https://groups.google.com/g/golang-announce/c/Xdv6JL9ENs8) **Checklist** - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`
Currently, Promtail always polls files every 250ms. This PR allows the polling period to be customized, starting at a minimum polling period and exponentially increasing the polling frequency to a bounded maximum if the file has no changes. When file changes are detected, the polling frequency resets to the minimum. The default behavior is the existing behavior; both the minimum and maximum polling intervals are set to 250ms. I'm opening this in draft to get early feedback. There's a few open questions: * Where can I add validations to ensure that the min/max polling intervals are configured properly? * Is this the right approach for implementation? Do I need to make any changes? * Does this need tests? I'll update the CHANGELOG and Documentation after we're agreed on an approach.
This PR makes the `unpack` parser sanitize label keys like all the other parsers. When invalid label names are included, it causes the user to get `could not write JSON response` errors when the response is serialized to send back to the user.
**What this PR does / why we need it**: Add a new helm value `singleBinary.extraContainers` to allow running sidecar containers in a singleBinary loki deployment. Useful for a container that keeps the data PV from filling up. **Checklist** - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`
Loki uses a `scanner.Scanner` to consume the characters in a query. `lex.go` relies heavily on the assumption a scanner can be copied to perform speculative parsing. The problem is that sometimes speculative parsing overruns the scanners internal buffer of `1024` bytes and incurs an additional read to the underlying reader. At that point, all subsequent scanners are out of sync with their underlying readers and the query may fail to be parsed. In the case of grafana#9132, there is a function call at line `1024`. `lex.go` tries to understand this is a function call by speculatively parsing and incurs an underlying read which consumers the rest of the reader. When the lexer tries to scan the next character with the original scanner, the reader is already consumed even though only 1024 bytes have been scanned The scanner has an internal buffer size of `1024 bytes`. This PR copies `scanner.Scanner` from the `text/scanner` package and increases the internal buffer to `maxInputSize` which guarantees that no calls to `Scan` will incur reads beyond the first one. Ideally we wouldn't pass Scanners around in such a way but the cost of that is to rethink much of Loki's lexing logic. Given we know a max input size and we already have that in memory, this solution should suffice. Fixes grafana#9132
Signed-off-by: Ashwanth Goli <iamashwanth@gmail.com> **What this PR does / why we need it**: Currently loki initializes a single instance of index-shipper to [handle all the table ranges](https://github.com/grafana/loki/blob/ff7b46297345b215fbf49c2cd4c364d125b6290b/pkg/storage/factory.go#L188) (from across periods) for a given index type `boltdb-shipper, tsdb`. Since index-shipper only has the object client handle to the store defined by `shared_store_type`, it limits the index uploads to a single store. Setting `shared_store_type` to a different store at a later point in time would mean losing access to the indexes stored in the previously configured store. With this PR, we initialize a separate index-shipper & table manager for each period if `shared_store_type` is not explicity configured. This offers the flexibility to store index in multiple stores (across providers). **Note**: - usage of `shared_store_type` in this commit text refers to one of these config options depending on the index in use: `-boltdb.shipper.shared-store`, `-tsdb.shipper.shared-store` - `shared_store_type` used to default to the `object_store` from the latest `period_config` if not explicitly configured. This PR removes these defaults in favor of supporting index uploads to multiple stores. **Which issue(s) this PR fixes**: Fixes grafana#7276 **Special notes for your reviewer**: All the instances of downloads table manager operate on the same cacheDir. But it shouldn't be a problem as the tableRanges do not overlap across periods. **Checklist** - [X] Reviewed the `CONTRIBUTING.md` guide - [ ] Documentation added - [X] Tests updated - [x] `CHANGELOG.md` updated - [x] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` --------- Signed-off-by: Ashwanth Goli <iamashwanth@gmail.com> Co-authored-by: J Stickler <julie.stickler@grafana.com>
**What this PR does / why we need it**: The following middlewares in the query frontend uses a downstream engine: - `NewQuerySizeLimiterMiddleware` and `NewQuerierSizeLimiterMiddleware` - `NewQueryShardMiddleware` - `NewSplitByRangeMiddleware` These were all creating the downstream engine as follows: ```go logql.NewDownstreamEngine(logql.EngineOpts{LogExecutingQuery: false}, DownstreamHandler{next: next, limits: limits}, limits, logger), ``` As can be seen, the [engine options configured in Loki][1] were not being used at all. In the case of `NewQuerySizeLimiterMiddleware`, `NewQuerierSizeLimiterMiddleware` and `NewQueryShardMiddleware`, the downstream engine was created to get the `MaxLookBackPeriod`. When creating a new Downstream Engine as above, the `MaxLookBackPeriod` [would always be the default][2] (30 seconds). This PR fixes this by passing down the engine config to these middlewares, so this config is used to create the new downstream engines. **Which issue(s) this PR fixes**: Adresses some pending tasks from grafana#8670 (comment). **Special notes for your reviewer**: **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [ ] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` [1]: https://github.com/grafana/loki/blob/1bcf683513b5e11fb1dcc5625761b9760cfbd958/pkg/querier/querier.go#L52 [2]: https://github.com/grafana/loki/blob/edc6b0bff76714ff584c20d7ff0235461a4f4a88/pkg/logql/engine.go#L136-L140
* use `main` instead of `master` branch * fix outdated URLs
…ana#9137) **What this PR does / why we need it**: Some changelog entries were misplaced into _Promtail Enhancements_ instead of _Loki Enhancements_. This PR fixes this by moving the items to the correct section. **Which issue(s) this PR fixes**: Fixes grafana#9042 **Special notes for your reviewer**: **Checklist** - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [ ] Documentation added - [ ] Tests updated - [x] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`
Removed the single binary from the scalability section as this is confusing. This should be called monolithic also. **What this PR does / why we need it**: **Which issue(s) this PR fixes**: Fixes #<issue number> **Special notes for your reviewer**: **Checklist** - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`
Changed "integrations and connections" to "connections" and removed the lightning bolt icon reference. --------- Co-authored-by: J Stickler <julie.stickler@grafana.com>
**What this PR does / why we need it**: Adds some context to the scrape configs section in Promtail Config Docs to ensure clarity around the connection to Prometheus Here is a link to a slack thread where the customer was confused on this: https://raintank-corp.slack.com/archives/C04KY6NJSCE/p1681761335754199 --------- Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com> Co-authored-by: J Stickler <julie.stickler@grafana.com>
**What this PR does / why we need it**: Add a help target to the Makefile. Only the main targets were added. ``` % make help Usage: make <target> Targets: help Display this help and any documented user-facing targets. Other undocumented targets may be present in the Makefile. all build all executables (loki, logcli, promtail, loki-canary) logcli build logcli executable logcli-debug build debug logcli executable logcli-image build logcli docker image loki build loki executable loki-debug build loki debug executable loki-canary build loki-canary executable helm-test run helm tests helm-lint run helm linter loki-querytee build loki-querytee executable promtail build promtail executable promtail-debug build debug promtail executable loki-mixin compile the loki mixin loki-mixin-check check the loki mixin is up to date lint run linters test run the unit tests clean clean the generated files docker-driver build the docker-driver executable fluent-bit-plugin build the fluent-bit plugin fluent-bit-image build the fluent-bit plugin docker image fluent-bit-push push the fluent-bit plugin docker image fluentd-plugin build the fluentd plugin fluentd-plugin-push push the fluentd plugin fluentd-image build the fluentd docker image fluentd-image-push push the fluentd docker image logstash-image build the logstash image logstash-push push the logstash image promtail-image build the promtail docker image promtail-debug-image build the promtail debug docker image loki-image build the loki docker image loki-debug-image build the debug loki docker image loki-canary-image build the loki canary docker image helm-test-image build the helm test image helm-test-push push the helm test image logql-analyzer-image build the LogQL Analyzer image logql-analyzer-push push the LogQL Analyzer image build-image build the docker build image build-image-push push the docker build image doc Generates the config file documentation check-doc Check the documentation files are up to date ``` **Checklist** - [X] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](grafana@d10549e) Signed-off-by: Michel Hollands <michel.hollands@grafana.com>
…#9509) **What this PR does / why we need it**: Allows annotating loki components with custom annotations. For instance: ```yaml loki: annotations: secret.reloader.stakater.com/reload: loki-aws-credentials ``` **Which issue(s) this PR fixes**: No issue **Special notes for your reviewer**: **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - Helm chart section is empty. - [x] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [x] N/A Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>
**What this PR does / why we need it**: Adds `loki.configStorageType` & `loki.externalConfigSecretName` values and associated helpers and templates to support choosing either `ConfigMap` or `Secret` as config medium types. **Which issue(s) this PR fixes**: Fixes grafana#8268 **Special notes for your reviewer**: **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [x] Documentation added - [ ] Tests updated - [x] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` --------- Signed-off-by: garcia.ryan <garcia.ryan@solute.us> Co-authored-by: Stefan Büringer <4662360+sbueringer@users.noreply.github.com> Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com>
**What this PR does / why we need it**: Currently the gateway only listens on IPv4. On IPv6-only clusters, this causes the readiness probe to fail, rendering the gateway unusable. **Which issue(s) this PR fixes**: No issue created **Special notes for your reviewer**: **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [ ] Documentation added - [ ] Tests updated - [x] `CHANGELOG.md` updated - [x] N/A Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>
**What this PR does / why we need it**: We saw a performance decrease when running the merge iterator with a very small pool of ingesters. The reason was that we would allocate a new buffer with each next call. **Special notes for your reviewer**: `main` is missing `merge_sort_dedupe-16` so I ran it once with a small patch to generate the data. ```goos: linux goarch: amd64 pkg: github.com/grafana/loki/pkg/iter cpu: AMD Ryzen 7 3700X 8-Core Processor │ mergeiter-old.txt │ mergeiter-new.txt │ │ sec/op │ sec/op vs base │ SortIterator/merge_sort-16 1.350m ± 4% 1.368m ± 4% ~ (p=0.579 n=10) SortIterator/merge_sort_dedupe-16 2.566m ± 4% 2.462m ± 5% ~ (p=0.105 n=10) SortIterator/sort-16 1.034m ± 6% 1.029m ± 6% ~ (p=0.684 n=10) SortSampleIterator/merge-16 1.877m ± 2% 1.857m ± 2% ~ (p=0.075 n=10) SortSampleIterator/sort-16 432.7µ ± 6% 436.0µ ± 4% ~ (p=0.579 n=10) geomean 1.238m 1.229m -0.72% │ mergeiter-old.txt │ mergeiter-new.txt │ │ B/op │ B/op vs base │ SortIterator/merge_sort-16 21.22Ki ± 0% 21.22Ki ± 0% ~ (p=1.000 n=10) SortIterator/merge_sort_dedupe-16 41.47Ki ± 0% 41.47Ki ± 0% ~ (p=1.000 n=10) ¹ SortIterator/sort-16 13.94Ki ± 0% 13.94Ki ± 0% ~ (p=1.000 n=10) ¹ geomean 23.06Ki 23.06Ki +0.00% ¹ all samples are equal │ mergeiter-old.txt │ mergeiter-new.txt │ │ allocs/op │ allocs/op vs base │ SortIterator/merge_sort-16 7.000 ± 0% 7.000 ± 0% ~ (p=1.000 n=10) ¹ SortIterator/merge_sort_dedupe-16 7.000 ± 0% 7.000 ± 0% ~ (p=1.000 n=10) ¹ SortIterator/sort-16 5.000 ± 0% 5.000 ± 0% ~ (p=1.000 n=10) ¹ geomean 6.257 6.257 +0.00% ¹ all samples are equal ``` **Checklist** - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](grafana@d10549e)
**What this PR does / why we need it**: Fix the link to the API docs. **Special notes for your reviewer**: **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [x] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](grafana@d10549e) Co-authored-by: Irapuan Bottosso <irapuan@Irapuans-MacBook-Pro-2.local>
…9497) **What this PR does / why we need it**: ### Add support for AWS CloudTrail audit logs ingestion using lambda-promtail Calls to AWS APIs are logged in AWS Cloudtrail and are helpful for security and debugging purposes. However, I've experienced difficulties with it: + The AWS CloudTrail service is not well integrated with Prometheus (no metrics, no alerts) and I don't want to manage alerts in CloudWatch Alerts + The search experience is painful with CloudTrail via the AWS Console (I will not elaborate 😅). This PR allows ingesting CloudTrail audit logs sent to an S3 bucket using the same approach as VPC flow logs or Load Balancer logs. **Special notes for your reviewer**: + Because the Cloudtrail file format is not text but json, we stream the json CloudTrail records instead of using the already existing scanner. + Because the Cloudtrail filename format is not the same as for the Flow log or the Load balancer log files, we need to split the regexes by service (although many AWS services seem to share the same `defaultFilenameRegex`). + In the `getLabels` function, we expect the `type` parameter to be found in the filename using the Regex. For some log files (ex: Cloudfront log files). The file name has no reference to the service name. This is why, as a default, when no type is found in the name of the file, I set it to use the key of the matching Regex expression. **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [x] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`
**What this PR does / why we need it**: **Which issue(s) this PR fixes**: Fixes grafana#9547 **Special notes for your reviewer**: **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [x] Documentation added - [ ] Tests updated - [x] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [x] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](grafana@d10549e) Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>
…9546) **What this PR does / why we need it**: This PR fix name to config checksum annotation **Which issue(s) this PR fixes**: Fixes grafana#9545 **Special notes for your reviewer**: **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [x] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [x] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](grafana@d10549e)
**What this PR does / why we need it**: Call functions like `FromStrings`, so we don't have to know how the data structure is laid out. `FromStrings` sorts labels, so we don't have to call `Sort()`. **Special notes for your reviewer**: This PR only touches tests. Upstream Prometheus is using a different implementation of `labels.Labels`, so I plan to create a series of PRs which make Loki compatible with that. **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - NA Documentation added - [x] Tests updated - NA `CHANGELOG.md` updated - NA Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - NA For Helm chart changes bump the Helm chart version. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
**What this PR does / why we need it**: Call functions like `FromStrings`, so we don't have to know how the data structure is laid out. **Special notes for your reviewer**: This PR only touches tests. Upstream Prometheus is using a different implementation of `labels.Labels`, so I plan to create a series of PRs which make Loki compatible with that. grafana#9532 was the first. **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - NA Documentation added - [x] Tests updated - NA `CHANGELOG.md` updated - NA Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - NA For Helm chart changes bump the Helm chart version. --------- Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
**What this PR does / why we need it**: Introduce a new configuration named `limited_log_push_errors` that will be implemented. The configuration is being introduced on a separated PR to facilitate future rollbacks. **Which issue(s) this PR fixes**: N/A
**What this PR does / why we need it**: There were 2 commits for the helm chart with the same version number. In this case the build scripts only releases 1 new helm chart, containing the first commit. This PR adds a new version so a helm chart will be created that contains the later PR. **Checklist** - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [X] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](grafana@d10549e) Signed-off-by: Michel Hollands <michel.hollands@grafana.com>
When marshalling to yaml or json, expect the value not a pointer. Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>
…afana#9586) Signed-off-by: Philip Gough <philip.p.gough@gmail.com>
grafana#9584) To optimize regex line and label filters, Loki rewrites them to be non-greedy. This can cause extra characters to appear in the regex when `String()` is called. In addition to confusing logging, this can also cause internal panics when a previously-parsable query becomes too long. This PR stores the original regex in the label/line filter and uses that in calls to `String`. This PR also removes regex optimizations from label selectors. Making the optimization at parse time caused the original regex to be lost.
) **What this PR does / why we need it**: Adds an initial implementation of `limited_log_push_errors`. This initial implementation encompass: - Runtime per-tenant configuration - Simple per-tenant rate-limiting based on error message size Notable features that will be added in future phases: - Instead of a single error with the final string, give to the manager the list of all entries - Once it supports per-entry error, make the rate-limiting separated per-reason - Hash the entry error and avoid repeating errors by caching seen errors in memory
Hello! I have small change suggestion. I think its too small to add documentation/changelog note, but if I should then just say and I can add. :) **What this PR does / why we need it**: it adds possibility to set labels for Loki gateway Ingress. **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [x] Documentation added - [ ] Tests updated - [x] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`
# Conflicts: # docs/sources/configuration/_index.md # pkg/distributor/distributor.go # pkg/distributor/http.go # pkg/ingester/ingester.go # pkg/ingester/instance.go # pkg/storage/chunk/client/aws/s3_storage_client.go # pkg/storage/chunk/client/object_client.go # pkg/storage/store.go # pkg/storage/stores/series/series_index_store.go
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What this PR does / why we need it:
Which issue(s) this PR fixes:
Fixes #
Special notes for your reviewer:
Checklist
CHANGELOG.md
.docs/sources/upgrading/_index.md