Backport 7c4882a from #11443

---

PR #9031 from @Alex3k has gone quite stale as the topic has been revised
since it was submitted.

This PR attempts to get at least part of Alex's update into the
documentation so that we can close his original PR.

Co-authored-by: J Stickler <julie.stickler@grafana.com>

**What this PR does / why we need it**:
 
Manual backport of #11581

…g tenant configs (#11603)

Backport cd3cf62 from #11601

Backport 3cb2a6b from #11615

---

Fix incorrect example

**What this PR does / why we need it**:
Fix incorrect example in [Template
functions](https://grafana.com/docs/loki/latest/query/template_functions/)

**Special notes for your reviewer**:

**Checklist**
- [x] Reviewed the
[`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md)
guide (**required**)
- [x] Documentation added
- [ ] Tests updated
- [ ] `CHANGELOG.md` updated
- [ ] If the change is worth mentioning in the release notes, add
`add-to-release-notes` label
- [ ] Changes that require user attention or interaction to upgrade are
documented in `docs/sources/setup/upgrade/_index.md`
- [ ] For Helm chart changes bump the Helm chart version in
`production/helm/loki/Chart.yaml` and update
`production/helm/loki/CHANGELOG.md` and
`production/helm/loki/README.md`. [Example
PR](d10549e)
- [ ] If the change is deprecating or removing a configuration option,
update the `deprecated-config.yaml` and `deleted-config.yaml` files
respectively in the `tools/deprecated-config-checker` directory.
[Example
PR](0d4416a)

Co-authored-by: Oğuzcan Küçükbayrak <o.kucukbayrak@icloud.com>

Backport 37a2f73 from #11548

---

**What this PR does / why we need it**:

doc is confusing/lacking detail on setting up regular fluentbit
container and we needed an example config for fluent-bit that works with
HG Loki.

**Which issue(s) this PR fixes**:
Fixes #<issue number>

**Special notes for your reviewer**:

**Checklist**
- [x] Reviewed the
[`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md)
guide (**required**)
- [x] Documentation added
- [ ] Tests updated
- [ ] `CHANGELOG.md` updated
- [ ] If the change is worth mentioning in the release notes, add
`add-to-release-notes` label
- [ ] Changes that require user attention or interaction to upgrade are
documented in `docs/sources/setup/upgrade/_index.md`
- [ ] For Helm chart changes bump the Helm chart version in
`production/helm/loki/Chart.yaml` and update
`production/helm/loki/CHANGELOG.md` and
`production/helm/loki/README.md`. [Example
PR](d10549e)
- [ ] If the change is deprecating or removing a configuration option,
update the `deprecated-config.yaml` and `deleted-config.yaml` files
respectively in the `tools/deprecated-config-checker` directory.
[Example
PR](0d4416a)

Co-authored-by: deresolution20 <85902399+deresolution20@users.noreply.github.com>

Backport 88aaa7d from #11654

---

**What this PR does / why we need it**:
The size check was not being performed atomically, which led to
flakiness in the `TestBackgroundSizeLimit` test.

---------

Co-authored-by: Danny Kopping <danny.kopping@grafana.com>

Backport 3b2278d from #11626

---

**What this PR does / why we need it**:
It improves the docs by addressing #11575 allowing users to use the
official fluent-bit helm chart over the deprecated grafana one

**Which issue(s) this PR fixes**:
Fixes #11575 

**Special notes for your reviewer**:
@JStickler wanted to take a look at this.

**Checklist**
- [ x ] Reviewed the
[`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md)
guide (**required**)
- [ x ] Documentation added
- [ ] Tests updated
- [ ] `CHANGELOG.md` updated
- [ ] If the change is worth mentioning in the release notes, add
`add-to-release-notes` label
- [ ] Changes that require user attention or interaction to upgrade are
documented in `docs/sources/setup/upgrade/_index.md`
- [ ] For Helm chart changes bump the Helm chart version in
`production/helm/loki/Chart.yaml` and update
`production/helm/loki/CHANGELOG.md` and
`production/helm/loki/README.md`. [Example
PR](d10549e)
- [ ] If the change is deprecating or removing a configuration option,
update the `deprecated-config.yaml` and `deleted-config.yaml` files
respectively in the `tools/deprecated-config-checker` directory.
[Example
PR](0d4416a)

Co-authored-by: Peter Stolz <50801264+PeterStolz@users.noreply.github.com>

Backport edba360 from #11670

Co-authored-by: Alexey Solodkiy <alexey@solodkiy.by>

…traction (#11667)

Backport 9759c13 from #11587

---

**What this PR does / why we need it**:
Fix label parsing logic so metric and log queries both only extract the first instance of a label that is requested multiple times.

**Which issue(s) this PR fixes**:
Fixes #11647

---------

Co-authored-by: Trevor Whitney <trevorjwhitney@gmail.com>

…on the request (#11659)

Backport e915efc from #11657

---

**What this PR does / why we need it**:
Log results cache when handling a hit composes an empty response based
on the cached request.
But the limit or direction fields in the cached request need not match
with the current request being served.

This causes the log results cache to return a response with incorrect
limit. This incorrect limit could then get applied when merging
responses upstream (split by interval mw for ex.)

This pr fixes this by composing the response based on the request being
served.

I also thought about updating the cache key to include both limit and
direction to have a clear separation, but I left it as is for the
following reason: if a time range contains no log lines, that result
would not change irrespective of a different limit or direction

**Which issue(s) this PR fixes**:
Fixes #<issue number>

**Special notes for your reviewer**:

**Checklist**
- [x] Reviewed the
[`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md)
guide (**required**)
- [ ] Documentation added
- [x] Tests updated
- [x] `CHANGELOG.md` updated
- [ ] If the change is worth mentioning in the release notes, add
`add-to-release-notes` label
- [ ] Changes that require user attention or interaction to upgrade are
documented in `docs/sources/setup/upgrade/_index.md`
- [ ] For Helm chart changes bump the Helm chart version in
`production/helm/loki/Chart.yaml` and update
`production/helm/loki/CHANGELOG.md` and
`production/helm/loki/README.md`. [Example
PR](d10549e)
- [ ] If the change is deprecating or removing a configuration option,
update the `deprecated-config.yaml` and `deleted-config.yaml` files
respectively in the `tools/deprecated-config-checker` directory.
[Example
PR](0d4416a)

---------

Co-authored-by: Ashwanth <iamashwanth@gmail.com>
Co-authored-by: Poyzan <31743851+poyzannur@users.noreply.github.com>

…ackaging. (#11676)

Backport 86f2001 from #11511

---

**What this PR does / why we need it**:
Related issue: #11398

This minimal config scrape only single file thus not overloading the
systems as described in the issue

**Which issue(s) this PR fixes**:
Fixes #<issue number>

**Special notes for your reviewer**:

**Checklist**
- [x] Reviewed the
[`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md)
guide (**required**)
- [x] Documentation added
- [ ] Tests updated
- [ ] `CHANGELOG.md` updated
- [ ] If the change is worth mentioning in the release notes, add
`add-to-release-notes` label
- [ ] Changes that require user attention or interaction to upgrade are
documented in `docs/sources/setup/upgrade/_index.md`
- [ ] For Helm chart changes bump the Helm chart version in
`production/helm/loki/Chart.yaml` and update
`production/helm/loki/CHANGELOG.md` and
`production/helm/loki/README.md`. [Example
PR](d10549e)
- [ ] If the change is deprecating or removing a configuration option,
update the `deprecated-config.yaml` and `deleted-config.yaml` files
respectively in the `tools/deprecated-config-checker` directory.
[Example
PR](0d4416a)

Co-authored-by: Kaviraj Kanagaraj <kavirajkanagaraj@gmail.com>
Co-authored-by: Poyzan <31743851+poyzannur@users.noreply.github.com>

#11714)

Backport 61a4205 from #11612

---

Expands on #11601

**What this PR does / why we need it**:
Turns out the previous tests didn't expose all possible causes for data
races (another one occurs at
https://github.com/grafana/loki/blob/5a55158cc751465846383bc758aa0c169363b292/pkg/ruler/registry.go#L204).
Moving the mutex to the calling function adds more safety.

**Which issue(s) this PR fixes**:
Fixes #11569

Co-authored-by: Danny Kopping <danny.kopping@grafana.com>

**What this PR does / why we need it**:
Updates the `golang.org/x/crypto` dependency to its latest version
`v0.18.0`

**Which issue(s) this PR fixes**:
Resolves
[CVE-2023-48795](GHSA-45x7-px36-x8w8)
Medium golang.org/x/crypto Go
grafana/support-escalations#8977
**Special notes for your reviewer**:

**Checklist**
- [x] Reviewed the
[`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md)
guide (**required**)
- [ ] Documentation added
- [ ] Tests updated
- [x] `CHANGELOG.md` updated
- [ ] If the change is worth mentioning in the release notes, add
`add-to-release-notes` label
- [ ] Changes that require user attention or interaction to upgrade are
documented in `docs/sources/setup/upgrade/_index.md`
- [ ] For Helm chart changes bump the Helm chart version in
`production/helm/loki/Chart.yaml` and update
`production/helm/loki/CHANGELOG.md` and
`production/helm/loki/README.md`. [Example
PR](d10549e)
- [ ] If the change is deprecating or removing a configuration option,
update the `deprecated-config.yaml` and `deleted-config.yaml` files
respectively in the `tools/deprecated-config-checker` directory.
[Example
PR](0d4416a)

**What this PR does / why we need it**:
Updates changelog and release notes for 2.9.4 patch release

**Checklist**
- [x] Reviewed the
[`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md)
guide (**required**)
- [x] Documentation added
- [x] `CHANGELOG.md` updated

**What this PR does / why we need it**:
Ran recently introduced tag updater `tools/release_update_tags.sh`
(#10646) to update version tags in
docs and yaml files