prep release: v1.43.1

[BrynCooke]

> **Note**
>
> When approved, this PR will merge into **the `1.43.1` branch** which will — upon being approved itself — merge into `main`.
>
> **Things to review in this PR**:
>  - Changelog correctness (There is a preview below, but it is not necessarily the most up to date.  See the _Files Changed_ for the true reality.)
>  - Version bumps
>  - That it targets the right release branch (`1.43.1` in this case!).
>
---

🚀 Features

Enable displaying trace and span id on logs (PR #4823)

To enable correlation between trace and logs trace_id and span_id can now be included on the log messages.

Json:

{"timestamp":"2024-03-19T15:37:41.516453239Z","level":"INFO","trace_id":"54ac7e5f0e8ab90ae67b822e95ffcbb8","span_id":"9b3f88c602de0ceb","message":"Supergraph GraphQL response".....

Text:

2024-03-19T15:14:46.040435Z INFO trace_id: bbafc3f048b6137375dd78c10df18f50 span_id: 40ede28c5df1b5cc router{

To configure this, use display_span_id and display_trace_id options in the logging exporter configuration.

Json (defaults to true):

telemetry:
  exporters:
    logging:
      stdout:
        format:
          json:
            display_span_id: true
            display_trace_id: true

Text (defaults to false):

telemetry:
  exporters:
    logging:
      stdout:
        format:
          text:
            display_span_id: false
            display_trace_id: false

By @BrynCooke in #4823

count errors in the apollo.router.graphql_error metrics (Issue #4749)

This introduces a new counter metric called apollo.router.graphql_error that counts GraphQL errors, split by their code

By @Geal in #4751

expose the operation signature in the context (Issue #4558)

It is available at the key apollo_operation_signature.

By @Geal in #4864

Experimental logging of broken pipe errors (PR #4870)

You can now emit a log message each time the client closes the connection early, which can help you debug issues with clients that close connections before the server can respond.

This feature is disabled by default but can be enabled by setting the experimental_log_broken_pipe option to true:

supergraph:
  experimental_log_on_broken_pipe: true

Users that have internet facing routers will likely not want to opt in to this log message as they have no control over the clients.

By @Geal in #4770 and @BrynCooke in #4870

🐛 Fixes

Entity cache: fix support for Redis cluster (PR #4790)

in a Redis cluster, entities can be stored in different nodes, and a query to one node should only refer to the keys it manages. This is challenging for the MGET operation which requests multiple entities in the same request from the same node. This splits the MGET query in multiple MGETs calls grouped by key hash, to make sure each one will get to the corresponding node, then merges responses in the correct order.

By @Geal in #4790

Give spans their proper parent in the plugin stack (Issue #4827)

Due to the way plugin spans were created and applied, they would appear as siblings instead of being nested, which creates some issues when displaying traces and accounting for time spent in Datadog. Plugin spans are now correctly nested within each other.

By @Geal in #4877

Fix(telemetry): keep consistency between tracing otlp endpoint (Issue #4798)

In our documentation when configuration http endpoint for tracing otlp exporter we say that users should only include the base address of the otlp endpoint. It was only working for grpc protocol and not http due to this bug. This inconsistency is now fixed with a workaround in the router waiting for the fix in openetelemetry crate.

So for example you'll have to specify the right path for http:

telemetry:
  exporters:
    tracing:
      otlp:
        enabled: true
        endpoint: "http://localhost:4318"
        protocol: http

By @bnjjj in #4801

Execute the entire request pipeline if the client closed the connection (Issue #4569), Issue #4576), (Issue #4589), (Issue #4590), (Issue #4611)

The router is now making sure that the entire request handling pipeline is executed when the client closes the connection early, to let telemetry and any rhai scrit or coprocessor perform their tasks before canceling. Before that, when a client canceled a request, the entire execution was dropped and parts of the router, like telemetry, could not run properly. It now executes up to the first response event (in the case of subscription or @defer usage), adds a 499 status code to the response and skips the remaining subgraph requests.

This change will report more requests to Studio and the configured telemetry, which will appear like a sudden increase in errors, because those failing requests were not reported before. To keep the previous behavior of immediately dropping execution for canceled requests, it is possible with the following option:

supergraph:
  early_cancel: true

By @Geal in #4770

null extensions incorrectly disallowed on request (Issue #4856)

The graphql over http spec mandates null is allowed for request extensions.

We were previously rejecting such payloads, but now we will allow them. For example:

{
  "query": "{ topProducts { upc name reviews { id product { name } author { id name } } } }",
  "variables": {
    "date": "2022-01-01T00:00:00+00:00"
  },
  "extensions": null
}

Fixes #4856

By @BrynCooke in #4865

fix the log lines for external extensibility (PR #4869)

The execution and supergraph responses were mistakenly reporting as router. This is now fixed.

By @garypen in #4869

Remove invalid payload on graphql-ws Ping message (Issue #4852)

According to graphql-ws spec Ping payload should be an object or null but router was sending a string.
To ensure better compatibility Ping's payload was removed.

By @IvanGoncharov in #4852

[router-perf]

CI performance tests

• reload - Reload test over a long period of time at a constant rate of users
• events_big_cap_high_rate_callback - Stress test for events with a lot of users, deduplication enabled and high rate event with a big queue capacity using callback mode
• events_without_dedup_callback - Stress test for events with a lot of users and deduplication DISABLED using callback mode
• large-request - Stress test with a 1 MB request payload
• const - Basic stress test that runs with a constant number of users
• no-graphos - Basic stress test, no GraphOS.
• step-jemalloc-tuning - Clone of the basic stress test for jemalloc tuning
• events - Stress test for events with a lot of users and deduplication ENABLED
• events_callback - Stress test for events with a lot of users and deduplication ENABLED in callback mode
• events_big_cap_high_rate - Stress test for events with a lot of users, deduplication enabled and high rate event with a big queue capacity
• events_without_dedup - Stress test for events with a lot of users and deduplication DISABLED
• xxlarge-request - Stress test with 100 MB request payload
• xlarge-request - Stress test with 10 MB request payload
• step - Basic stress test that steps up the number of users over time

[shorgi]

Reviewed changelog

[nicholascioli]

LGTM!