[Console] Missing "user-agent" header when sending requests

[yuliacech]

A post on discuss has revealed that in some setups, the requests via Dev tools Console fail due to them not having the correct user-agent header set. In the current implementation of the Console proxy, all headers except those specified in the elasticsearch.requestHeadersWhitelist are filtered out (see this code). The header user-agent can't be added to that config though as Kibana has its own logic regarding this header (#141003).

To fix the failing requests, we could maybe copy the existing header user-agent when proxying the requests via Console?

[elasticmachine]

Pinging @elastic/kibana-management (Team:Kibana Management)

[lukeelmers]

From an observability perspective it might be helpful to set a different UA for console than we do from Kibana... sometimes it is helpful to understand what requests originate from Kibana (i.e. we have some level of control over the queries), and which ones are raw queries written by users.

We have the product-origin header which could also be used for this purpose, but I'm wondering if we would still risk confusion by using the same UA in the console proxy.

Any thoughts on this @elastic/kibana-core?

[pgayvallet]

I think it would be fine for requests performed from the dev console to be sending a user-agent header, and I agree that for observability purposes, it could be a good think to have a different header than the one used by the "programatic" ES client used by Kibana.

I would, however, avoid copying the browser's user agent, as it doesn't really reflect the reality of the product that performs the request.

FWIW, the user-agent header used by the Kibana ES client is following a Kibana/{version} format:

kibana/packages/core/elasticsearch/core-elasticsearch-client-server-internal/src/headers.ts

Line 46 in 681e2b5

[USER_AGENT_HEADER]: `Kibana/${kibanaVersion}`,

If we were to follow the user-agent's spec, we could imagine something like this for the dev console:

Kibana/8.12.0 (DevConsole)

or even a totally different product identifier

KibanaDevConsole/8.12.0

of course those are only example from the top of my mind, we would probably be fine with any naming, as long as we can distinguish it from the "normal" client's user agent (and that it follows the header's spec).