@type elasticsearch_data_stream mandates host to be configured

[Ga13Ou]

• read the contribution guideline
• (optional) already reported 3rd party upstream repository or mailing list if you use k8s addon or helm charts.

Problem

elasticsearch_data_stream mandates host to be configured even though hosts is configured. Otherwise following error is seen in the logs.
failed to flush the buffer. retry_time=0 next_retry_seconds=2023-02-07 21:08:54 +0000 chunk="5f41e9b7cda5b02d1982342447579" error_class=Fluent ::ConfigError error="Failed to create data stream: <datastream-test-application-test> Connection refused - connect(2) for 127.0.0.1:9200 (Errno::ECONNREFUSED)"

This issue might seem familiar because the same problem was posted by someone else while using the opensearch plugin. The problem here is that the documentation doesn't clearly say that the use of "hosts" is not compatible with the use of data_streams
That can be found here fluent/fluent-plugin-opensearch#5
...

Steps to replicate

<match **>
          @id elasticsearch
          @type elasticsearch_data_stream
         ...
          hosts es-1.mycompany.ai:9200,es-2.mycompany.ai:9200,es-3.mycompany.ai:9200
          ...
          <buffer tag>
           ...
          </buffer>
        </match>

Expected Behavior or What you need to ask

As the documentation says: If you specify hosts option, host and port options are ignored.
https://github.com/uken/fluent-plugin-elasticsearch#hosts

Using Fluentd and ES plugin versions

The fluend helm chart version 0.3.9

Result of the fluent-gem list:
fluent-plugin-elasticsearch (5.2.4, 5.1.5)

The version 5.1.5 apparently comes shipped with the fluentd-elastic image of the chart and can't be remove (or I don't know how to do it if it can). The version 5.2.4 is installed by using the helm value plugins of the helm chart. According to fluentd documentation, if there are multiple versions then the most recent version is the one loaded by fluentd. Which mean in this case the version 5.2.4 (Correct me if I'm wrong).