Unable to activate fluent-plugin-elasticsearch-5.2.5, because faraday-2.7.4 conflicts with faraday (~> 1.10) (Gem::ConflictError) #1014

danfinn · 2023-04-05T22:11:19Z

(check apply)

read the contribution guideline
(optional) already reported 3rd party upstream repository or mailing list if you use k8s addon or helm charts.

Problem

Recently started seeing this error when starting our fluentd pods. We did recently upgrade the elasticsearch gem from 7.17.7 to 8.6 but I'm nearly sure we weren't seeing this error after making that change.

2023-04-05 22:00:03 +0000 [info]: init supervisor logger path=nil rotate_age=nil rotate_size=nil
2023-04-05 22:00:03 +0000 [info]: parsing config file is succeeded path="/fluentd/etc/main.conf"
2023-04-05 22:00:03 +0000 [info]: gem 'fluentd' version '1.16.0'
2023-04-05 22:00:03 +0000 [info]: gem 'fluent-plugin-azure-storage-append-blob' version '0.2.1'
2023-04-05 22:00:03 +0000 [info]: gem 'fluent-plugin-concat' version '2.5.0'
2023-04-05 22:00:03 +0000 [info]: gem 'fluent-plugin-dedot_filter' version '1.0.0'
2023-04-05 22:00:03 +0000 [info]: gem 'fluent-plugin-detect-exceptions' version '0.0.15'
2023-04-05 22:00:03 +0000 [info]: gem 'fluent-plugin-elasticsearch' version '5.2.5'
2023-04-05 22:00:03 +0000 [info]: gem 'fluent-plugin-grok-parser' version '2.6.2'
2023-04-05 22:00:03 +0000 [info]: gem 'fluent-plugin-json-in-json-2' version '1.0.2'
2023-04-05 22:00:03 +0000 [info]: gem 'fluent-plugin-kubernetes_metadata_filter' version '3.1.3'
2023-04-05 22:00:03 +0000 [info]: gem 'fluent-plugin-multi-format-parser' version '1.0.0'
2023-04-05 22:00:03 +0000 [info]: gem 'fluent-plugin-parser-cri' version '0.1.1'
2023-04-05 22:00:03 +0000 [info]: gem 'fluent-plugin-prometheus' version '2.0.3'
2023-04-05 22:00:03 +0000 [info]: gem 'fluent-plugin-record-modifier' version '2.1.1'
2023-04-05 22:00:03 +0000 [info]: gem 'fluent-plugin-rewrite-tag-filter' version '2.4.0'
2023-04-05 22:00:03 +0000 [info]: gem 'fluent-plugin-systemd' version '1.0.5'
2023-04-05 22:00:03 +0000 [info]: gem 'fluentd' version '1.15.3'
2023-04-05 22:00:03 +0000 [info]: DeDot will recurse nested hashes and arrays
/usr/local/lib/ruby/3.1.0/rubygems/specification.rb:2288:in `raise_if_conflicts': Unable to activate fluent-plugin-elasticsearch-5.2.5, because faraday-2.7.4 conflicts with faraday (~> 1.10) (Gem::ConflictError)
	from /usr/local/lib/ruby/3.1.0/rubygems/specification.rb:1419:in `activate'
	from /usr/local/lib/ruby/3.1.0/rubygems.rb:211:in `rescue in try_activate'
	from /usr/local/lib/ruby/3.1.0/rubygems.rb:204:in `try_activate'
	from <internal:/usr/local/lib/ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:153:in `rescue in require'
	from <internal:/usr/local/lib/ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:149:in `require'
	from /usr/local/lib/ruby/gems/3.1.0/gems/fluent-plugin-elasticsearch-5.2.5/lib/fluent/plugin/out_elasticsearch.rb:20:in `<top (required)>'
	from <internal:/usr/local/lib/ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
	from <internal:/usr/local/lib/ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
	from /usr/local/lib/ruby/gems/3.1.0/gems/fluentd-1.16.0/lib/fluent/registry.rb:103:in `block in search'
	from /usr/local/lib/ruby/gems/3.1.0/gems/fluentd-1.16.0/lib/fluent/registry.rb:100:in `each'
	from /usr/local/lib/ruby/gems/3.1.0/gems/fluentd-1.16.0/lib/fluent/registry.rb:100:in `search'
	from /usr/local/lib/ruby/gems/3.1.0/gems/fluentd-1.16.0/lib/fluent/registry.rb:44:in `lookup'
	from /usr/local/lib/ruby/gems/3.1.0/gems/fluentd-1.16.0/lib/fluent/plugin.rb:169:in `new_impl'
	from /usr/local/lib/ruby/gems/3.1.0/gems/fluentd-1.16.0/lib/fluent/plugin.rb:114:in `new_output'
	from /usr/local/lib/ruby/gems/3.1.0/gems/fluentd-1.16.0/lib/fluent/agent.rb:130:in `add_match'
	from /usr/local/lib/ruby/gems/3.1.0/gems/fluentd-1.16.0/lib/fluent/agent.rb:74:in `block in configure'
	from /usr/local/lib/ruby/gems/3.1.0/gems/fluentd-1.16.0/lib/fluent/agent.rb:64:in `each'
	from /usr/local/lib/ruby/gems/3.1.0/gems/fluentd-1.16.0/lib/fluent/agent.rb:64:in `configure'
	from /usr/local/lib/ruby/gems/3.1.0/gems/fluentd-1.16.0/lib/fluent/root_agent.rb:149:in `configure'
	from /usr/local/lib/ruby/gems/3.1.0/gems/fluentd-1.16.0/lib/fluent/engine.rb:105:in `configure'
	from /usr/local/lib/ruby/gems/3.1.0/gems/fluentd-1.16.0/lib/fluent/engine.rb:80:in `run_configure'
	from /usr/local/lib/ruby/gems/3.1.0/gems/fluentd-1.16.0/lib/fluent/supervisor.rb:571:in `run_supervisor'
	from /usr/local/lib/ruby/gems/3.1.0/gems/fluentd-1.16.0/lib/fluent/command/fluentd.rb:352:in `<top (required)>'
	from <internal:/usr/local/lib/ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
	from <internal:/usr/local/lib/ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
	from /usr/local/lib/ruby/gems/3.1.0/gems/fluentd-1.16.0/bin/fluentd:15:in `<top (required)>'
	from /usr/local/bundle/bin/fluentd:25:in `load'
	from /usr/local/bundle/bin/fluentd:25:in `<main>'
/usr/local/lib/ruby/3.1.0/rubygems/specification.rb:2288:in `raise_if_conflicts': Unable to activate fluent-plugin-elasticsearch-5.2.5, because faraday-2.7.4 conflicts with faraday (~> 1.10) (Gem::ConflictError)
	from /usr/local/lib/ruby/3.1.0/rubygems/specification.rb:1419:in `activate'
	from /usr/local/lib/ruby/3.1.0/rubygems.rb:205:in `try_activate'
	from <internal:/usr/local/lib/ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:153:in `rescue in require'
	from <internal:/usr/local/lib/ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:149:in `require'
	from /usr/local/lib/ruby/gems/3.1.0/gems/fluent-plugin-elasticsearch-5.2.5/lib/fluent/plugin/out_elasticsearch.rb:20:in `<top (required)>'
	from <internal:/usr/local/lib/ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
	from <internal:/usr/local/lib/ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
	from /usr/local/lib/ruby/gems/3.1.0/gems/fluentd-1.16.0/lib/fluent/registry.rb:103:in `block in search'
	from /usr/local/lib/ruby/gems/3.1.0/gems/fluentd-1.16.0/lib/fluent/registry.rb:100:in `each'
	from /usr/local/lib/ruby/gems/3.1.0/gems/fluentd-1.16.0/lib/fluent/registry.rb:100:in `search'
	from /usr/local/lib/ruby/gems/3.1.0/gems/fluentd-1.16.0/lib/fluent/registry.rb:44:in `lookup'
	from /usr/local/lib/ruby/gems/3.1.0/gems/fluentd-1.16.0/lib/fluent/plugin.rb:169:in `new_impl'
	from /usr/local/lib/ruby/gems/3.1.0/gems/fluentd-1.16.0/lib/fluent/plugin.rb:114:in `new_output'
	from /usr/local/lib/ruby/gems/3.1.0/gems/fluentd-1.16.0/lib/fluent/agent.rb:130:in `add_match'
	from /usr/local/lib/ruby/gems/3.1.0/gems/fluentd-1.16.0/lib/fluent/agent.rb:74:in `block in configure'
	from /usr/local/lib/ruby/gems/3.1.0/gems/fluentd-1.16.0/lib/fluent/agent.rb:64:in `each'
	from /usr/local/lib/ruby/gems/3.1.0/gems/fluentd-1.16.0/lib/fluent/agent.rb:64:in `configure'
	from /usr/local/lib/ruby/gems/3.1.0/gems/fluentd-1.16.0/lib/fluent/root_agent.rb:149:in `configure'
	from /usr/local/lib/ruby/gems/3.1.0/gems/fluentd-1.16.0/lib/fluent/engine.rb:105:in `configure'
	from /usr/local/lib/ruby/gems/3.1.0/gems/fluentd-1.16.0/lib/fluent/engine.rb:80:in `run_configure'
	from /usr/local/lib/ruby/gems/3.1.0/gems/fluentd-1.16.0/lib/fluent/supervisor.rb:571:in `run_supervisor'
	from /usr/local/lib/ruby/gems/3.1.0/gems/fluentd-1.16.0/lib/fluent/command/fluentd.rb:352:in `<top (required)>'
	from <internal:/usr/local/lib/ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
	from <internal:/usr/local/lib/ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
	from /usr/local/lib/ruby/gems/3.1.0/gems/fluentd-1.16.0/bin/fluentd:15:in `<top (required)>'
	from /usr/local/bundle/bin/fluentd:25:in `load'
	from /usr/local/bundle/bin/fluentd:25:in `<main>'
<internal:/usr/local/lib/ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:85:in `require': cannot load such file -- fluent/log-ext (LoadError)
	from <internal:/usr/local/lib/ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
	from /usr/local/lib/ruby/gems/3.1.0/gems/fluent-plugin-elasticsearch-5.2.5/lib/fluent/plugin/out_elasticsearch.rb:20:in `<top (required)>'
	from <internal:/usr/local/lib/ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
	from <internal:/usr/local/lib/ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
	from /usr/local/lib/ruby/gems/3.1.0/gems/fluentd-1.16.0/lib/fluent/registry.rb:103:in `block in search'
	from /usr/local/lib/ruby/gems/3.1.0/gems/fluentd-1.16.0/lib/fluent/registry.rb:100:in `each'
	from /usr/local/lib/ruby/gems/3.1.0/gems/fluentd-1.16.0/lib/fluent/registry.rb:100:in `search'
	from /usr/local/lib/ruby/gems/3.1.0/gems/fluentd-1.16.0/lib/fluent/registry.rb:44:in `lookup'
	from /usr/local/lib/ruby/gems/3.1.0/gems/fluentd-1.16.0/lib/fluent/plugin.rb:169:in `new_impl'
	from /usr/local/lib/ruby/gems/3.1.0/gems/fluentd-1.16.0/lib/fluent/plugin.rb:114:in `new_output'
	from /usr/local/lib/ruby/gems/3.1.0/gems/fluentd-1.16.0/lib/fluent/agent.rb:130:in `add_match'
	from /usr/local/lib/ruby/gems/3.1.0/gems/fluentd-1.16.0/lib/fluent/agent.rb:74:in `block in configure'
	from /usr/local/lib/ruby/gems/3.1.0/gems/fluentd-1.16.0/lib/fluent/agent.rb:64:in `each'
	from /usr/local/lib/ruby/gems/3.1.0/gems/fluentd-1.16.0/lib/fluent/agent.rb:64:in `configure'
	from /usr/local/lib/ruby/gems/3.1.0/gems/fluentd-1.16.0/lib/fluent/root_agent.rb:149:in `configure'
	from /usr/local/lib/ruby/gems/3.1.0/gems/fluentd-1.16.0/lib/fluent/engine.rb:105:in `configure'
	from /usr/local/lib/ruby/gems/3.1.0/gems/fluentd-1.16.0/lib/fluent/engine.rb:80:in `run_configure'
	from /usr/local/lib/ruby/gems/3.1.0/gems/fluentd-1.16.0/lib/fluent/supervisor.rb:571:in `run_supervisor'
	from /usr/local/lib/ruby/gems/3.1.0/gems/fluentd-1.16.0/lib/fluent/command/fluentd.rb:352:in `<top (required)>'
	from <internal:/usr/local/lib/ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
	from <internal:/usr/local/lib/ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
	from /usr/local/lib/ruby/gems/3.1.0/gems/fluentd-1.16.0/bin/fluentd:15:in `<top (required)>'
	from /usr/local/bundle/bin/fluentd:25:in `load'
	from /usr/local/bundle/bin/fluentd:25:in `<main>'

...

Steps to replicate

Here's our Dockerfile:

FROM fluent/fluentd:v1.15-debian-1

USER root
WORKDIR /home/fluent

#ENV PATH /fluentd/vendor/bundle/ruby/2.6.0/bin:$PATH
#ENV GEM_PATH /fluentd/vendor/bundle/ruby/2.6.0
#ENV GEM_HOME /fluentd/vendor/bundle/ruby/2.6.0
# skip runtime bundler installation
ENV FLUENTD_DISABLE_BUNDLER_INJECTION 1

RUN buildDeps="sudo make gcc g++ libc-dev" \
 && apt-get update \
 && apt-get install -y --no-install-recommends $buildDeps \
 && sudo gem install elasticsearch -v 8.6 \
 && sudo gem install fluent-plugin-elasticsearch \
 && sudo gem install fluent-plugin-multi-format-parser \
 && sudo gem install fluent-plugin-concat \
 && sudo gem install fluent-plugin-grok-parser \
 && sudo gem install fluent-plugin-prometheus \
 && sudo gem install fluent-plugin-json-in-json-2 \
 && sudo gem install fluent-plugin-record-modifier \
 && sudo gem install fluent-plugin-detect-exceptions \
 && sudo gem install fluent-plugin-rewrite-tag-filter \
 && sudo gem install fluent-plugin-dedot_filter \
 && sudo gem install fluent-plugin-kubernetes_metadata_filter \
 && sudo gem install ffi \
 && sudo gem install fluent-plugin-systemd \
 && sudo gem install fluent-plugin-parser-cri \
 && sudo gem install fluent-plugin-azure-storage-append-blob \
 && sudo gem sources --clear-all \
 && SUDO_FORCE_REMOVE=yes \
    apt-get purge -y --auto-remove \
                  -o APT::AutoRemove::RecommendsImportant=false \
                  $buildDeps \
 && rm -rf /var/lib/apt/lists/* \
 && rm -rf /tmp/* /var/tmp/* /usr/lib/ruby/gems/*/cache/*.gem

ADD conf /fluentd/etc/
ADD plugins /fluentd/plugins/

COPY entrypoint.sh /fluentd/entrypoint.sh

# Environment variables
ENV FLUENTD_OPT=""
ENV FLUENTD_CONF="fluent.conf"

EXPOSE 9292:9292

# Overwrite ENTRYPOINT to run fluentd as root for /var/log / /var/lib
ENTRYPOINT ["tini", "--", "/fluentd/entrypoint.sh"]

Expected Behavior or What you need to ask

the pod to start without errors

Using Fluentd and ES plugin versions

see Dockerfile above
kubernetes
Fluentd v1.15
ES plugin 8.6

I did also try using FROM fluent/fluentd:v1.16-debian-1 but there was no change there. I saw that there was a PR 3 days ago related to unpinning the version of faraday, perhaps that is related to this?
#1012

The text was updated successfully, but these errors were encountered:

danfinn · 2023-04-05T22:18:44Z

I tried to downgrade the elasticsearch gem to 7.17.7 which is what we were previously using and still get the same error on startup.

danfinn · 2023-04-05T23:54:25Z

Pinning the install of fluent-plugin-elasticsearch to 5.2.3 gets the pod running but now I get a new error:

2023-04-06 00:30:36 +0000 [error]: #0 unexpected error error_class=Faraday::Error error=":excon is not registered on Faraday::Adapter"
  2023-04-06 00:30:36 +0000 [error]: #0 /usr/local/lib/ruby/gems/3.1.0/gems/faraday-2.7.4/lib/faraday/middleware_registry.rb:57:in `lookup_middleware'
  2023-04-06 00:30:36 +0000 [error]: #0 /usr/local/lib/ruby/gems/3.1.0/gems/faraday-2.7.4/lib/faraday/rack_builder.rb:113:in `adapter'
  2023-04-06 00:30:36 +0000 [error]: #0 /usr/local/lib/ruby/3.1.0/forwardable.rb:238:in `adapter'
  2023-04-06 00:30:36 +0000 [error]: #0 /usr/local/lib/ruby/gems/3.1.0/gems/fluent-plugin-elasticsearch-5.2.3/lib/fluent/plugin/out_elasticsearch.rb:600:in `block in client'
  2023-04-06 00:30:36 +0000 [error]: #0 /usr/local/lib/ruby/gems/3.1.0/gems/faraday-2.7.4/lib/faraday/connection.rb:91:in `initialize'
  2023-04-06 00:30:36 +0000 [error]: #0 /usr/local/lib/ruby/gems/3.1.0/gems/faraday-2.7.4/lib/faraday.rb:98:in `new'
  2023-04-06 00:30:36 +0000 [error]: #0 /usr/local/lib/ruby/gems/3.1.0/gems/faraday-2.7.4/lib/faraday.rb:98:in `new'
  2023-04-06 00:30:36 +0000 [error]: #0 /usr/local/lib/ruby/gems/3.1.0/gems/elastic-transport-8.2.1/lib/elastic/transport/transport/http/faraday.rb:66:in `__build_connection'
  2023-04-06 00:30:36 +0000 [error]: #0 /usr/local/lib/ruby/gems/3.1.0/gems/elastic-transport-8.2.1/lib/elastic/transport/transport/base.rb:169:in `block in __connections_from_host'
  2023-04-06 00:30:36 +0000 [error]: #0 /usr/local/lib/ruby/gems/3.1.0/gems/elastic-transport-8.2.1/lib/elastic/transport/transport/base.rb:161:in `map'
  2023-04-06 00:30:36 +0000 [error]: #0 /usr/local/lib/ruby/gems/3.1.0/gems/elastic-transport-8.2.1/lib/elastic/transport/transport/base.rb:161:in `__connections_from_host'
  2023-04-06 00:30:36 +0000 [error]: #0 /usr/local/lib/ruby/gems/3.1.0/gems/elastic-transport-8.2.1/lib/elastic/transport/transport/base.rb:149:in `__build_connections'
  2023-04-06 00:30:36 +0000 [error]: #0 /usr/local/lib/ruby/gems/3.1.0/gems/elastic-transport-8.2.1/lib/elastic/transport/transport/base.rb:60:in `initialize'
  2023-04-06 00:30:36 +0000 [error]: #0 /usr/local/lib/ruby/gems/3.1.0/gems/fluent-plugin-elasticsearch-5.2.3/lib/fluent/plugin/out_elasticsearch.rb:617:in `new'
  2023-04-06 00:30:36 +0000 [error]: #0 /usr/local/lib/ruby/gems/3.1.0/gems/fluent-plugin-elasticsearch-5.2.3/lib/fluent/plugin/out_elasticsearch.rb:617:in `client'
  2023-04-06 00:30:36 +0000 [error]: #0 /usr/local/lib/ruby/gems/3.1.0/gems/fluent-plugin-elasticsearch-5.2.3/lib/fluent/plugin/elasticsearch_index_template.rb:39:in `host_unreachable_exceptions'
  2023-04-06 00:30:36 +0000 [error]: #0 /usr/local/lib/ruby/gems/3.1.0/gems/fluent-plugin-elasticsearch-5.2.3/lib/fluent/plugin/elasticsearch_index_template.rb:57:in `rescue in retry_operate'
  2023-04-06 00:30:36 +0000 [error]: #0 /usr/local/lib/ruby/gems/3.1.0/gems/fluent-plugin-elasticsearch-5.2.3/lib/fluent/plugin/elasticsearch_index_template.rb:55:in `retry_operate'
  2023-04-06 00:30:36 +0000 [error]: #0 /usr/local/lib/ruby/gems/3.1.0/gems/fluent-plugin-elasticsearch-5.2.3/lib/fluent/plugin/out_elasticsearch.rb:484:in `handle_last_seen_es_major_version'
  2023-04-06 00:30:36 +0000 [error]: #0 /usr/local/lib/ruby/gems/3.1.0/gems/fluent-plugin-elasticsearch-5.2.3/lib/fluent/plugin/out_elasticsearch.rb:336:in `configure'
  2023-04-06 00:30:36 +0000 [error]: #0 /usr/local/lib/ruby/gems/3.1.0/gems/fluentd-1.16.0/lib/fluent/plugin.rb:187:in `configure'
  2023-04-06 00:30:36 +0000 [error]: #0 /usr/local/lib/ruby/gems/3.1.0/gems/fluentd-1.16.0/lib/fluent/agent.rb:132:in `add_match'
  2023-04-06 00:30:36 +0000 [error]: #0 /usr/local/lib/ruby/gems/3.1.0/gems/fluentd-1.16.0/lib/fluent/agent.rb:74:in `block in configure'
  2023-04-06 00:30:36 +0000 [error]: #0 /usr/local/lib/ruby/gems/3.1.0/gems/fluentd-1.16.0/lib/fluent/agent.rb:64:in `each'
  2023-04-06 00:30:36 +0000 [error]: #0 /usr/local/lib/ruby/gems/3.1.0/gems/fluentd-1.16.0/lib/fluent/agent.rb:64:in `configure'
  2023-04-06 00:30:36 +0000 [error]: #0 /usr/local/lib/ruby/gems/3.1.0/gems/fluentd-1.16.0/lib/fluent/root_agent.rb:149:in `configure'
  2023-04-06 00:30:36 +0000 [error]: #0 /usr/local/lib/ruby/gems/3.1.0/gems/fluentd-1.16.0/lib/fluent/engine.rb:105:in `configure'
  2023-04-06 00:30:36 +0000 [error]: #0 /usr/local/lib/ruby/gems/3.1.0/gems/fluentd-1.16.0/lib/fluent/engine.rb:80:in `run_configure'
  2023-04-06 00:30:36 +0000 [error]: #0 /usr/local/lib/ruby/gems/3.1.0/gems/fluentd-1.16.0/lib/fluent/supervisor.rb:616:in `block in run_worker'
  2023-04-06 00:30:36 +0000 [error]: #0 /usr/local/lib/ruby/gems/3.1.0/gems/fluentd-1.16.0/lib/fluent/supervisor.rb:962:in `main_process'
  2023-04-06 00:30:36 +0000 [error]: #0 /usr/local/lib/ruby/gems/3.1.0/gems/fluentd-1.16.0/lib/fluent/supervisor.rb:608:in `run_worker'
  2023-04-06 00:30:36 +0000 [error]: #0 /usr/local/lib/ruby/gems/3.1.0/gems/fluentd-1.16.0/lib/fluent/command/fluentd.rb:372:in `<top (required)>'
  2023-04-06 00:30:36 +0000 [error]: #0 <internal:/usr/local/lib/ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
  2023-04-06 00:30:36 +0000 [error]: #0 <internal:/usr/local/lib/ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
  2023-04-06 00:30:36 +0000 [error]: #0 /usr/local/lib/ruby/gems/3.1.0/gems/fluentd-1.16.0/bin/fluentd:15:in `<top (required)>'
  2023-04-06 00:30:36 +0000 [error]: #0 /usr/local/bundle/bin/fluentd:25:in `load'
  2023-04-06 00:30:36 +0000 [error]: #0 /usr/local/bundle/bin/fluentd:25:in `<main>'

This is now the same error as
#984

raytung · 2023-04-06T00:37:57Z

I saw that there was a PR 3 days ago related to unpinning the version of faraday, perhaps that is related to this?

Hey @danfinn, my understanding is that while the PR is merged we haven't done a release yet. One thing to note is that I can see you are using fluent-plugin-azure-storage-append-blob which is now unsupported and uses Faraday v1.x so this will definitely cause a conflict when the author eventually release #1012

danfinn · 2023-04-06T00:42:05Z

Unsupported or unmaintained? I think those are 2 different things?

Do you know if
https://github.com/elsesiy/fluent-plugin-azure-storage-append-blob-lts
also uses faraday v1.x?

danfinn · 2023-04-06T00:48:41Z

It seems like the suggestion from #984 of installing faraday 1.10.0 has things working again.

raytung · 2023-04-06T01:07:37Z

Do you know if
https://github.com/elsesiy/fluent-plugin-azure-storage-append-blob-lts
also uses faraday v1.x?

@danfinn Unfortunately it does. https://github.com/Azure/azure-storage-ruby/blob/master/common/azure-storage-common.gemspec#L44. The problem actually lies in the 2nd layer of transitive dependency

fluent-plugin-azure-storage-append-blob -> azure-storage-blob -> azure-storage-common. Looks like there's an open issue on the Azure library on this Azure/azure-storage-ruby#225

danfinn · 2023-04-06T18:36:15Z

The issue does not appear to be the fluent-plugin-azure-storage-append-blob gem. I removed that from my dockerfile and I still get the same error:

/usr/local/lib/ruby/3.1.0/rubygems/specification.rb:2288:in `raise_if_conflicts': Unable to activate fluent-plugin-elasticsearch-5.2.5, because faraday-2.7.4 conflicts with faraday (~> 1.10) (Gem::ConflictError)

Here is my Dockerfile that generated this error:

FROM fluent/fluentd:v1.15-debian-1

USER root
WORKDIR /home/fluent

#ENV PATH /fluentd/vendor/bundle/ruby/2.6.0/bin:$PATH
#ENV GEM_PATH /fluentd/vendor/bundle/ruby/2.6.0
#ENV GEM_HOME /fluentd/vendor/bundle/ruby/2.6.0
# skip runtime bundler installation
ENV FLUENTD_DISABLE_BUNDLER_INJECTION 1

RUN buildDeps="sudo make gcc g++ libc-dev" \
 && apt-get update \
 && apt-get install -y --no-install-recommends $buildDeps \
 # && sudo gem install faraday --no-document --version 1.10.0 \
 && sudo gem install elasticsearch -v 8.6 \
 && sudo gem install fluent-plugin-elasticsearch \
 && sudo gem install fluent-plugin-multi-format-parser \
 && sudo gem install fluent-plugin-concat \
 && sudo gem install fluent-plugin-grok-parser \
 && sudo gem install fluent-plugin-prometheus \
 && sudo gem install fluent-plugin-json-in-json-2 \
 && sudo gem install fluent-plugin-record-modifier \
 && sudo gem install fluent-plugin-detect-exceptions \
 && sudo gem install fluent-plugin-rewrite-tag-filter \
 && sudo gem install fluent-plugin-dedot_filter \
 && sudo gem install fluent-plugin-kubernetes_metadata_filter \
 && sudo gem install ffi \
 && sudo gem install fluent-plugin-systemd \
 && sudo gem install fluent-plugin-parser-cri \
 # && sudo gem install fluent-plugin-azure-storage-append-blob \
 && sudo gem sources --clear-all \
 && SUDO_FORCE_REMOVE=yes \
    apt-get purge -y --auto-remove \
                  -o APT::AutoRemove::RecommendsImportant=false \
                  $buildDeps \
 && rm -rf /var/lib/apt/lists/* \
 && rm -rf /tmp/* /var/tmp/* /usr/lib/ruby/gems/*/cache/*.gem

ADD conf /fluentd/etc/
ADD plugins /fluentd/plugins/

COPY entrypoint.sh /fluentd/entrypoint.sh

# Environment variables
ENV FLUENTD_OPT=""
ENV FLUENTD_CONF="fluent.conf"

EXPOSE 9292:9292

# Overwrite ENTRYPOINT to run fluentd as root for /var/log / /var/lib
ENTRYPOINT ["tini", "--", "/fluentd/entrypoint.sh"]

cosmo0920 · 2023-04-08T13:12:35Z

Could you use bundle and Gemfile to manage your gem dependencies instead?
This is because gem install always should use/fetch the latest gems.
Bundler should follow the versions that are user specified.

cosmo0920 · 2023-04-11T06:11:13Z

Will be handled to publish this plugin as v5.3.0.

kenhys · 2023-04-11T08:19:38Z

v5.3.0 has been released so I hope your issue will be solved.

danfinn · 2023-04-13T15:45:16Z

ok, I will try building again with v5.3.0

brook-w · 2023-06-12T07:50:04Z

Unable to activate fluent-plugin-elasticsearch-5.3.0, because faraday-1.10.3 conflicts with faraday (>= 2.0.0), faraday-excon-1.1.0 conflicts with faraday-excon (>= 2.0.0) WTF

brook-w · 2023-06-12T07:52:13Z

#1014 (comment)

brook-w · 2023-06-12T07:52:48Z

Unable to activate fluent-plugin-elasticsearch-5.3.0, because faraday-1.10.3 conflicts with faraday (>= 2.0.0), faraday-excon-1.1.0 conflicts with faraday-excon (>= 2.0.0) WTF

v5.3.0 has been released so I hope your issue will be solved.

sebastiangaiser mentioned this issue Apr 17, 2023

Gem::ConflictError when applying ClusterFlow kube-logging/logging-operator#1251

Closed

yelouarti mentioned this issue Dec 5, 2023

error=":excon is not registered on Faraday::Adapter" with v1.15.0 Docker and fluent-plugin-opensearch fluent/fluentd-docker-image#338

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Unable to activate fluent-plugin-elasticsearch-5.2.5, because faraday-2.7.4 conflicts with faraday (~> 1.10) (Gem::ConflictError) #1014

Unable to activate fluent-plugin-elasticsearch-5.2.5, because faraday-2.7.4 conflicts with faraday (~> 1.10) (Gem::ConflictError) #1014

danfinn commented Apr 5, 2023 •

edited

danfinn commented Apr 5, 2023

danfinn commented Apr 5, 2023 •

edited

raytung commented Apr 6, 2023

danfinn commented Apr 6, 2023 •

edited

danfinn commented Apr 6, 2023

raytung commented Apr 6, 2023 •

edited

danfinn commented Apr 6, 2023 •

edited

cosmo0920 commented Apr 8, 2023 •

edited

cosmo0920 commented Apr 11, 2023

kenhys commented Apr 11, 2023

danfinn commented Apr 13, 2023

brook-w commented Jun 12, 2023

brook-w commented Jun 12, 2023

brook-w commented Jun 12, 2023

Unable to activate fluent-plugin-elasticsearch-5.2.5, because faraday-2.7.4 conflicts with faraday (~> 1.10) (Gem::ConflictError) #1014

Unable to activate fluent-plugin-elasticsearch-5.2.5, because faraday-2.7.4 conflicts with faraday (~> 1.10) (Gem::ConflictError) #1014

Comments

danfinn commented Apr 5, 2023 • edited

Problem

Steps to replicate

Expected Behavior or What you need to ask

Using Fluentd and ES plugin versions

danfinn commented Apr 5, 2023

danfinn commented Apr 5, 2023 • edited

raytung commented Apr 6, 2023

danfinn commented Apr 6, 2023 • edited

danfinn commented Apr 6, 2023

raytung commented Apr 6, 2023 • edited

danfinn commented Apr 6, 2023 • edited

cosmo0920 commented Apr 8, 2023 • edited

cosmo0920 commented Apr 11, 2023

kenhys commented Apr 11, 2023

danfinn commented Apr 13, 2023

brook-w commented Jun 12, 2023

brook-w commented Jun 12, 2023

brook-w commented Jun 12, 2023

danfinn commented Apr 5, 2023 •

edited

danfinn commented Apr 5, 2023 •

edited

danfinn commented Apr 6, 2023 •

edited

raytung commented Apr 6, 2023 •

edited

danfinn commented Apr 6, 2023 •

edited

cosmo0920 commented Apr 8, 2023 •

edited