Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Output resources connects even if unused #2523

Open
alessandro-gomma opened this issue Apr 16, 2024 · 0 comments
Open

Output resources connects even if unused #2523

alessandro-gomma opened this issue Apr 16, 2024 · 0 comments
Labels
bughancement It's kind of a bug, but mostly an enhancement. configuration Any tasks or issues relating to Benthos configuration needs investigation It looks as though have all the information needed but investigation is required

Comments

@alessandro-gomma
Copy link

Hi!

while building our streams we tried to avoid code duplication by replacing our outputs with a resource.
While trying different configurations, we noticed that if we declare multiple output resources, they immediately connect upon startup even if these resources aren't referenced in any stream.

Here's our resource.yaml file:

output_resources:
  - label: "first_output"
    nats:
      urls: ["nats://127.0.0.1:1234"]
      subject: ${! @output_subject }
  - label: "second_output"
    nats:
      urls: ["nats://127.0.0.1:5678"]
      subject: ${! @output_subject }

Here's the stream file:

input:
  broker:
    inputs:
      [...]

pipeline:
  processors:
    [...]

output:
  label: output
  resource: "first_output"

And this is the startup script:

benthos  -e "config.env" -c "./common/common.yaml" -r "./resources/*.yaml" -t "./templates/*.yaml" streams "./config/*.yaml"

In our solution, we would like to define multiple output resources in a single yaml file. Since we plan to have plenty of different outputs in the same file, we want to be sure that only the referenced resources are actually instantiated.
Is there any way to avoid the "second_output" resource to be used when not referenced? (e.g. lazy loading?)

@mihaitodor mihaitodor added bughancement It's kind of a bug, but mostly an enhancement. needs investigation It looks as though have all the information needed but investigation is required configuration Any tasks or issues relating to Benthos configuration labels May 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bughancement It's kind of a bug, but mostly an enhancement. configuration Any tasks or issues relating to Benthos configuration needs investigation It looks as though have all the information needed but investigation is required
Projects
None yet
Development

No branches or pull requests

2 participants