Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow to override the client class via fiber-local storage #6297

Merged
merged 2 commits into from
Jun 3, 2024

Conversation

Aerdayne
Copy link
Contributor

Hi!

This PR consists in a single addition to Sidekiq::Job.build_client - this method now checks Thread.current[:sidekiq_client_class] before falling back to the client class configured in the job configuration options, or to Sidekiq::Client in the absence of the former.

Context:

I'm currently working on a generic implementation of the transactional outbox pattern, the purpose of it is to stage any kind of deferred executions in the form of network requests to the database table within the same database transaction used by the business logic.

Some of the entities that can be staged in this manner are Sidekiq jobs, and the library provides a corresponding adapter out of the box. The action of staging a job/event/etc should be as seamless as invoking the original API, e.g. perform_async in Sidekiq's case. In the context of Sidekiq, this means that all argument manipulation should be completed and all middlewares should run before the entries end up in the database table, not when they're picked up by the background outbox daemon and are about to be relayed to their original destination. In order to achieve this, substitution of a default Sidekiq client with a custom client seems to be the most promising and the least intruding in terms of meddling with Sidekiq's internals.

The problem is that right now, it's impossible to temporary override the client class in an ad-hoc manner. Users of the outbox library should have both options when dispatching instances of the same job class: either in an immediate or in a staged manner when enqueuing from within a transaction. This makes setting the client_class via the options of a job class not possible.

One possible approach that I could take without changing anything in Sidekiq is creating a client class that would check for a fiber-local variable and delegate to the original Sidekiq::Client in case it's not set, but:

  • Technically it would override the client for all jobs. Even though it would delegate to the original client, this does not feel right;
  • This functionality might be useful to other integrations/gems that make use of their custom clients.

# This is useful when you want to use a custom client class
# in an ad-hoc manner in a specific context, while still
# using the default client class for the rest of the application.
def via_client(client_class)
Copy link
Contributor Author

@Aerdayne Aerdayne May 16, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't feel strongly about this convenience method, at least I don't need it for my use-case. Added it primarily for documentation purposes, can remove it if it's unnecessary.

P.S. It might also turn out treacherous since users might expect that the client is only substituted for the job class on which via_client was called, which is not the case. If there's a better place to define this method - please let me know.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd rather sharp edges like this not be documented at all. If you really need it, you'll find it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed the method and tests d9426b8.

@mperham mperham added this to the 7.3 milestone Jun 3, 2024
@mperham mperham merged commit fdd82b7 into sidekiq:main Jun 3, 2024
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants