💬 RFC: plugin-events-node requires the same instance to work

[marleypowell]

🔖 Need

The problem I'm having is that the DefaultEventsService stores subscriptions in an in memory Map. The problem with this is that it's possible to unintentionally have multiple versions of the @backstage/plugin-events-node package.

I have wasted a couple hours on two occasions trying to debug signals thinking that I had missed something but it turned out that my plugins were using a slightly different version of the events package.

I can see this causing problems with 3rd party plugins where you don't know what version you're going to get.

The current limitations are actually documented here:

backstage/plugins/events-node/src/api/DefaultEventsService.ts

Lines 21 to 28 in 96ee519

	/**
	* In-process event broker which will pass the event to all registered subscribers
	* interested in it.
	* Events will not be persisted in any form.
	* Events will not be passed to subscribers at other instances of the same cluster.
	*
	* @public
	*/

🎉 Proposal

Solutions with current DefaultEventsService

NPM resolutions

In your root application you can force everything to use the same version with NPM resolutions.

{
  "resolutions": {
    "@backstage/plugin-events-node": "0.3.3-next.1"
  },
}

Peer dependency

It could alternatively make sense that plugins reference the events package as a peer dependency instead so that the consuming backend is the one that provides the final version of the package.

Issue Visibility

There's the command yarn backstage-cli versions:check which can help find some of these issues but I'm not sure if that detects your dependencies own dependencies so you can still end up with a mismatch.
https://backstage.io/docs/getting-started/keeping-backstage-updated#more-information-on-dependency-mismatches

eventsServiceRef

Does some of the problem maybe come from here because the JSDoc says it uses the root scope but from what I can tell it uses plugin scope. Unless the createRootContext is what it means by root scope?

Anyway is it still causing some kind of problem because the root context must be created twice if there's a version mismatch because different event service instances end up being used...

backstage/plugins/events-node/src/service.ts

Lines 24 to 47 in 96ee519

	/**
	* The {@link EventsService} that allows to publish events, and subscribe to topics.
	* Uses the `root` scope so that events can be shared across all plugins, modules, and more.
	*
	* @public
	*/
	export const eventsServiceRef = createServiceRef<EventsService>({
	id: 'events.service',
	scope: 'plugin',
	defaultFactory: async service =>
	createServiceFactory({
	service,
	deps: {
	pluginMetadata: coreServices.pluginMetadata,
	rootLogger: coreServices.rootLogger,
	},
	async createRootContext({ rootLogger }) {
	return DefaultEventsService.create({ logger: rootLogger });
	},
	async factory({ pluginMetadata }, eventsService) {
	return eventsService.forPlugin(pluginMetadata.getId());
	},
	}),
	});

〽️ Alternatives

Persistent events service

These are maybe not really alternatives but more opportunities for extension given that the default implementation isn't supposed to support persistence.

RedisEventsService

Make use of Redis Pub/Sub - https://redis.io/docs/latest/develop/interact/pubsub/

AzureServiceBusEventsService

Make use of ServiceBus topics - https://learn.microsoft.com/en-us/azure/service-bus-messaging/service-bus-queues-topics-subscriptions#topics-and-subscriptions

AWS SNS

Make use of SNS Pub/Sub - https://aws.amazon.com/sns/

❌ Risks

NPM resolutions and Peer Dependencies could force breaking changes if plugin maintainers have not updated the package recently enough or in some cases it could force downgrade which could also cause problems.

👀 Have you spent some time to check if this RFC has been raised before?

• I checked and didn't find similar issue

🏢 Have you read the Code of Conduct?

• I have read the Code of Conduct

[pjungermann]

The limitation to one instance is known. We are thinking about providing a DB-based implementation. However, also that one will have its limitations.

The events service can be replaced by custom implementations. There might be some alternative implementations provided in the future like for Kafka, AWS EventBridge, etc.

Does some of the problem maybe come from here because the JSDoc says it uses the root scope but from what I can tell it uses plugin scope. Unless the createRootContext is what it means by root scope?

I would say the docs were not updated when it was changed from root to plugin scope. However, as you have mentioned already, it uses a root context. All plugin-scoped instances share the same. The plugin-scoped part is subtle (e.g., when a subscriber gets set up at a plugin, it uses the plugin ID as part of the subscriber IDs).

The issue with the different versions is very unfortunate... At least, we should highlight this in the documentation. There is a solution for frontend plugins I looked at when we discussed it. However, it couldn't be used in this context and so far, there is nothing yet for it.

The version mismatch issue is a more generic issue not limited to this service. However, this impacts inter-plugin communication.

It would be great to have a solution for backend plugins and services, too. Regardless of the considered DB-based implementation.

[marleypowell]

Thinking about it more, would this still be a problem if it didn't have a default factory?
That's mainly where the problem comes from because if you somehow have different versions it will automatically create one with the factory whereas if it was required to be set up by the application they would all sync based on that? From what I can tell the ServiceRegistry resolves based off of ID so as long as the IDs match it doesn't matter as much if you have different package versions.

Also another solution for now could just be to create your own service factory which overrides the default factory with the event service that the backend has.

From the tests different service refs are used but they share the same ID so they can be interchanged.

[Rugvip]

Ah, let's add the events service factory here to avoid this issue, it was the intention from the start:

backstage/packages/backend-defaults/src/CreateBackend.ts

Line 40 in 6ce253d

export const defaultServiceFactories = [

[marleypowell]

Ah, let's add the events service factory here to avoid this issue, it was the intention from the start:

backstage/packages/backend-defaults/src/CreateBackend.ts

Line 40 in 6ce253d

export const defaultServiceFactories = [

Should the factory live in @backstage/backend-app-api with the others?

[Rugvip]

@marleypowell no it's alright for backend-defaults to have a dependency on events-node, we can import it from there.

[marleypowell]

@marleypowell no it's alright for backend-defaults to have a dependency on events-node, we can import it from there.

I gave the fix a go #24376

[pjungermann]

@marleypowell as your PR was merged, I assume this issue can be closed, right?