Multitenancy #5159
Draft
Multitenancy #5159
+15,960
−9,665
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* feat(multi-tenancy): add tenantId to entities and additionnal configuration to dbContext * feat(multi-tenancy): split code in a Elsa.Tenants project, add some configuration to the DbContext for strategies * feat(multi-tenancy): fix queryfilter to split data between tenants * feat(multi-tenancy): add tenantId to entity, generate migrations and create a sample * feat(multi-tenancy): add strategies to always have tenantId when saving * feat(multi-tenancy): add external user provider support for tenant * feat(multi-tenancy): fix dbcontext filter on tenantid * feat(multi-tenancy): manage background execution of workflows * feat(multi-tenancy): change tenant accessor and middlewares * feat(multi-tenancy): fix tenantId missing in some cases * feat(multitenancy): fix efcore store for multitenant * feat(multi-tenancy): fix some comment of the pullrequest (naming, use Sqlite for example, etc.) and split efcore from tenant project * Move Elsa.Samples.AspNet.Tenants and Elsa.Samples.AspNet.Tenants.External projects * Refactor tenant middleware and enhance code documentation Multiple files including middleware and entity classes related to tenants have been refactored for more straightforward implementation. Unnecessary code has been removed and constructor parameters have been directly utilized, increasing the code's readability and efficiency. Moreover, documentation for classes and interfaces has been enhanced providing better understanding of their role and function. * Refactor tenant-related classes and move to Elsa.Tenants module Moved ITenantAccessor and related classes from Elsa.Common to Elsa.Tenants module. Adjusted necessary references in files where these classes were used. Made corresponding changes to the various services where these classes were instantiated or used. Updated project references accordingly. * Refactor DbContext strategies and streamline code The DbContextStrategy interfaces were moved from Abstractions to Contracts for better semantics. Async methods in ElsaDbContextBase and IBeforeSavingDbContextStrategy were made synchronous to simplify usage. This included the alteration of methods in related classes to reflect the changes. Unnecessarily complex lines of code were also rewritten for better readability and standardization. * Refactor multi-tenancy feature and rename workflow provider interfaces The update refers to multi-tenancy and workflow providers. For the multi-tenancy feature, the 'ConfigurationTenantProvider' was refactored to 'ConfigurationTenantsProvider' and used in 'TenantsFeature'. It was consolidated into one function 'UseConfigurationBasedTenantsProvider'. Middleware to handle this scenario was also adjusted. Regarding workflow providers, the 'IWorkflowProvider' has been renamed to 'IWorkflowsProvider' and corresponding changes made in classes consuming this interface. Name adjustments also affected functions and configuration options. Various non-related code cleanups and removals have also been made. * Replace initial migration files for various databases The replacement ensures that TenantId columns will be added when users migrate to 3.1. * Remove old migration files This commit removes multiple old database migration files from the source. These files are related to the Elsa.EntityFrameworkCore module and they cover database alterations and management. They are no longer needed due to changes in the application's data structure and updates in the database schema. * Update database model and migration snapshot This commit updates the database model and migration snapshot in Elsa.EntityFrameworkCore.MySql. The "ProductVersion" annotation has been updated, and the "Status" property's type has been changed from int to string. Furthermore, a "TenantId" property has been added which is indexed for faster lookups. * Add TenantId to app settings This update modifies the appsettings.json file to include a "TenantId" for Administrators, Users, and Applications. Moreover, the key names are updated to follow the PascalCase convention consistently. * Refactored workflow command handler and added indexes to entities The DispatchWorkflowRequestHandler has been replaced with DispatchWorkflowCommandHandler in the WorkflowRuntimeFeature. Also removed the DispatchTenantWorkflowRequestHandler from the TenantsFeature. In the entity framework, added indexes to SerializedKeyValuePair's Key and TenantId in the Configurations. The SerializedKeyValuePair class now inherits from Entity. * Reset migrations to Initial * "Added TenantId column and altered status column type in AlterationPlans and AlterationJobs tables" This commit introduces new TenantId columns to the AlterationPlans and AlterationJobs tables across different DB contexts (MySql, SqlServer, Sqlite, PostgreSql). The type of the Status column has also been changed from int to string in both tables. Indexes for the new TenantId column are also created as part of this change. * Update authentication settings and improve code structure Several modifications have been made to improve our code and update our authentication setup. Made changes to disable Dapper and utilize JWT Bearer for authentication in the Elsa.Server.Web program settings. Additionally, we've updated a method on IdentityTokenOptions to be internal and adjusted token option assignments in ModuleExtensions. Syntax improvements were also made in ApplicationProviderExtensions for better consistency with best practices. * Remove V3_1 migration from different databases The V3_1 migration from MySql, SqlServer, Sqlite, and PostgreSQL databases has been removed. This includes alterations such as the addition of the "TenantId" column and changes in the "Status" column type within Elsa's schema. * Remove old and add new tenant handling components Removed files related to the old way of handling tenants in the application and added new ones for improved management. This change introduces a tenant resolution strategy pipeline, allowing different strategies to be used in sequence to resolve the current tenant. Various schemes have been implemented, including resolving tenants based on route prefixes, the currently authenticated user, and the user's claims. The update also enhances the way entities are handled when working with SQLite and Oracle databases. * Added SQLite setup to PersistenceFeatureBase The code now adds a line to include SetupForSqlite in PersistenceFeatureBase's services list. The SetupForOracle class was removed from the SetupForSqlite file - it is possibly now dedicated to SQLite setup only. * Add CommonPersistenceFeature, refactor related classes Implemented a new class, CommonPersistenceFeature, which handles aspects previously managed in PersistenceFeatureBase. ServiceScope registrations were removed from PersistenceFeatureBase and added to the new CommonPersistenceFeature. In addition, the tenant filter in SetTenantIdFilter has been simplified for more direct accessibility. Changes were also made in WorkflowManagementPersistenceFeature to accommodate these implementations. * Refactor multi-tenancy configuration and resolution Updated the code to use "MultiTenancy" section instead of "Tenants" section in all configuration files. Also, refactored the Store class to use IServiceProvider instead of IDbContextFactory and added tenant resolution logic. Renamed classes and variables accordingly to better represent multi-tenancy. * Add Tenant Dymanic Filter options for Workflow Definition Tenant Dymanic Filter provides the ability to include or exclude tenant matching in the filter. The corresponding queries and method signatures were modified to account for this new feature. Added `HttpContextTenantExtensions` and `HttpContextTenantResolver` classes to enable tenant-based filtering of HTTP context. --------- Co-authored-by: Jean-Baptiste Dalle <jean-baptiste.dalle@stereograph.fr> Co-authored-by: Sipke Schoorstra <sipkeschoorstra@outlook.com>
The AuthenticatingMediator, GetPlacedCallHubSpotId activity, and IAuthenticatingMediator interface have been removed. Middleware for authorization context workflow execution and tenant providers have been added to enhance multi-tenancy support.
Introduce tenant resolution strategies in the Identity module and refactor namespaces across modules to align with the new multi-tenancy structure. This includes moving various tenant-related classes from the Tenants module to Common and Identity, adjusting references, and consolidating tenant constants and options.
Deleted `IdentityOptions` and moved `TenantIdClaimsType` to `IdentityTokenOptions`. Renamed `ClaimConstants` to `CustomClaimTypes` and updated references across multiple services and samples accordingly. Adjusted `ClaimsTenantResolver` to use `IdentityTokenOptions` and removed deprecated method `Deconstruct` from `IdentityTokenOptions`.
Changed the dependency injection scope for ISystemClock from scoped to singleton to ensure a single instance is used application-wide. This ensures consistent time references across different components and enhances performance by reducing the number of instances created.
The `IQueueProvider`, `ITopicProvider`, and `ISubscriptionProvider` are now registered as singletons to ensure single instances throughout the application. Additionally, dependency resolution for `ITriggerStore` and `IBookmarkStore` is moved to a scoped lifetime within `StartWorkers` to maintain correct scope per operation.
Simplify the extraction of tenant ID from claims by removing the fallback to a default type, given that it is non-nullable.
The import of Elsa.Identity.Constants was removed as it was not used in the ClaimsTenantResolver class. This cleanup helps maintain code clarity and reduces unnecessary dependencies.
Removed several deprecated workflow activities pertaining to lead and deal updates in preparation for new workflow structure implementation.
A default value was added to the WorkflowDefinitionModel constructor to align with the expected parameter count. Additionally, removed unused tenant-related code from ElsaDbContextBase and redundant entity configurations from DbContext specific to Oracle, simplifying the model setup.
Adjusted the type passed to the DefaultWorkflowDefinitionStorePopulator constructor to use the correct IWorkflowsProvider interface. This change aligns the type expectation with the implemented providers list, ensuring proper dependency resolution.
The model snapshots for various database providers have been updated to reflect the new product version 7.0.14. Additionally, new properties and indices for `IsSystem`, `TenantId`, and compression information have been introduced across entities such as `WorkflowDefinition`, `WorkflowInstance`, `Label`, and others to support multi-tenancy and system-level workflows.
Migration V3_1 adds a tenant ID column and indexes, and changes the status column's type to a string across all database providers. It also renames the SerializedWorkflowInstanceIds to SerializedWorkflowInstanceFilter.
Updated migration versions and internalized records to extend from a new base Record class with tenant ID property for multi-tenancy. Added TenantId across stores and migrations to support tenant-specific data isolation within the Dapper module.
Introduced TenantId to various tables and records to enable multi-tenancy. Migration scripts have been added across modules for managing TenantId columns effectively.
Optimized TenantResolver for caching current tenant, refactored MongoDB stores to include tenant resolution, and updated index creation to add tenant-specific indices. Enforced multi-tenancy across various Elsa modules ensuring tenant isolation and efficient data retrieval.
Changed usage of BookmarkId to Id across several files, ensuring consistent referencing of the bookmark identifier for clarity and future deprecation of obsolete properties.
Provide an option for tenant-agnostic operations across all MongoDB store methods, including Find, Count, Any, and Delete operations. Default behavior respects tenant boundaries, but adding a flag allows operations to ignore tenant constraints. Removed unnecessary using directive.
Added Dapper persistence stores for various entities and refactored existing stores for better maintainability.
Introduced an option to perform tenant-agnostic queries in Elsa Dapper, allowing retrieval of records without tenant filtering. Refactored query methods to include a tenantAgnostic parameter and applied tenant filters conditionally based on this flag. This supports scenarios where tenant-specific data isolation is not required.
Changed the code to asynchronously resolve the tenant and handle possible null values before assigning the tenant ID. This improvement ensures tenant resolution is done properly and enhances code reliability by checking for nulls.
Refactored Mongo collection references to use IMongoCollection interface across various modules. This change allows for more consistent and flexible database operations. Additionally, updated program configuration to enable MongoDB and disable Dapper.
The workflow file ac-call-ring-group.json has been deleted, as it is no longer needed in our project. This may be due to changes in requirements or workflow optimizations.
Removed outdated migration scripts (efcore-3.0.sh, efcore-3.1.sh, efcore-3.1-sql.sh) and added new versions (generate.sh, generate-sql.sh). Also optimized the dependency list in Elsa.Tenants.csproj - switched individual package references into a single reference to Microsoft.AspNetCore.App. This streamlined the project's dependencies and reduced redundancy.
The commit removes the WorkflowGrainSnapshot and WorkflowInstanceGrainSnapshot classes from the Elsa.ProtoActor module. These classes were part of an older architecture and are no longer required.
The _refCount field in the Worker class from the Azure ServiceBus module was unused. To maintain a clean codebase and improve readability, this field has been removed.
The commit includes the deletion of the entire WorkflowInboxMessageRecord class from Elsa.Dapper module. This file was managing various activities related to the workflow inbox messages, such as delivering messages to a workflow instance.
This commit introduces new V3_2 migrations for MySQL, SQLite, PostgreSQL, and SQLServer database providers. Included are `Up` and `Down` migrations, the corresponding Designer files, and specific Alterations and Runtime changes. SqlDbType specifications were also added for each.
The changes update the namespaces of various files and refactor code for efficiency and readability. In addition, some unnecessary comments and redundant namespaces were removed, and import statements were corrected. Functions were also simplified by removing unneeded parameters and streamlining logic.
The refactor operation includes the removal of 'Contracts' from the namespace under the Elsa.Tenants module. This modification affects several files across different modules where the namespace is being referenced. In addition, a new folder for Multitenancy scenarios was created in the Elsa.Workflows.ComponentTests project.
|
Hi @Itzalive Thank you for trying out the feature so early and for offering your concern. The mediator handler should have the same context as the code that fires the notification or command. If this is not the case, can you provide me with an example of a case where this context gets lost? |
The commit includes removal of EntityFrameworkCore database migrations files across various databases such as MySql, PostgreSql, Sqlite, SqlServer etc. The V3_2 migration files were presumably outdated, unnecessary or causing issues in the project.
This update adds a new 'TenantId' column to several tables in the database including 'WorkflowExecutionLogRecords', 'Triggers', 'KeyValuePairs', 'Bookmarks', and 'ActivityExecutionRecords'. It also creates indices for this new column in each corresponding table. This enhancement is added across various database types including MySQL, SQL Server, SQLite and PostgreSQL.
In the database migration script, the "TenantId" column has been added to the Triggers, Bookmarks, WorkflowExecutionLogRecords, ActivityExecutionRecords, and KeyValuePairs tables. Also, the process of removal of the same column from these tables has been scripted for rollback scenarios.
The commit contains an update in the version number of the migration Elsa:Runtime:V3.2 in V3_2.cs file. The previous version was 20002; it has now been updated to 20003.
The update introduces tenantID support to the workflow builder. This allows workflows to be associated with different tenants, improving the multi-tenant support. The changes include adding a `TenantId` property and a `WithTenantId` method, and adjusting the workflow identity creation to include the tenant ID.
The WorkflowServer now supports multi-tenancy. Added `UseTenants` extension method where new TestTenantsProvider is configured which will provide multiple tenants. A TenantResolutionStrategy is appended to tenant options to decide the tenant context in runtime. Also, `ITenantResolutionStrategy` is registered in the dependency injection container.
Added test scenarios for multitenancy in the Elsa workflow component. Utility classes for tenant resolution strategy and tenants provider are also added to help perform the tests. Different workflows for different tenants are set up for testing.
…rvice An additional catch block has been added to handle OperationCanceledException within the BackgroundEventPublisherHostedService. This provides a more specific message when an operation is cancelled during the queue processing.
Empty lines were removed between property declarations to improve readability in WorkflowDefinitionFilter. An error where 'filter.IsReadonly' was used instead of 'IsReadonly' was also corrected to ensure proper functionality. Additionally, WorkflowDefinitionFilter was imported in the Delete and Revert endpoint modules of the Elsa.Workflow.Api class to maintain consistency across the codebase.
The MongoUserStore class was previously tagged as an abstract class in the Identity module of Elsa.MongoDb. This update changes the MongoUserStore class from an abstract to a non-abstract (concrete) class to enable direct instantiation.
This commit introduces a new `ForwardedType` attribute to aid in forwarding types to new types when deserializing JSON. Additionally, the existing bookmarks related to workflow runtime activities have been updated. These changes also require modifying the `PolymorphicSerializer` in the Elsa.MongoDb project to handle types using the new `TypeHelper.GetLatestType` method.
Fixed typographical errors in comments of the "JobId" property in both BackgroundActivityStimulus and BackgroundActivityBookmark classes. This minor change ensures accuracy and clarity in the code documentation.
This commit modifies the import statements in various files under the Workflow API endpoints. Most changes involve replacing Elsa.Workflows.Management.Contracts import with Elsa.Workflows.Management. This refactoring ensures the correct classes and interfaces are used from the updated module path. Few methods have additional parameters added as well, enhancing their functionality.
Response to Multi-Tenancy Configuration QuestionsScenario 1: Disabling Multi-Tenancy After Being EnabledTechnical Impact:
Scenario 2: Enabling Multi-Tenancy on an Existing Data SystemTechnical Impact:
|
This commit introduces a new DefaultTenantResolver class which always returns the default tenant. The previous TenantResolver is renamed to PipelinedTenantResolver. The ITenantResolver interface has also been moved to a more general namespace. All services using ITenantResolver are updated accordingly. Furthermore, TenantResolverFeature is now a dependency of WorkflowsFeature.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds "shared database, shared schema" multitenancy capabilities.
This change is