Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DSIP-38] Add plugin directory in binary package #15960

Open
2 tasks done
Tracked by #14102
ruanwenjun opened this issue May 9, 2024 · 0 comments
Open
2 tasks done
Tracked by #14102

[DSIP-38] Add plugin directory in binary package #15960

ruanwenjun opened this issue May 9, 2024 · 0 comments

Comments

@ruanwenjun
Copy link
Member

Search before asking

  • I had searched in the DSIP and found no similar DSIP.

Motivation

In the previous binary package, the plugins are put at server/libs.

- dolphinscheduler-bin
   - api-server
     - bin
     - conf
     - libs 
   - master-server
     - bin
     - conf
     - libs 
   - worker-server
     - bin
     - conf
     - libs
   - alert-server
     - bin
     - conf
     - libs

This will bring below problems:

  • Multiple copies of the plugin exist within different services, cause the binary package is very big.
  • Multiple copies of the plugin exist within different services, cause if we want to replace a plugin we need to replace it at different service.
  • It's difficult to support different version plugins, e.g. hive2, hive3

Design Detail

Firstly, we need to add a plugin directory, the binary package will be

- dolphinscheduler-bin
  - api-server
  - master-server
  - alert-server
  - worker-server
  - plugins
    - task-plugin
    - alert-plugin
    - storage-plugin
    - registry-plugin
    - xx

Then we need to shade the plugins and put them in target plugins.

The server should only rely on plugin-api, cannot rely on plugins, and use child first classloader to load plugins, each plugin will have a classloader.

Compatibility, Deprecation, and Migration Plan

Compatibility with the previous version

Test Plan

Test by manual/e2e/UT

Code of Conduct

@ruanwenjun ruanwenjun added Waiting for reply Waiting for reply DSIP discussion discussion and removed Waiting for reply Waiting for reply labels May 9, 2024
@ruanwenjun ruanwenjun self-assigned this May 9, 2024
@ruanwenjun ruanwenjun added 3.3.0 minor Chore Chores about the project, like code clean, typo, etc. major priority:middle and removed minor Chore Chores about the project, like code clean, typo, etc. labels May 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant