bndtools should assist with importing projects from known bnd workspaces

[joe-chacko]

Problem description

In the bnd workspace model, when bnd is "in charge" of eclipse project dependencies, there are various usability issues:

1. Importing the projects can take a very long time.
2. If the projects are imported out of dependency order, errors can show up due to missing dependencies. Resolving these errors can take a lot of trial and error with cleaning, rebuilding, and even closing and re-opening projects.
3. Even once the projects are imported, the workspace can be slow to load, slow to rebuild, and very susceptible to incoming changes from version control that alter the dependencies.

Proposed solution

The eclipse bndtools plugin should offer a project import wizard that looks at a specific bnd workspace on the filesystem and allows the user to select which projects to work on. It would then import all the dependent projects too (as known to bnd), and optionally the using projects too (the ones that depend on the selection).

Proof of Concept

While working on OpenLiberty, we developed some tooling that used bnd to discover and navigate the dependency tree and allow the importing of related projects too. This brought down the time to start working on a new workspace with some projects loaded from several hours to 1 minute.

The eclipse plugin we created did the following as a project import wizard:

• asked the user to identify the bnd workspace
• discovered the projects in it
• presented these to the user to select from
• automatically imported depended-upon projects
• optionally imported projects that depended on the selected projects (including their dependencies)

Although our implementation is specific to OpenLiberty, the approach is easily generalised to the bnd workspace model.

Here is the (liberty-specific) workspace location dialog:

Here is the project import page. I have selected three projects to import, and requested the users of those projects be imported too:

One minute later, we had a working workspace, compiling without errors, with 144 projects imported:

[pkriens]

This is really good! Where is the frigging PR? :-) I like the idea!

Interested how you do this. You got one bnd/git workspace on location A and then open up different new Eclipse workspace on location B with this partial view?

There is a synchronize button on the Bndtools Explorer. This currently synchronizes with all the projects in the bnd workspace. This will need some adaptation so it won't drag in the rest. Your UI could be connected to this button though.

Looking forward to a PR.

[joe-chacko]

Cool — it will take me some time to port it, but good to know the idea has legs.

The screenshots shown are for a new Eclipse workspace. It is in a separate location from the bnd workspace.
Once you tell the import wizard where your bnd workspace is, it imports the cnf project automatically (as you can see in the background of the first screenshot).
After that, you select which projects to import from the bnd workspace into the Eclipse workspace.

Your comment has me thinking though — what do we do when the workspaces are the same filesystem location?
It should be doable, but it will need some consideration.

[pkriens]

I am working on a new bnd workspace wizard and the automatic syncing with Eclipse. This is awfully close. I wonder if we could integrate this?

My current focus is to use fragment templates that could be easily extended with a simple PR. They are fragments that just do one thing: gradle, maven layout, Java 17, OSGi R8, distribution. etc.

What I could so is start with a page that selects between the classic workspace templates (which should be deprecated over time), my fragment templates, and sub-setting an existing workspace (as you propose).

In the subset case, we only need a File to the bnd workspace and let the user select the projects it is interested in. You can only create/delete projects in the current instance of Eclipse so it will require acknowledgment to delete the Eclipse projects. With some changes to the WorkspaceSynchronizer, the project creation/deletion should be straightforward. It now compares against all projects in the bnd workspace but it should be simple to give it a list.

You have a fancy drag/drop UI. Although you got >1000 projects, I think this could be easily handled with a CheckboxTableViewer? When you click on the projects you want, we can immediately calculate which projects are included. I guess we need a search bar on top of it.

The workflow would then be. Open a new Eclipse workspace, create new bnd workspace wizard, select the sub-set, chose the bnd workspace, pick the projects, and sync against the list.

If you like this idea, could you make your sources available on github? Maybe we can work together on this?

[pkriens]

I've done some more pondering and came to the conclusion that adding it to the new bnd workspace wizard is not efficient.

Trying to see the use case I'd say a user creates an Eclipse workspace on a bnd workspace. We currently have the Bndtools Explorer sync action to synchronize this completely. What we need I guess is a button to show up your UI that allows you to efficiently select another subset of projects. It would then rearrange the current Eclipse workspace. as fast as possible.

Anyway, commit a PR and lets take it from there.

[joe-chacko]

Your new workspace wizard — is that to create a new BND workspace? Presumably the user already has an Eclipse workspace they are using at that point.

I will look into the checkbox table viewer. I'm not particularly wedded to the drag & drop UI. That was just what we came up with starting from a non-UI background (our first few iterations were command-line tools that kicked off Eclipse project imports).

[pkriens]

Yes, the new workspace wizard is always from existing workspace. However, I allow the selection of another bnd workspace and then use that as the Eclipse workspace.

The CheckboxTableViewer is a very easy way to select from a large input set.