Extensions - Add/pre-download legacy addons to Extensions "Core" addons

[Draise14]

Right now, Extensions is now released and official. This means a few things.....

This means on download, you are stripped bare. Bforartists should have atleast the same features from before, so we should move or copy or download latest versions of all legacy addons so they are "pre-downloaded" and included in the default installation (without needing to go onto the internet to get.. node wrangler, wth).

Task

• Add legacy addons or extensions variants to the Core repository
• Remove old addon folders from repo

Details

SHA-1: 13a3603

• Move addons from the addons repo to "addons_core"

Ref: !121825

Details

SHA-1: c4a0bbb

• Extensions: Support online extensions and move add-ons outside Blender

The extensions system allows to extend Blender with connectivity to the internet. Right now it means Blender can
discover and install add-ons and themes directly from the internet, and notify users about their updates.

By default this is disabled (opt-in), and users can enable it the first time they try to install an extension or visit
the Prefences > Extensions tab. If this is enabled, Blender will automatically check for updates for
extensions.blender.org upon startup.

When will Blender access the remote repositories:

• Every time you open the Preferences → Extensions: ALL the enabled repositories get checked for the latest info (json)
• Every time you try to install by dragging: ALL the enabled repositories get checked for the latest info (json).
• Every time you start Blender: selected repositories get checked for the latest info (json).

---

From the Blender code point of view, this means that most of the add-ons and themes originally bundled with Blender
will now be available from the online platform, instead of bundled with Blender. The exception are add-ons which are
deemed core functionality which just happened to be written as Python add-ons.

Links:

• Original Extenesions Platform Announcement: https://code.blender.org/2022/10/blender-extensions-platform/
• Extensions website: https://extensions.blender.org/
• User Manual: https://docs.blender.org/manual/en/4.2/extensions/index.html#extensions-index
• Technical specifications: https://developer.blender.org/docs/features/extensions/
• Changes on add-ons bundling: https://devtalk.blender.org/t/changes-to-add-on-bundling-4-2-onwards/34593

---

This PR does the following:

• Move extensions out of experimental.
• No longer install scripts/addons & scripts/addons_contrib.
• Add scripts/addons_core to blender's repository.

These add-ons will still be bundled with Blender and will be always enabled in the future, with their preferences
moved to be more closely integrated with the rest of Blender. This will happen during the remaining bcon2 period.
For more details, see #121830

From scripts/addons:

• copy_global_transform.py
• hydra_storm
• io_anim_bvh
• io_curve_svg
• io_mesh_uv_layout
• io_scene_fbx
• io_scene_gltf2
• pose_library
• ui_translate
• viewport_vr_preview

Extra: bl_pkg (scripts/addons_contrib)

Note: The STL (legacy) add-on is going to be moved to the extensions platform. There is already a C++ version on core
which is enabled by default.

All the other add-ons are already available at extensions.blender.org. To use them you need to:

• Go to User Preferences > Extensions
• You will be greated with an "Online Extensions" message, click on "Enable Repository".
• Search the add-on you are looking for (e.g, Import Images as Planes).
• Click on Install

Over time their maintaince will be transferred over to the community so their development can carry on. If you used to
help maintain a bundled add-on please read: https://devtalk.blender.org/t/changes-to-add-on-bundling-4-2-onwards/34593

Ref: !121825

[Draise14]

Adding in online extensions to core is.. gonna be tricker than I expected. Extensions are stored to USER, not SYSTEM, so I can't ship an already downloaded extensions.. UNLESS... hmmmm

Why?

1. If copied, it will be listed as "Core", dubbling up. You shouldn't have two of the same addon at once, double registrations bug out
2. Downloaded is to USER, not SYSTEM, so we can't ship an already downloaded addon till a user has "saved" user preferences and made the config folder

Ideally we can shift the extensions to store to SYSTEM instead of USER and then we can ship already downloaded and curated addons that would "update" when user wants to or activate without downloading when they want to.

[Draise14]

Got an idea.. maybe we don't need to change core code and make a "bundled" addon that is pre-activated and writes these already downloaded extensions and datatbase of addons to the config folder on first use. In this case, these addons are "installed" due to the other addon, making migration easier and less buggy.

Right now these core default addons missing is causing migration issues: user_pref.blend file defines X default addon from 4.1, user migrates from 4.1, the user_pref.blend file is missing addon scripts, user has to install manually VS something has the X default addon already downloaded and installed, ready to be updated, and when the user migrates everything is fine, ready to activate, ready to uninstall (if you don't want bloat) and ready to update.

[Blenux]

Yeah still trying to wrap my head around this one.

[Draise14]

Looks like I'm not the only one finding this issue, so reported it to Blender directly with steps to reproduce.
https://projects.blender.org/blender/blender/issues/121917

[Draise14]

Looks like some of our addons have changes, meaning we have "forks" of addons that would ideally need to be shipped and most likely can't be included into "Extensions" website.

But this also means we would have a Bforartists version of the same addon, meaning we need to do this:

1. Find a way to include them as core extensions
2. Rename the registries so they don't conflict if a user activates both
3. Make it clear that there could be user error by using the online version
4. Maintain the forks somehow (similar to what we do with code comparisons, but now pulling is gonna be weird)

We could try to keep the work a minimum by only including the addons we changed, and none else?

[Draise14]

Legacy Addons Bundle is now an addon that will ship, meaning we can ship it too with our changes.

https://devtalk.blender.org/t/changes-to-add-on-and-themes-bundling-4-2-onwards/34593

EDIT: not really and addon, just a bundle.

[Draise14]

Looks like I found a way to do this with an addon that could improve things. Also did a poll, it's unanimously in pro of keeping and pre-downloading addons. Lot's of users from Blender have voiced their feedback, but a week later there still is no news of if they will add things like this...

• Bforartists can install legacy addons, but won't detect them at first.
• I can make an addon that will install legacy addons by default on first use, so if a user migrates settings it won't bug out
• I can make the addon also remove legacy addons once a user opts in to extensions by modifying the extensinos addons
• I can technically "pre-download" extensions which can contain come pre-shipped with Bforartists, meaning users can update and use extensions from a legacy database

Workflow

1. If no extensions activated (default installation)
   a. Have the bfa_default_addon activated by default, with a copy of the legacy addons in the bfa_default_addon folder
   b. Create user addons folder if it doesn't exist
   c. Copy legacy addons from source to user
   d. On addon activation - if the folder does exist, don't worry about it (assume the addons or are now extensions)

2. If extensions enabled (by user opt-in with the bl_pkg addon)
   a. Remove legacy addons (by comparing the addon folder contents) to not get doubles
   b. Copy in the pre-downloaded extensions from the bfa_default_addon, no need to be online to activate
   c. User can opt-out of any addon

Next Steps

• Rebuild our database of legacy addons
• Predownload the extensions for equivalents
• Add the extensions legacy replacer system (theory works)
• Apply our changes to the extension equivalents or "replace" as core Bforartists versions (and occasionally update from source when community updates these addons)
• Test, test, and test again, specially with migrations

[ReinerBforartists]

I would still love to see a way where the users does not need download an extra bundle. We are not bound to the Blender way. We can ship this bundle directly with Bforartists.

What worries me more is to keep this bundle updated then. Previously this was done with the merge already. Now we would need to download the bundle and compare it with our bundle regularly.

[Draise14]

Success, legacy is there.

Maintenance will be curious....

1. Download Extensions
2. Compare Extension to Legacy in our new repo
3. Update Pre-downloaded Extension + Legacy

[Draise14]

💻 Progress for default legacy addons:

1. Now they install to USER (means you can opt-out if you want)
2. They work from offline, so no need to connect to internet
3. Migrating settings from 4.1 will not error if you had settings with legacy addons

Next step: extensions pre-downloaded opt-in

nlmy4Gu1kX.mp4

EDIT: I am also wondering if I should not pre-download the modified extensions, and show a warning that we made changes to those addons.

[CHANGES]
achm_room_maker.py
edit_dimensions.py
io_mesh_dxf\
io_mesh_stl\
io_mesh_uv_layout\
io_scene_3d\
meshinset\
meshedgetools.py
minilightlib\
space_view3d_3d_navigation.py
space_view3d_copy_attributes.py
space_view3d_pie_menus\

[REMOVED]
render_freestyle_svg.py

[ReinerBforartists]

That's the problem now. We should fork these addons now, and make sure they have no conflicts with the Blender ones: Means other class names and such. I have no solution to this dilemma.

[Draise14]

💻 Eureka! Pre-downloaded blender extensions (thus we can fork them for every release) and get users to opt-in without needing to download all the legacy addons one by one! The prototype works. Opens many doors, could even make a pre-downloaded curated list, that can update.

Next steps:

• clean up the code

Plan A:

• modify the pre-downloaded extensions to contain our changes
• add user warnings if they plan to "update" our extensions saying they will loose our changes, with a hint that they are updated every release

Plan B:

• Add our forked addons as core legacy addons or pre-downlaoded extensions (that can't be updated) and re-write them to not conflict if a user "doubles up" the same addon (like two 3D view pie mienus, the Bforartists one and the blender one), as core.
• Only setup and pre-download the addons that haven't been forked.

[Draise14]

Here is the new Bforartists Extensions workflow:

Problem:

Migration from 4.1 to 4.2 will break, user won't know why (unless in the know)

More problems: User might not have a good internet connection or no connection at all, but they use legacy addons

Solution:

Pre-install default legacy addons
Pre-download/install default extensions

A. When a user migrates 4.1 settings, there will be no bugs (missing scripts), they can get to work immediately (addon pre-install legacy addons)
B. When a user opts into extensions, they don't need to download everything that was default, it's already there. It can still work offline (installed pre-downloaded extensions)
C. When a user sees legacy vs extension double up, they can then opt-out of all legacy (from addon) or on a case by case bases (opt out of legacy, activate extension) to "ease the migration to the new" instead of being forced to solve.
D. When they opt-into extension equivalents, they won't need to download the pre-selected defaults, only activate (can work on limited internet).

Core forks of addons are updated every release and can't be updated remotely. Core forks have warnings ("Bforartists version"). They are exempt from Legacy.
Legacy can be removed with "all" and/or removed individually
Extensions can be removed individually also.

Eventually we can phase out the legacy addons once Blender does (if they ever do) and still use the addon to "pre-download" extensions without the internet opt-in.

Interface

• Clear instructions
• Clear UX warning
• Install/Remove all legacy addons (opt-out) vs obligated opt-out, no user choice.