How to keep bundled extensions up-to-date

[jakevdp]

Currently the vega and vega-lite extensions are bundled with the JupyterLab frontend. On one hand, this is great for projects like Altair (renderers are built-in! No extra installation necessary!)

On the other hand, when the extensions are evolving much faster than the frontend, this makes things quite tough on users: for example, Altair 3.0 was released a month ago, and requires the vega-lite 3 renderer that is currently only available in Jupyterlab 1.0.0a3. This has been a real point of confusion to users.

Now the Vega team is getting close to releasing vega-lite 4, and I'm anticipating a situation in which I need to say "To use Altair 3 on Jupyterlab, install version 1.0.0a3. To use Altair 4, install version 1.0.0a4 or later".

On top of the confusion for new users, I've heard of frustration from power users who don't want to have to re-install their jupyterlab frontend every time they switch versions of their charting library.

I have a phone call scheduled with @ellisonbg next week to talk (among other things) about how to best address this going forward; any ideas are welcome!

[ian-r-rose]

Thanks for bringing this up @jakevdp. Our position on this has been that the versions of vega and vega-lite that we support in JupyterLab core should be the most recent major versions. This, of course, can lead to version syncing issues, as you point out.

The good news is that different versions of the @jupyterlab/vega* package should be able to happily coexist alongside each other. We have gone to some lengths to keep the interface for renderer extensions stable so that they have to be updated as infrequently as possible (those interfaces are, and have been semver compatible with 1.x for a while now). So I am reasonably sure that users of JupyterLab 0.35 should be able to install the most recent version @jupyterlab/vega5-extension:

jupyter labextension install @jupyterlab/vega5-extension

If this doesn't work, I would consider it to be a bug, so we should check. The downside of this is that (1) it is still kind of confusing to users, and (2) It requires them to have node installed in their environment.

Note, if you are not aware, we maintain previous major versions of the vega extension here, and @domoritz has been pretty timely with keeping us up-to-date.

[jakevdp]

Thanks for the info - poking around I found a similar discussion from a year ago: #4695

[jakevdp]

Quick question regarding previous major versions: it looks like the vega 5 update PR added vega 5 to jupyterlab itself, but neither vega 4 nor vega 5 is present at the link you mentioned above: https://github.com/jupyterlab/jupyter-renderers/tree/master/packages

Should those versions of the extensions be there as well?

[jasongrout]

Setting this as Reference for now - if an actionable change comes out of the discussion, we can assign it back to Future or some other milestone.

Thanks for bringing this up! We faced a similar situation in ipywidgets, which is why the ipywidgets extension is still distributed outside of jlab.

[ian-r-rose]

@jakevdp Looks like there is an open PR for adding vega 4 to that repo which just needs to fix some CI issues before being merged, we can try to move that along.

The vega 5 extension is currently in the core of JupyterLab, but should be possible to install separately in JupyterLab 0.35

[ian-r-rose]

That is to say, even though the code for vega5-extension is in the core repository, it is an independent npm module that should be able to be installed as its own extension.

[jakevdp]

Thanks - I didn't realize it could be installed as a separate npm package. I'll update the altair docs with that info.

[ian-r-rose]

I just checked, and there is indeed a bug that prevents side-by-side versions of the vega plugins (a name collision in the plugin ID). This is fixable, but in the meantime, users of JupyterLab 0.35 can update their vega version by disabling the old vega plugin:

juptyer labextension install @jupyterlab/vega5-extension
jupyter labextension disable @jupyterlab/vega4-extension

[jakevdp]

Great, thanks for checking!

[domoritz]

@ian-r-rose it should be jupyter, not juptyer.

[davidanthoff]

I have a slightly different but related question: are older versions of the vega extension shipped by default with jupyterlab? Looking around, it looks as if the latest master of jupyterlab might only ship vega5 and vega-lite3, but no older versions?

I understand that one can manually install those older versions as an extension, but I'm wondering whether it wouldn't be better to ship them out of the box as well?

Here is the scenario I have in mind: say a user created a jupyter notebook half a year ago that has some vega-lite figure in it (created by e.g. VegaLite.jl, or I assume altair). The embedded vega-lite figure will use a MIME version that is older than the one for vega-lite3. So now, they update their jupyterlab to 1.0.0, and try to open that file, and all of a sudden their notebook no longer works the way it did half a year ago. That seems really not ideal?

Is there any reason to not just ship support for any vega and vega-lite version out-of-the-box? Anything else to me at least seems like a breaking change that could be really quite disruptive for users.

[davidanthoff]

Just as a comparison, it seems that nteract plans to ship support for all currently existing vega and vega-lite MIME types by default, see nteract/nteract#4442. That does strike me as the most user friendly and sensible option.

We do the same for the Julia extension for Visual Studio Code and another figure display for julia called ElectronDisplay.jl.

[davidanthoff]

From my point of view the most important question is whether this could still be sorted out before 1.0.0 is released? I think it should, because from a user point of view this is really a regression.

Would it be enough to move vega2-extension, vega3-extension and vega4-extension into this repo here, and then somehow configure them to ship out of the box? I can't judge whether that is difficult, or how much work that is...