Adding Option to Customize Parameters when Launching & Changing Kernels

[desmondyeoh]

Problem

We were using Jupyter Enterprise Gateway to spawn remote kernels. It would be great if JupyterLab could expose an extra text field in Launcher & Choose Kernel form to let users specify additional arguments for the kernel (e.g. the CPU, RAM & GPU resources they need). And the options in this field will be passed together in the request body of API call to /api/kernels.

As far as I know, the Jupyter Enterprise Gateway has already been configured to take in any KERNEL_* parameters in the request body. (See https://gitter.im/jupyter/enterprise_gateway?at=5f9c2a1ed5a5a635f28e9645)

Proposed Solution

• Add an extra text field for user to enter customised configuration when user is (1) Launching kernel from Launcher and (2) Changing kernel
• Pass the value of that text field as KERNEL_EXTRA_CONFIG to request body when calling /api/kernels
• Optional: It would be nice if we could customize the form (e.g. making it to be dropdown) but the text field is sufficient.
• Optional: Preprocessing the text field, splitting by , and turning them into KERNEL_NUM_GPU, KERNEL_CPU and KERNEL_RAM in request body

Additional context

Similar feature in Google Colab to specify resources:

Thank you

[mlucool]

Moving #9528 here per @echarles @jasongrout

Problem

Jupyter assumes that there is some small number of variants of kernels I may want, so I should configure them on the command line and reuse them between notebooks. In practice this is only partially true; I find there are many parameters I wish I could pass to the kernel when creating a notebook (e.g. memory limit, environment variables, parameters to tell it where to find a host or reserve a GPU). We should make kernels drastically easier for users to parameterize.

Proposed Solution

Create an API where kernel can be parameterized on the fly (this is an extension of the argv already in the kernel.json)
Create a small, extendable UI for common operations for all kernels (e.g. env variables).
a. Taking this one step further, kernels could include opt-parse like data so the UI can let a user enter options in a constrained way along with descriptive help. If the UI is extendable, then the this is less important for me.
b. Similar to kernel name, all kernel parameters should be stored in the notebook metadata.

A generalized google hardware accelerator-like popup would be awesome.

[blink1073]

cc @kevin-bates for how this might impact jupyter/jupyter_client#608

[kevin-bates]

Thanks for the copy @blink1073.

By all means, Kernel Environment Provisioning will allow for the specification of parameters. However, I'd prefer we move away from EG's stop-gap of using KERNEL_-prefixed environment variables - although that would work fine provided that the backing environment provisioner knows how to deal with those values (which EG-based environment provisioners would).

Since only the environment provisioner knows what parameters it can support, I'd prefer we adopt a proposal similar to Parameterized Kernel Launch. This proposal had been based on the "Kernel Providers" proposal and should be updated/re-submitted relative to environment provisioners (if that gets adopted within jupyter_client).

In that proposal, the kernelspec will contain a schema consisting of the supported parameters (i.e., capabilities). This schema should be rich enough for consumers to produce sufficient dialogs for specifying parameters. The REST API would then need to be plumbed to take the specified parameters - although this should just be a matter of adding the dict of parameter values to the request body. Ultimately, the parameters make their way to the kernel environment provisioner which will apply the parameters and/or pass the remaining parameters to the kernel.

What seems to be missing in this issue is the ability to determine the capabilities of the target kernel. For example, how does the front-end know that this particular kernel can support additional CPUs? And, in the absence of any kind of provisioner mechanism, what interprets the meaning of KERNEL_EXTRA_CONFIG or KERNEL_NUM_GPU should they get conveyed to the kernel's launch?

[anirrudh]

@kevin-bates, I was pointed to you as I am currently working with @Carreau on a JupyterLab Extension to manage iPython Kernels.

I presented it at last week's Jupyter Developer meeting, and got some feedback regarding the flow and use cases.

In any case, I will be sending an email so that we can have a video chat to update where this work is at - and then we can update this issue accordingly!

[telamonian]

I recently started doing some experimentation of my own with dynamically configurable kernels. It's nice to see that there are other folks who are starting to see the possibilities.

Since only the environment provisioner knows what parameters it can support, I'd prefer we adopt a proposal similar to Parameterized Kernel Launch.

@kevin-bates +100 for the suggestion of using something JSON schema based. Strong typing, good support for constraints and/or validation, and a widely known syntax/semantics that we don't have to design and implement ourselves all seem like huge wins to me. It's sad to see that this proposal has been languishing since 2019. A good implementation would have probably saved me weeks worth of work I spent last year on figuring out how to hack a poor implementation into enterprise-gateway.

[kevin-bates]

Thanks Max - it's good to see parameterization gaining some traction.

For parameterization to be useful, I believe we need a mechanism like kernel provisioners - which are responsible for publishing at least a portion of the parameters - with the others being kernel-specific. So you'd essentially have two sets of schema returned from the api/kernelspecs endpoint, one pertaining to what the actual kernel supports (and probably settable via substitutions, but that's something we'd want to define for kernel and kernel provisioner authors) and one pertaining to the kernel provisioner (which reflect the environment in which the kernel will run and which is provisioned by the kernel provisioner). I suspect most of the common parameters (cpus, gpus, num-workers, memory) would likely fall into the latter set.

[anirrudh]

@telamonian I am getting ready to release a beta of the KernelSpec Manager Extension later today. I have a small bug that I need to fix while setting environment variables - the UI uses json schema to generate the components, and I need to fix the corresponding schema for a python dict (the typings are a little off).

I'll leave a link to the repo here, and I plan to update it this weekend, after releasing a beta sometime between today/tomorrow.

[telamonian]

two sets of schema

@kevin-bates Those are my thoughts exactly. What I really want to see is some concept of composability (along the lines of react's higher-order-components) and/or reusability (hierarchical merge of kernelspecs, actual inheritance, etc) brought into the kernel specification space. It sounds like your current plans for a provisioner that (in one way or another) "wraps" the kernel execution specified by a kernelspec sounds like a great example of composable parts.

I would like to know more about the details about the implementation of providers, and about exactly how they can interact with kernelspecs. I can kind of see two different possible models:

• a provisioner defines a set of lifecycle hooks that each fire at a well defined point in the lifecycle of the kernel execution. There'd have to at least be an entry and exit hook that run before kernel start and after kernel termination, respectively. But there's likely to be other good places to hook in as well
• similar to ProcessProxy, each provisioner is completely responsible for managing the entire lifecycle of the kernel under it, and is free to execute arbitrary code to do so

I definitely favor the hook-based provisioner, but with all of your very complex ipython + kube + spark kinds of kernels there will probably end up being a need for the arbitrary execution provisioner as well. So maybe it makes sense to have both? I'm sure you've had a chance to think through all of these details yourself already.

In any case, I'll take a look at your ongoing work on jupyter/jupyter_client#612. It's great to see that the implementation of your Kerenel Provisioning proposal is coming along

[telamonian]

@anirrudh I'll definitely also be taking a look at ksmm. Being able to select the machine in the UI alongside the kernel type is something I've wanted to see realized for a while

[echarles]

https://github.com/Quansight/ksmm is still WIP, but I would like anticipating some important aspects to move forward.

The current POC is developed with React.js and pulls react-bootstrap, which is yet-another widget toolkit on top of Lumino and Blueprint.js, and rely on @rjsf/bootstrap-4 to create the form. The cards are also created with React and diverge quite a lot from the current card of the launcher.

• Are we happy to pull more deps like in core?
• If not, what would be the minimal acceptable tradef-off? e.g. Have lumino cards, but leave the form built as react (bootstrap?) with json-form?

[newfrontier0007]

This will be a very useful feature.

[snehasaharawat]

Is there a latest update on this issue? We also have a similar usecase and this would be an extremely useful feature to us