Tagging for visibility: @jungaretti @samruddhikhandale

Hi 👋

Can you try with the following .devcontainer.json?

{
  "name": "Only Jupyter",
  "image": "mcr.microsoft.com/devcontainers/universal:2",
  "hostRequirements": {
    "cpus": 4
  }
}

I vaguely remember there needs another configuration change (when using Python Feature) which doesn't seem to be documented.

Thank you @samruddhikhandale,

Yes, the "mcr.microsoft.com/devcontainers/universal:2" image opens with the Jupyterlab editor. Probably because it already has Jupyter installed within it.

This image works with JupyterLab, with or without the python feature.

However, my goal is to use features to introduce the Python & JupyterLab into an image that didn't already have it; and then use that image to start in Codespace's Jupyter Editor.

The "mcr.microsoft.com/devcontainers/universal:2" image is over 13.7GB, way more than what I need in my containers.

Any other thoughts?

Could it be because of the containerUser specification?

Is this a problem with the $PATH variable in the base image?

JupyterLab is installed in the $HOME/.local/bin/ directory. If I remember correctly, this directory is only included in $PATH for login shells.

We hardcode this value into the non-login $PATH in the universal image, but not in other images: https://github.com/devcontainers/images/blob/9bb5c4f0540a9a54b0d4ea4151b737c7af2f8aec/src/universal/.devcontainer/local-features/setup-user/install.sh#L92

Surprisingly, These problematic containers do have jupyter available on path As shown below:

@prabhakk-mw ➜ /workspaces/matlab-codespaces (mybranch) $ which jupyter
/home/vscode/.local/bin/jupyter

@prabhakk-mw ➜ /workspaces/matlab-codespaces (mybranch) $ python -m pip show jupyterlab | grep -i location
Location: /home/vscode/.local/lib/python3.10/site-packages

I can even start JuptyerLab in the terminal and click on the link to view the Lab console, albeit with a refresh of the vscode interface:

prabhakk-mw ➜ /workspaces/matlab-codespaces (deleteme) $ jupyter lab
[I 2024-02-23 09:45:25.096 ServerApp] jupyter_lsp | extension was successfully linked.
[I 2024-02-23 09:45:25.100 ServerApp] jupyter_server_mathjax | extension was successfully linked.
[I 2024-02-23 09:45:25.102 ServerApp] jupyter_server_terminals | extension was successfully linked.
[W 2024-02-23 09:45:25.104 LabApp] 'allow_origin' has moved from NotebookApp to ServerApp. This config will be passed to ServerApp. Be sure to update your config before our next release.
[I 2024-02-23 09:45:25.106 ServerApp] jupyterlab | extension was successfully linked.
[I 2024-02-23 09:45:25.107 ServerApp] jupyterlab_git | extension was successfully linked.
[I 2024-02-23 09:45:25.107 ServerApp] nbdime | extension was successfully linked.
[I 2024-02-23 09:45:25.108 ServerApp] Writing Jupyter server cookie secret to /home/vscode/.local/share/jupyter/runtime/jupyter_cookie_secret
[I 2024-02-23 09:45:25.322 ServerApp] notebook_shim | extension was successfully linked.
[I 2024-02-23 09:45:25.336 ServerApp] notebook_shim | extension was successfully loaded.
[I 2024-02-23 09:45:25.338 ServerApp] jupyter_lsp | extension was successfully loaded.
[I 2024-02-23 09:45:25.338 ServerApp] jupyter_server_mathjax | extension was successfully loaded.
[I 2024-02-23 09:45:25.339 ServerApp] jupyter_server_terminals | extension was successfully loaded.
[I 2024-02-23 09:45:25.340 LabApp] JupyterLab extension loaded from /home/vscode/.local/lib/python3.10/site-packages/jupyterlab
[I 2024-02-23 09:45:25.341 LabApp] JupyterLab application directory is /home/vscode/.local/share/jupyter/lab
[I 2024-02-23 09:45:25.341 LabApp] Extension Manager is 'pypi'.
[I 2024-02-23 09:45:25.370 ServerApp] jupyterlab | extension was successfully loaded.
[I 2024-02-23 09:45:25.374 ServerApp] jupyterlab_git | extension was successfully loaded.
[I 2024-02-23 09:45:25.412 ServerApp] nbdime | extension was successfully loaded.
[I 2024-02-23 09:45:25.413 ServerApp] Serving notebooks from local directory: /workspaces/matlab-codespaces
[I 2024-02-23 09:45:25.413 ServerApp] Jupyter Server 2.12.5 is running at:
[I 2024-02-23 09:45:25.413 ServerApp] http://localhost:8888/lab?token=5ae6f9cf1ff0bac6f94a55f1332d7cfc85361f2625cf1959
[I 2024-02-23 09:45:25.413 ServerApp]     http://127.0.0.1:8888/lab?token=5ae6f9cf1ff0bac6f94a55f1332d7cfc85361f2625cf1959
[I 2024-02-23 09:45:25.413 ServerApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[C 2024-02-23 09:45:25.417 ServerApp] 
    
    To access the server, open this file in a browser:
        file:///home/vscode/.local/share/jupyter/runtime/jpserver-2104-open.html
    Or copy and paste one of these URLs:
        http://localhost:8888/lab?token=5ae6f9cf1ff0bac6f94a55f1332d7cfc85361f2625cf1959
        http://127.0.0.1:8888/lab?token=5ae6f9cf1ff0bac6f94a55f1332d7cfc85361f2625cf1959

Clicking on the last link does open up JuptyerLab, but still requires me to copy-paste the token into the authentication page that is presented.

Pasting the token from VSCode terminal lets me through.

The exact same image fails when you attempt to open it through

Does this error message at the bottom of the presented screen mean anything to you guys?
Do you know of anyone on the codespace team that could help?

Is this a problem with the $PATH variable in the base image?

JupyterLab is installed in the $HOME/.local/bin/ directory. If I remember correctly, this directory is only included in $PATH for login shells.

We hardcode this value into the non-login $PATH in the universal image, but not in other images: https://github.com/devcontainers/images/blob/9bb5c4f0540a9a54b0d4ea4151b737c7af2f8aec/src/universal/.devcontainer/local-features/setup-user/install.sh#L92

Thanks @jungaretti for pointing it out, and yes ^ this is the dev container configuration change needed for a codespace to be opened with JupyeterLab.

@prabhakk-mw Can you add a similar change with :/home/${USERNAME}/.local/bin added to PATH in your Dockerfile? Let us know if that helps.

Looping in Codespaces PM @craiglpeters, this is a known gap ^ in our documents where the Python Feature don't work without the PATH change. We should update our docs to reflect it until this gets fixed in the code. (Last we check, we ran into multiple issues to fix it on Codespaces end, hence, the doc updates should help avoid such confusions in the meanwhile)

Surprisingly, These problematic containers do have jupyter available on path

@prabhakk-mw It looks like you are using a login shell that adds the $HOME/.local/bin/ to your PATH variable. This directory is scoped to a certain user (with $HOME) and relies on a login shell. To accurately reproduce this bug, you need to use a non-login shell.

You can use docker run in a Codespace to test this.

base Image

Command: docker run --rm mcr.microsoft.com/devcontainers/base:ubuntu /bin/sh -c 'echo $PATH'

/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

/home/vscode/.local/bin is missing from this PATH variable.

universal:2 image

Command: docker run --privileged --rm mcr.microsoft.com/devcontainers/universal:2 /bin/sh -c 'echo $PATH'

/home/codespace/.dotnet:/home/codespace/nvm/current/bin:/home/codespace/.php/current/bin:/home/codespace/.python/current/bin:/home/codespace/java/current/bin:/home/codespace/.ruby/current/bin:/home/codespace/.local/bin:/usr/local/python/current/bin:/usr/local/py-utils/bin:/usr/local/oryx:/usr/local/go/bin:/go/bin:/usr/local/sdkman/bin:/usr/local/sdkman/candidates/java/current/bin:/usr/local/sdkman/candidates/gradle/current/bin:/usr/local/sdkman/candidates/maven/current/bin:/usr/local/sdkman/candidates/ant/current/bin:/usr/local/rvm/gems/default/bin:/usr/local/rvm/gems/default@global/bin:/usr/local/rvm/rubies/default/bin:/usr/local/share/rbenv/bin:/usr/local/php/current/bin:/opt/conda/bin:/usr/local/nvs:/usr/local/share/nvm/current/bin:/usr/local/hugo/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/share/dotnet:~/.dotnet/tools

This PATH variable contains /home/codespace/.local/bin and works as expected.

@samruddhikhandale , Thank you for your suggestions.
Unfortunately, my usecase doesn't have a Dockerfile that I can update.

@jungaretti, Thank you for the clarification.
It does feel very quirky that the Codespaces infrastructure does not check against the specified containerUsers PATH before declaring that it is inaccessible.
Having users place binaries from their HOME folders for all possible users of a container also feels uncomfortable.

However, I've taken a pass at fixing the issue at the feature level in #887
Thoughts and suggestions are welcomed!

I created a dedicated branch on my repo that point to the exact last version of the python feature (1.6.0): https://github.com/gee-community/ipygee/tree/test-jupyterlab

I created a codespace that still works fine on VS code (https://github.com/gee-community/ipygee/codespaces) but still fail to run in jupyterlab. So unfortunately it seems we are back to square 1.

{
  "name": "Python 3",
  "image": "mcr.microsoft.com/devcontainers/python:1-3.11-bullseye",
  "features": {
    "ghcr.io/devcontainers/features/python:1.6.0": {
      "installJupyterlab": true
    },
    "ghcr.io/devcontainers-contrib/features/nox:2": {},
    "ghcr.io/devcontainers-contrib/features/pre-commit:2": {}
  },
  "postCreateCommand": "python -m pip install commitizen && pre-commit install"
}

@prabhakk-mw did you try your solution from your side ? If yes can you explain what I did wrong ?

@12rambau I also confirm that the fix proposed in #870 (comment) which was implemented in #887 does NOT fix the issue.

Now when I start the codespace, jupyter is on PATH and is also present in the sudoers.d file as shown below:

@prabhakk-mw ➜ /workspaces/test-matlab-codespaces (main) $ which jupyter
/home/vscode/.local/bin/jupyter
@prabhakk-mw ➜ /workspaces/test-matlab-codespaces (main) $ sudo cat /etc/sudoers.d/vscode
vscode ALL=(root) NOPASSWD:ALL
Defaults secure_path=/home/vscode/.local/bin:/usr/local/python/3.10.14/bin:/usr/local/python/current/bin:/usr/local/py-utils/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/py-utils/bin
@prabhakk-mw ➜ /workspaces/test-matlab-codespaces (main) $ 

I still see the following screen:

@jungaretti do you have any ideas as to why this might be happening?

The devcontainer.json file I used was:

{
  "name": "Only Jupyter",
  "image": "mcr.microsoft.com/devcontainers/base:ubuntu",
  "hostRequirements": {
    "cpus": 4
  },
  "features": {
    "ghcr.io/devcontainers/features/python": {
      "version": "3.10",
      "installJupyterlab": true,
      "configureJupyterlabAllowOrigin": "*"
    }
  },
  "containerUser": "vscode"
}

What else I have tried:

1. Switched to using "containerUser":"codespace"
   This caused the container creation to fail
2. Removed the containerUser specification.
   This resulted in the same error as shown above when attempting to open in JupyterLab IDE

@samruddhikhandale , can you please reopen this issue? Also, do you know anyone from the Codespace + JupyterLab project that could help us?

@prabhakk-mw,

Have you had any luck getting this working with the changes in #887 ?

I tried using the following devcontainer.json.

{
  "name": "Only Jupyter",
  "image": "mcr.microsoft.com/devcontainers/base:ubuntu",
  "hostRequirements": {
    "cpus": 4
  },
  "features": {
    "ghcr.io/devcontainers/features/python:1.6.0": {
      "version": "3.10",
      "installJupyterlab": true,
      "configureJupyterlabAllowOrigin": "*"
    }
  },
  "containerUser": "vscode"
}

...but see the following error.

Failed to load the JupyterLab workbench: Unable to start server.

Thanks for any suggestions!

Have you had any luck getting this working with the changes in #887 ?

yes if you look at my current branch, I'm force pointing to 1.6.0 hich includes changes made in #887, still no luck....

@prabhakk-mw,

Have you had any luck getting this working with the changes in #887 ?

I tried using the following devcontainer.json.

{
  "name": "Only Jupyter",
  "image": "mcr.microsoft.com/devcontainers/base:ubuntu",
  "hostRequirements": {
    "cpus": 4
  },
  "features": {
    "ghcr.io/devcontainers/features/python:1.6.0": {
      "version": "3.10",
      "installJupyterlab": true,
      "configureJupyterlabAllowOrigin": "*"
    }
  },
  "containerUser": "vscode"
}

...but see the following error.

Failed to load the JupyterLab workbench: Unable to start server.

Thanks for any suggestions!

No, @jcansdale ... It didn't work. See #870 (comment)

Looking at the Codespace export logs, the codespace is still not able to find jupyer. See 👇

Looks like we also need to add /home/$USERNAME/.local/bin to containerEnv:PATH, see ref. However, it's tricky and we currently don't have an easy way to auto-compute the user in devcontainer-feature.json. I wonder if we should symlink the binary to a static path and then refer it in containerEnv

I tested with 👇 and Open in JupyterLab worked.

{
	"image": "mcr.microsoft.com/devcontainers/python:3.10",
	"features": {
		"ghcr.io/devcontainers/features/python:1": {
			"version": "none",
			"installTools": "false",
			"installJupyterlab": "true",
			"configureJupyterlabAllowOrigin": "*"
		}
	},
	"containerEnv": {
            // testing
            "PATH" : "/vscode/bin/linux-x64/dc96b837cf6bb4af9cd736aa3af08cf8279f7685/bin/remote-cli:/home/vscode/.local/bin:/usr/local/python/current/bin:/usr/local/py-utils/bin:/usr/local/python/current/bin:/usr/local/py-utils/bin:/usr/local/share/nvm/current/bin:/usr/local/bin:/usr/local/python/current/bin:/usr/local/py-utils/bin:/usr/local/python/current/bin:/usr/local/py-utils/bin:/usr/local/share/nvm/current/bin:/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/home/vscode/.local/bin"
    }
}