Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix pytorch cpu jupyter server rock #79

Merged
merged 8 commits into from
Mar 21, 2024

Conversation

misohu
Copy link
Member

@misohu misohu commented Mar 19, 2024

This PR fixes (refer to the issue for the whole discussion):
canonical/data-science-stack#47

This PR also fixes some problems from here #78. Namely it updates python, miniforge and it uses mamba. We still need to figure out the s6 and builds for different architectures.

I updated the rock based of upstream changes in branch v1.8-branch here

Key changes:

  • Python 3.11
  • Miniforge 23.3.1-1
  • Torch 2.1
  • Using Mamba for installing some ML tools
  • adding mlflow 2.1. lib

Changes in tests:

  • now we test for mlflow but also for scikit learn and scikit learn image
  • fastai is no longer part of torch its a separate instalaltion which no longer part of upstream Dockerfile
  • kfserving lib is also no longer part of upstream image

CI is failing because trivy scan times out as the rock is too big. I fixed this in this PR which should be merged first and then we can rerun ci (I did that in previous commit with pinned branch in CI).

@misohu misohu requested a review from a team as a code owner March 19, 2024 12:02
@misohu misohu changed the title Fix pytorch gpu jupyter server rock Fix pytorch cpu jupyter server rock Mar 19, 2024
Copy link

@orfeas-k orfeas-k left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I 'm adding some comments here since I could not comment on unchanged code:

  • Is there a chance we need to update the following line? Although I see they still set the same NB_USER .
chown -R 584792:users home/jovyan
  • I also noticed that upstream has removed that line and modified the one below as seen here.

jupyter-pytorch-full/rockcraft.yaml Show resolved Hide resolved
jupyter-pytorch-full/rockcraft.yaml Show resolved Hide resolved
jupyter-pytorch-full/rockcraft.yaml Outdated Show resolved Hide resolved
jupyter-pytorch-full/rockcraft.yaml Outdated Show resolved Hide resolved
jupyter-pytorch-full/rockcraft.yaml Outdated Show resolved Hide resolved
jupyter-pytorch-full/tests/test_rock.py Outdated Show resolved Hide resolved
jupyter-pytorch-full/tests/test_imports.py Outdated Show resolved Hide resolved
jupyter-pytorch-full/tests/test_access.py Outdated Show resolved Hide resolved
jupyter-pytorch-full/tests/test_imports.py Show resolved Hide resolved
jupyter-pytorch-full/tox.ini Outdated Show resolved Hide resolved
Copy link

@orfeas-k orfeas-k left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved! Good job @misohu. This was a convoluted ROCK actually with a lot of digging needed but you pulled through as always 👏

@misohu misohu merged commit 3058193 into main Mar 21, 2024
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants