Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix Docker file permission issue #3323

Merged
merged 4 commits into from
May 15, 2024

Conversation

raidensakura
Copy link
Member

@raidensakura raidensakura commented Dec 20, 2023

This PR fixes the permission issue (#3319) by installing the Python dependencies as a non-root user.

It also changes the base python image to alpine version since it's smaller in size.

This reduced image size from 1.07GB ~> 250MB. Tested with locally-built image on Docker Desktop 4.25.2 (129061)

Ignore the messy commits I'm too lazy to squash them

@raidensakura raidensakura changed the title Fix Docker file permission issue #3319 Fix Docker file permission issue Dec 20, 2023
@raidensakura raidensakura changed the base branch from master to development December 21, 2023 09:50
Copy link
Contributor

@martinbndr martinbndr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried building the image locally on my ubuntu 20.04 vps.
Facing this issue:

root@vserver:/home/masteradmin/modmail-dev/Modmail# docker build -t modmailbotimage:latest . --no-cache
[+] Building 0.9s (10/12)                                                                                                                                       docker:default
 => [internal] load .dockerignore                                                                                                                                         0.0s
 => => transferring context: 1.75kB                                                                                                                                       0.0s
 => [internal] load build definition from Dockerfile                                                                                                                      0.0s
 => => transferring dockerfile: 418B                                                                                                                                      0.0s
 => [internal] load metadata for docker.io/library/python:3.10                                                                                                            0.0s
 => CACHED [1/8] FROM docker.io/library/python:3.10                                                                                                                       0.0s
 => [internal] load build context                                                                                                                                         0.1s
 => => transferring context: 953B                                                                                                                                         0.0s
 => CANCELED [2/8] RUN apt update && apt install -y g++ git && pip install --upgrade pip                                                                                  0.7s
 => CACHED [3/8] RUN useradd modmail                                                                                                                                      0.0s
 => CACHED [4/8] WORKDIR /home/modmail                                                                                                                                    0.0s
 => CACHED [5/8] RUN pip install --user pipenv                                                                                                                            0.0s
 => ERROR [6/8] COPY --chown=modmail:modmail Pipfile Pipfile.lock ./                                                                                                      0.0s
------
 > [6/8] COPY --chown=modmail:modmail Pipfile Pipfile.lock ./:
------
Dockerfile:13
--------------------
  11 |     ENV PATH="/home/modmail/.local/bin:${PATH}"
  12 |     
  13 | >>> COPY --chown=modmail:modmail Pipfile Pipfile.lock ./
  14 |     RUN pipenv install
  15 |     
--------------------
ERROR: failed to solve: failed to compute cache key: failed to calculate checksum of ref 06d46631-a422-4771-b4ae-d1c886f53ef5::vkn69h4jf1xigfqsebd97mhy2: "/Pipfile.lock": not found

I tried to edit the Dockerfile to improve it and got a solution without using pipenv inside the container and I also like it more not using the pipenv as we already have a isolated environment inside the image/container so we could just install the modules as modmail user.
I tested the following image built locally the bot works including installation of plugins. No Errors regarding permissions anymore.

Improved image:

FROM python:3.10

RUN apt update && apt install -y g++ git && pip install --upgrade pip

RUN useradd modmail
USER modmail
WORKDIR /home/modmail

ENV PATH="/home/modmail/.local/bin:${PATH}"

COPY --chown=modmail:modmail . .
RUN pip install -r requirements.txt --user

ENV USING_DOCKER yes

CMD ["python","bot.py"]

Can you also test this one and if you like change it in the PR?

@martinbndr
Copy link
Contributor

Don´t understand me wrong, your idea installing that via pipenv is also nice.
But I feel like the improvement could be better because like already said, we have a isolated python environment in the image/container build so we could just install the modules inside the modmail user using the --user flag of pip install.

And it also fixes the issue I am getting on the issue I´m getting with pipenv aka the Pipfile.lock file

@raidensakura
Copy link
Member Author

I tried building the image locally on my ubuntu 20.04 vps. Facing this issue:

root@vserver:/home/masteradmin/modmail-dev/Modmail# docker build -t modmailbotimage:latest . --no-cache
[+] Building 0.9s (10/12)                                                                                                                                       docker:default
 => [internal] load .dockerignore                                                                                                                                         0.0s
 => => transferring context: 1.75kB                                                                                                                                       0.0s
 => [internal] load build definition from Dockerfile                                                                                                                      0.0s
 => => transferring dockerfile: 418B                                                                                                                                      0.0s
 => [internal] load metadata for docker.io/library/python:3.10                                                                                                            0.0s
 => CACHED [1/8] FROM docker.io/library/python:3.10                                                                                                                       0.0s
 => [internal] load build context                                                                                                                                         0.1s
 => => transferring context: 953B                                                                                                                                         0.0s
 => CANCELED [2/8] RUN apt update && apt install -y g++ git && pip install --upgrade pip                                                                                  0.7s
 => CACHED [3/8] RUN useradd modmail                                                                                                                                      0.0s
 => CACHED [4/8] WORKDIR /home/modmail                                                                                                                                    0.0s
 => CACHED [5/8] RUN pip install --user pipenv                                                                                                                            0.0s
 => ERROR [6/8] COPY --chown=modmail:modmail Pipfile Pipfile.lock ./                                                                                                      0.0s
------
 > [6/8] COPY --chown=modmail:modmail Pipfile Pipfile.lock ./:
------
Dockerfile:13
--------------------
  11 |     ENV PATH="/home/modmail/.local/bin:${PATH}"
  12 |     
  13 | >>> COPY --chown=modmail:modmail Pipfile Pipfile.lock ./
  14 |     RUN pipenv install
  15 |     
--------------------
ERROR: failed to solve: failed to compute cache key: failed to calculate checksum of ref 06d46631-a422-4771-b4ae-d1c886f53ef5::vkn69h4jf1xigfqsebd97mhy2: "/Pipfile.lock": not found

I tried to edit the Dockerfile to improve it and got a solution without using pipenv inside the container and I also like it more not using the pipenv as we already have a isolated environment inside the image/container so we could just install the modules as modmail user. I tested the following image built locally the bot works including installation of plugins. No Errors regarding permissions anymore.

Improved image:

FROM python:3.10

RUN apt update && apt install -y g++ git && pip install --upgrade pip

RUN useradd modmail
USER modmail
WORKDIR /home/modmail

ENV PATH="/home/modmail/.local/bin:${PATH}"

COPY --chown=modmail:modmail . .
RUN pip install -r requirements.txt --user

ENV USING_DOCKER yes

CMD ["python","bot.py"]

Can you also test this one and if you like change it in the PR?

The error is related to Pipfile.lock not existing either because you don't have the file or it's specified in .dockerignore.

@martinbndr
Copy link
Contributor

Ah yea true. Ignore what I said earlier about the error in first part of the change suggestion.

@raidensakura
Copy link
Member Author

raidensakura commented Dec 23, 2023

@martinbndr I refactored the Dockerfile as per your suggestion to use pip, but with some additional improvements like using the alpine-based image and multi-stage build for minimum image size. It went down from ~1.19GB to ~250MB in size.

@martinbndr
Copy link
Contributor

Would be nice to update this as it´s a quite important fix for all users hosting this on docker. @Taaku18

@laundmo
Copy link

laundmo commented Jan 22, 2024

I ran into this issue trying to run a plugin which installed a dependency using pip: Jerrie-Aries/modmail-plugins#42

@raidensakura
Copy link
Member Author

I ran into this issue trying to run a plugin which installed a dependency using pip: Jerrie-Aries/modmail-plugins#42

That is the issue this PR is trying to fix...

@laundmo
Copy link

laundmo commented Jan 23, 2024

I ran into this issue trying to run a plugin which installed a dependency using pip: Jerrie-Aries/modmail-plugins#42

That is the issue this PR is trying to fix...

as far as i can tell the original issue linked was slightly different, as it concerned temp not home

@raidensakura
Copy link
Member Author

I ran into this issue trying to run a plugin which installed a dependency using pip: Jerrie-Aries/modmail-plugins#42

That is the issue this PR is trying to fix...

as far as i can tell the original issue linked was slightly different, as it concerned temp not home

The main issue is the Python deps have root file ownership while the bot is running as a user, regardless of where they're being installed. I can install the plugin mentioned in your linked issue just fine using this PR, when proper chown is used in the Dockerfile

Screenshot_20240123-133026_Discord

@laundmo
Copy link

laundmo commented Jan 23, 2024

yes exactly this PR fixes that, thats what i was trying to say

@raidensakura raidensakura changed the base branch from development to master February 17, 2024 03:06
@raidensakura raidensakura deleted the docker-perm-fix branch April 10, 2024 11:42
@raidensakura raidensakura restored the docker-perm-fix branch April 10, 2024 11:51
@raidensakura raidensakura reopened this Apr 10, 2024
@Taaku18
Copy link
Collaborator

Taaku18 commented May 14, 2024

I updated the Dockerfile to use the Debian based slim image instead of Alpine. Reasons being: 1. Alpine images suffers from a performance penalty, 2. potential compatibility issues with plugins, 3. longer build time. Furthermore, building Modmail using the slim image as opposed to alpine also yields a marginally smaller image (251MB -> 239MB). I also changed /home/modmail to /opt/modmail since modmail isn't a login user.

@Taaku18 Taaku18 merged commit 285e336 into modmail-dev:master May 15, 2024
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants