Three Workarounds for Failing to Install cryptography When Setting Up a Python Environment in an Alpine Linux Docker Image
2022-08-12 00:06 (2 years ago)
ytyng
I have written a similar article on my blog in the past,
If Python cryptography fails to install, install rust via rustup | ytyng.com
When trying to set up a Python environment using a version lock system like Pipenv on an Alpine Linux Docker image, the installation often fails and encounters obstacles.
Even if the build works fine on an Intel CPU, it may fail on an Apple Silicon chip using the same Dockerfile, making it quite unstable.
※ Even when building with the docker build --platform linux/amd64 option on both Intel and Apple CPUs, the results may still be unstable.
I will note down several possible solutions to this issue.
A. Prepare the environment and compile with Rust
If Python cryptography fails to install, install rust via rustup | ytyng.com
We will proceed using the method I described previously. However, in this case, while the build is stable on Intel CPUs, you may encounter the following error on Apple CPUs, preventing the build.
Failed to install some dependency or packages. The following have failed installation and attempted retry: [Requirement(_name='cryptography', vcs=None, req=NamedRequirement(name='cryptography', version='==37.0.4', req=Requirement.parse('cryptography==37.0.4; python_version >= "3.6"'), extras=[], editable=False,
Additionally, installing Rust and building cryptography takes a significant amount of time.
The Dockerfile would look something like this:
FROM alpine:3.15 as my-app-builder
MAINTAINER ytyng
# If you use apk --virtual=.build-deps add and then apk del .build-deps,
# requests and six will be removed, so do not do this.
RUN apk --no-cache add \
python3 \
py3-pip \
python3-dev \
gcc \
make \
python3-dev \
musl-dev \
libffi-dev \
mariadb-dev \
postgresql-dev \
g++ \
libgcc \
libstdc++ \
libxml2-dev \
libxslt-dev \
jpeg-dev \
git \
openssh \
curl \
&& curl https://sh.rustup.rs -sSf | sh -s -- -y \
&& pip3 install pipenv --ignore-installed distlib \
&& mkdir ~/.ssh/ && ssh-keyscan -t rsa github.com > ~/.ssh/known_hosts
COPY Pipfile /tmp/Pipfile
COPY Pipfile.lock /tmp/Pipfile.lock
RUN --mount=type=ssh source /root/.cargo/env \
&& PIPENV_PIPFILE=/tmp/Pipfile pipenv sync --system \
&& rustup self uninstall -y \
&& rm -rf /var/cache/apk/* \
&& rm -rf /tmp/*
FROM alpine:3.15
## Copy the binaries created in the build image
COPY --from=my-app-builder /usr/lib/python3.9/site-packages/ \
/usr/lib/python3.9/site-packages/
RUN apk --no-cache add \
python3 \
bash \
libxml2 \
libxslt \
libressl \
&& ln -s /usr/bin/python3 /usr/bin/python
USER nobody
WORKDIR /var/src/
COPY . /var/src/
B. Remove cryptography from Pipenv.lock and install it via apk
If the above method of installing cryptography with Rust does not work (or you wish to avoid it because it's slow), you can skip installing cryptography with pipenv and instead install it via apk. (In other words, you give up on version locking for cryptography.)
If you want to create an environment using pipenv sync with Pipenv.lock, make sure to delete the cryptography item from Pipenv.lock and pre-install py3-cryptography with apk. This way, you don't need Rust, and the Dockerfile becomes simpler.
However, I couldn't find a method to automatically remove cryptography from Pipenv.lock or exclude it from installation with pipenv sync, so a separate script may be necessary for continued use.
FROM alpine:3.15 as my-app-builder
RUN apk --no-cache add \
python3 \
py3-pip \
python3-dev \
py3-cryptography \
gcc \
make \
python3-dev \
musl-dev \
libffi-dev \
mariadb-dev \
postgresql-dev \
g++ \
libgcc \
libstdc++ \
libxml2-dev \
libxslt-dev \
jpeg-dev \
git \
openssh \
curl \
&& pip3 install pipenv --ignore-installed distlib
COPY Pipfile /tmp/Pipfile
COPY Pipfile.lock /tmp/Pipfile.lock
RUN PIPENV_PIPFILE=/tmp/Pipfile pipenv sync --system
FROM alpine:3.15
## Copy the binaries created in the build image
COPY --from=my-app-builder /usr/lib/python3.9/site-packages/ \
/usr/lib/python3.9/site-packages/
...
C. Install cryptography via apk and do not use lock files
This is a somewhat forceful method, but you can install a pre-built version of cryptography via apk and install other dependencies without Pipfile.lock, using only the Pipfile with pipenv install. This approach might be acceptable for Docker images where dependency strictness is not critical.
(Not only for cryptography, but this method also gives up on version locking for all dependencies not listed in the Pipfile.)
When building Docker images for x86_64 on Apple CPUs, methods B or C have been stable thus far.
FROM alpine:3.15 as my-app-builder
RUN apk --no-cache add \
python3 \
py3-pip \
python3-dev \
py3-cryptography \
gcc \
make \
python3-dev \
musl-dev \
libffi-dev \
mariadb-dev \
postgresql-dev \
g++ \
libgcc \
libstdc++ \
libxml2-dev \
libxslt-dev \
jpeg-dev \
git \
openssh \
curl \
&& pip3 install pipenv --ignore-installed distlib
COPY Pipfile /tmp/Pipfile
RUN PIPENV_PIPFILE=/tmp/Pipfile pipenv install --system --skip-lock --deploy
Comments