国内Ubuntu环境Docker部署Stable Diffusion入坑记录

本文旨在记录使用docker+python进行部署 stable-diffusion-webui 项目时遇到的一些问题，以及解决方案，原项目地址: https://github.com/AUTOMATIC1111/stable-diffusion-webui

问题一览：

国内如何下载大模型？
手写Docker运行时遇到的问题记录。

国内如何快速下载大模型？

使用 modelscope 模块进行下载，你可以在官网找到你需要的模型。
官网地址：https://modelscope.cn/models

# SDK模型下载
from modelscope import snapshot_download
# 示例
snapshot_download('iic/CosyVoice2-0.5B', local_dir='pretrained_models/CosyVoice2-0.5B')

Docker运行时遇到的问题

1、安装带gpu版本的torch？

在requirements_versions.txt 和 requirements.txt 的第一行加入以下文本：

国外：--extra-index-url https://download.pytorch.org/whl/cu121
国内：--extra-index-url https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/pytorch/wheel/cu121/

一般我们安装带GPU的torch时通常会去 download.pytorch.org 找对应的torch版本。
加上上述文本后会自动从该网址查找并下载安装。

2、以下问题需要重新编译python

是因为手动编译python时，系统依赖安装不足，导致有些依赖库找不到指定的系统包。这种情况只能安装相关系统软件后，重新编译python。

ModuleNotFoundError: No module named '_ctypes'

使用 apt-get install libffi-dev 安装 libffi-dev。

ModuleNotFoundError: No module named '_lzma'

使用 apt-get install liblzma-dev 安装 liblzma-dev。

重新编译方法：

wget https://www.python.org/ftp/python/3.10.13/Python-3.10.13.tgz
tar -xzf Python-3.10.13.tgz
cd Python-3.10.13
./configure --with-system-ffi --enable-shared --enable-optimizations && make && make install && echo "/usr/local/lib" | tee /etc/ld.so.conf.d/python3.conf && ldconfig

3、以下问题安装需要安装git

root@58befcdcc1d7:/workspace/automatic1111-stable-diffusion-webui# python3 webui.py 
/usr/local/lib/python3.10/site-packages/torchvision/transforms/functional_tensor.py:5: UserWarning: The torchvision.transforms.functional_tensor module is deprecated in 0.15 and will be **removed in 0.17**. Please don't rely on it. You probably just need to use APIs in torchvision.transforms.functional or in torchvision.transforms.v2.functional.
  warnings.warn(
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/git/__init__.py", line 87, in <module>
    refresh()
  File "/usr/local/lib/python3.10/site-packages/git/__init__.py", line 76, in refresh
    if not Git.refresh(path=path):
  File "/usr/local/lib/python3.10/site-packages/git/cmd.py", line 341, in refresh
    raise ImportError(err)
ImportError: Bad git executable.
The git executable must be specified in one of the following ways:
    - be included in your $PATH
    - be set via $GIT_PYTHON_GIT_EXECUTABLE
    - explicitly set via git.refresh()

All git commands will error until this is rectified.

This initial warning can be silenced or aggravated in the future by setting the
$GIT_PYTHON_REFRESH environment variable. Use one of the following values:
    - quiet|q|silence|s|none|n|0: for no warning or exception
    - warn|w|warning|1: for a printed warning
    - error|e|raise|r|2: for a raised exception

Example:
    export GIT_PYTHON_REFRESH=quiet


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/workspace/automatic1111-stable-diffusion-webui/webui.py", line 15, in <module>
    from modules import import_hook, errors, extra_networks, ui_extra_networks_checkpoints
  File "/workspace/automatic1111-stable-diffusion-webui/modules/ui_extra_networks_checkpoints.py", line 6, in <module>
    from modules import shared, ui_extra_networks, sd_models
  File "/workspace/automatic1111-stable-diffusion-webui/modules/shared.py", line 16, in <module>
    from modules import localization, extensions, script_loading, errors, ui_components, shared_items
  File "/workspace/automatic1111-stable-diffusion-webui/modules/extensions.py", line 6, in <module>
    import git
  File "/usr/local/lib/python3.10/site-packages/git/__init__.py", line 89, in <module>
    raise ImportError('Failed to initialize: {0}'.format(exc)) from exc
ImportError: Failed to initialize: Bad git executable.
The git executable must be specified in one of the following ways:
    - be included in your $PATH
    - be set via $GIT_PYTHON_GIT_EXECUTABLE
    - explicitly set via git.refresh()

All git commands will error until this is rectified.

This initial warning can be silenced or aggravated in the future by setting the
$GIT_PYTHON_REFRESH environment variable. Use one of the following values:
    - quiet|q|silence|s|none|n|0: for no warning or exception
    - warn|w|warning|1: for a printed warning
    - error|e|raise|r|2: for a raised exception

Example:
    export GIT_PYTHON_REFRESH=quiet

使用 apt install git 安装 git即可。

4、以下问题是因为缺少python依赖包或依赖包的版本问题

ModuleNotFoundError: No module named 'torchvision.transforms.functional_tensor'

这是因为高版本的torch有些包已经移动或者更改了导入方式，可选择降torch版本到1.13以下或者更改高版本的源码的导入方式。
请参考这篇博客：
https://blog.csdn.net/lanxing147/article/details/136625264

这篇博客讲到了有些版本是可以使用的，比如笔者使用的 torch 2.12 + torchvision0.16.2 + torchaudio2.1.2。
可使用以下命令安装:

pip install torch==2.1.2 torchvision==0.16.2 torchaudio==2.1.2 --index-url https://download.pytorch.org/whl/cu118

但是国内比较慢，可以选择使用国内镜像源进行提速。使用以下命令：

pip install torch==2.1.2 torchvision torchaudio -f https://mirrors.aliyun.com/pytorch-wheels/cu121/

参考以下文章：
https://docs.infini-ai.com/posts/download-pytorch-from-mirror.html

TypeError: AsyncConnectionPool.__init__() got an unexpected keyword argument 'socket_options'

这是由于 httpcore 版本太低导致的，使用 pip3 install -U httpcore 升级 httpcore 到最新版本即可。

ModuleNotFoundError: No module named 'clip'

使用 pip3 install clip 安装 clip 库即可。

No module named 'pytorch_lightning.utilities.distributed'

pytorch_lightning的版本较低，安装高版本的 pytorch_lightning 即可。
笔者使用的是 pip3 pytorch_lightning==1.9。

No module named 'gdown'

使用 pip3 install gdown 安装 gdown库即可。

No module named 'open_clip_torch'

使用 pip3 install open_clip_torch 安装 open_clip_torch库即可。

5、启动时遇到的问题

最终启动用命令：

python3 webui.py --listen --port=7860 --no-half --disable-nan-check

写在最后

附上完整的Docker 镜像构筑的相关文件，国内基本上也能够很顺畅的一键构筑好docker镜像，总大小约28G。
请在项目根目录下创建一个 docker 目录，然后将 Dockerfile、compose.yaml、requirements.txt 、 requirements_versions.txt、start.sh等文件放进去。
然后执行 cd docker && docker compose -f compose.yaml up 命令。
如果访问浏览器的 7860 端口出现以下界面，恭喜你成功了。

在这里插入图片描述

Dockerfile

FROM nvidia/cuda:11.8.0-cudnn8-devel-ubuntu22.04

ARG VENV_NAME="sd-webui"
ENV VENV=$VENV_NAME
ENV LANG=C.UTF-8 LC_ALL=C.UTF-8

ENV DEBIAN_FRONTEN=noninteractive
SHELL ["/bin/bash", "-c"]

RUN apt-get update -y
RUN apt-get install -y libgl1-mesa-glx libglib2.0-0
RUN apt-get install -y net-tools wget curl git

RUN apt-get install -y make build-essential libssl-dev zlib1g-dev libbz2-dev libreadline-dev libsqlite3-dev libffi-dev liblzma-dev

# 从国内镜像源下载安装python
# wget https://www.python.org/ftp/python/3.10.13/Python-3.10.13.tar.xz && tar Jxf Python-3.10.13.tar.xz 
RUN wget https://mirrors.huaweicloud.com/python/3.10.13/Python-3.10.13.tar.xz && tar Jxf Python-3.10.13.tar.xz
RUN cd Python-3.10.13 && ./configure --with-system-ffi --enable-shared --enable-optimizations && make && make install && echo "/usr/local/lib" | tee /etc/ld.so.conf.d/python3.conf && ldconfig
RUN python3 -V && pip3 -V

# 设置国内镜像源
RUN pip3 config set global.index-url https://mirrors.aliyun.com/pypi/simple/ && pip3 config set install.trusted-host mirrors.aliyun.com

WORKDIR /workspace
COPY ./requirements_versions.txt ./
COPY ./requirements.txt ./

RUN pip3 install -r requirements_versions.txt
RUN pip3 install -r requirements.txt

compose.yaml

services:
  sd-webui:
    container_name: sd-webui
    image: sd-webui:1.0
    restart: always
    ports:
      - 7860:7860
    environment:
      - TZ=Asia/Tokyo
      - NVIDIA_VISIBLE_DEVICES=all
    volumes:
      - ../../automatic1111-stable-diffusion-webui:/workspace/automatic1111-stable-diffusion-webui
    # command: tail -f /dev/null
    command: sh -c "sh /workspace/automatic1111-stable-diffusion-webui/docker/start.sh"
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              capabilities: [gpu]

requirements.txt

--extra-index-url https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/pytorch/wheel/cu121/
blendmodes
accelerate
basicsr
fonts
font-roboto
gfpgan
gradio
invisible-watermark
numpy
omegaconf
opencv-contrib-python
requests
piexif
Pillow
# pytorch_lightning==1.7.7
pytorch_lightning==1.9
realesrgan
scikit-image>=0.19
timm==0.4.12
transformers==4.25.1
# torch
torch==2.1.2
torchaudio==2.1.2
einops
jsonmerge
clean-fid
resize-right
torchdiffeq
kornia
lark
inflection
GitPython
torchsde
safetensors
psutil
open_clip_torch
gdown
clip

requirements_versions.txt

--extra-index-url https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/pytorch/wheel/cu121/
blendmodes==2022
transformers==4.25.1
accelerate==0.12.0
basicsr==1.4.2
gfpgan==1.3.8
gradio==3.16.2
numpy==1.23.3
Pillow==9.4.0
realesrgan==0.3.0
torch
omegaconf==2.2.3
# pytorch_lightning==1.7.6
pytorch_lightning
scikit-image==0.19.2
fonts
font-roboto
timm==0.6.7
piexif==1.1.3
einops==0.4.1
jsonmerge==1.8.0
clean-fid==0.1.29
resize-right==0.0.2
torchdiffeq==0.2.3
kornia==0.6.7
lark==1.1.2
inflection==0.5.1
GitPython==3.1.27
torchsde==0.2.5
safetensors==0.2.7
# httpcore<=0.15
httpcore
fastapi==0.90.1
open_clip_torch

start.sh

#! /bin/bash
cd automatic1111-stable-diffusion-webui && python3 webui.py --listen --port=7860 --no-half --disable-nan-check

以上就是本次踩坑的记录。愿看到的小伙伴不迷路。