GPT-SoVITS语音克隆部署与使用

news2024/10/7 16:26:22

GPT-SoVITS是一款强大的少量样本语音转换与语音合成开源工具。当前，GPT-SoVITS实现了如下几个方面的功能：

由参考音频的情感、音色、语速控制合成音频的情感、音色、语速
可以少量语音微调训练，也可不训练直接推理
可以跨语种生成，即参考音频（训练集）和推理文本的语种为不同语种

硬件建议要求：
INT4 ： RTX3090*1，显存24GB，内存32GB，系统盘200GB
更低配的GPU硬件也可以进行推理，但是推理速度会更慢。

环境准备

模型准备

手动下载以下几个模型（体验时几个模型不一定需全下载）
本文统一放在模型存档目录：/u01/workspace/models/GPT-SoVITS
pretrained_models

git clone https://huggingface.co/lj1995/GPT-SoVITS

uvr5_weights

https://huggingface.co/lj1995/VoiceConversionWebUI/tree/main/uvr5_weights

asr model：

git clone https://hf-mirror.com/Systran/faster-whisper-large-v3

可选模型： speech_fsmn_vad_zh-cn-16k-common-pytorch,speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-pytorch, punc_ct-transformer_zh-cn-common-vocab272727-pytorch
下载地址分别为：

git clone https://www.modelscope.cn/iic/speech_fsmn_vad_zh-cn-16k-common-pytorch.git
git clone https://www.modelscope.cn/iic/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-pytorch.git
git clone https://www.modelscope.cn/iic/punc_ct-transformer_zh-cn-common-vocab272727-pytorch.git

注意存放目录结构方便后面Docker 挂载目录后能直接找到相关的模型

root@itserver03:/u01/workspace/models/GPT-SoVITS# tree  -d
.
├── GPT_weights
├── pretrained_models
│   ├── chinese-hubert-base
│   └── chinese-roberta-wwm-ext-large
├── SoVITS_weights
└── tools
    ├── asr
    │   └── models
    │       ├── faster-whisper-large-v3
    │       ├── punc_ct-transformer_zh-cn-common-vocab272727-pytorch
    │       │   ├── example
    │       │   └── fig
    │       ├── speech_fsmn_vad_zh-cn-16k-common-pytorch
    │       │   ├── example
    │       │   └── fig
    │       └── speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-pytorch
    │           ├── example
    │           └── fig
    └── uvr5
        └── uvr5_weights
            └── Onnx_dereverb_By_FoxJoy

下载源码

git clone https://github.com/RVC-Boss/GPT-SoVITS.git; 
cd GPT-SoVITS

Docker 容器化部署

Dockerfile样例

注意根据官方的Dockerfile自己build出满足自身环境需要的镜像。

# Base CUDA image
# FROM cnstark/pytorch:2.0.1-py3.9.17-cuda11.8.0-ubuntu20.04
FROM pytorch/pytorch:2.2.1-cuda12.1-cudnn8-runtime

LABEL maintainer="breakstring@hotmail.com"
LABEL version="dev-20240209"
LABEL description="Docker image for GPT-SoVITS"


# Install 3rd party apps
ENV DEBIAN_FRONTEND=noninteractive
ENV TZ=Etc/UTC
ENV TZ=Etc/UTC
ENV LANG=zh_CN.UTF-8
ENV LC_ALL=zh_CN.UTF-8

RUN apt-get update && \
    apt-get install -y --no-install-recommends tzdata ffmpeg libsox-dev parallel aria2 git git-lfs && \
    git lfs install && \
    rm -rf /var/lib/apt/lists/*

# Copy only requirements.txt initially to leverage Docker cache
WORKDIR /workspace
COPY requirements.txt /workspace/
RUN pip install --no-cache-dir -r requirements.txt

# Define a build-time argument for image type
ARG IMAGE_TYPE=full

# Conditional logic based on the IMAGE_TYPE argument
# Always copy the Docker directory, but only use it if IMAGE_TYPE is not "elite"
COPY ./Docker /workspace/Docker 
# elite 类型的镜像里面不包含额外的模型
RUN if [ "$IMAGE_TYPE" != "elite" ]; then \
        chmod +x /workspace/Docker/download.sh && \
        /workspace/Docker/download.sh && \
        python /workspace/Docker/download.py && \
        python -m nltk.downloader averaged_perceptron_tagger cmudict; \
    fi


# Copy the rest of the application
COPY . /workspace

EXPOSE 9871 9872 9873 9874 9880

CMD ["python", "webui.py"]

本文对官方的Dockerfile做了简单修改以便满足自己需要。采用基础镜像pytorch/pytorch:2.2.1-cuda12.1-cudnn8-runtime
同时需要修改镜像的语言的环境变量，否则webui 会展示为英文界面：
ENV LANG=zh_CN.UTF-8

nltk_data 文件下载

如果启动时找不到nltk_data 的几个语料库，系统会自动下载，但时前提是你的网络已经完美的科学上网了。否则不出意外应该会出错。 nltk_data 文件可以挂载到容器，但也可以通过更改Dockerfile直接拷贝到镜像中。
下载地址：
https://www.nltk.org/nltk_data/
需要下载的语料库：

cmudict
averaged_perceptron_tagger

存放目录结构：

root@itserver03:/u01/workspace/nltk_data# tree
.
├── corpora
│   └── cmudict
│       └── cmudict
└── taggers
    └── averaged_perceptron_tagger
        └── averaged_perceptron_tagger.pickle

4 directories, 2 files

构建image

docker build -t qingcloudtech/gpt-sovits:v1.0 .

运行

修改docker-compose.yaml文件用自己编译的容器镜像：

version: '3.8'

services:
  gpt-sovits:
    image: qingcloudtech/gpt-sovits:v1.0
    container_name: gpt-sovits-container
    environment:
      - is_half=False
      - is_share=False
    volumes:
      - ./output:/workspace/output
      - ./logs:/workspace/logs:rw
      - ./TEMP:/workspace/TEMP
      - /u01/workspace/nltk_data:/usr/share/nltk_data
      - /u01/workspace/models/GPT-SoVITS/pretrained_models:/workspace/GPT_SoVITS/pretrained_models
      - /u01/workspace/models/GPT-SoVITS/SoVITS_weights:/workspace/SoVITS_weights
      - /u01/workspace/models/GPT-SoVITS/GPT_weights:/workspace/GPT_weights
      - /u01/workspace/models/GPT-SoVITS/tools/asr/models:/workspace/tools/asr/models
      - /u01/workspace/models/GPT-SoVITS/tools/uvr5/uvr5_weights:/workspace/tools/uvr5/uvr5_weights
      - ./reference:/workspace/reference
    working_dir: /workspace
    ports:
      - "9880:9880"
      - "9871:9871"
      - "9872:9872"
      - "9873:9873"
      - "9874:9874"
    shm_size: 16G
    deploy:
      resources:
        reservations:
          devices:
          - driver: nvidia
            count: "all"
            capabilities: [gpu]
    stdin_open: true
    tty: true
    restart: unless-stopped

** 注意重点：** 几个挂载文件的路径：

./output:/workspace/output
./logs:/workspace/logs:rw
./TEMP:/workspace/TEMP
/u01/workspace/nltk_data:/usr/share/nltk_data
/u01/workspace/models/GPT-SoVITS/pretrained_models:/workspace/GPT_SoVITS/pretrained_models
/u01/workspace/models/GPT-SoVITS/SoVITS_weights:/workspace/SoVITS_weights
/u01/workspace/models/GPT-SoVITS/GPT_weights:/workspace/GPT_weights
/u01/workspace/models/GPT-SoVITS/tools/asr/models:/workspace/tools/asr/models
/u01/workspace/models/GPT-SoVITS/tools/uvr5/uvr5_weights:/workspace/tools/uvr5/uvr5_weights

运行

docker-compose up -d

几个重要的页面：
下面几个页面并非启动后都可以直接访问，需要根据需要点选相应的启动按钮后方可访问。
主页面： http://127.0.0.1:9874/
UVR5人声伴奏分离&去混响去延迟工具：** http://127.0.0.1:9873/**
TTS推理WebUI：http://127.0.0.1:9872/
语音文本校对标注工具：** http://127.0.0.1:9871/**