快速体验LLaMA-Factory 私有化部署和高效微调Llama3模型FAQ

news2024/11/25 10:44:29

序言

之前已经介绍了在超算互联网平台SCNet上使用异构加速卡AI 显存64GB PCIE，私有化部署Llama3模型，并对 Llama3-8B-Instruct 模型进行 LoRA 微调、推理和合并 ，详细内容请参考另一篇博客：快速体验LLaMA-Factory 私有化部署和高效微调Llama3模型（曙光超算互联网平台异构加速卡DCU）。

由于博主调试过程中遇到较多问题，本文记录FAQ相关问题，仅提供解决思路。

一、参考资料

曙光超算互联网平台SCNet之国产异构加速卡DCU

Llama3本地部署与高效微调入门

二、重要说明

当遇到包冲突时，通常使用 pip install --no-deps -e . 可解决绝大多数问题。

三、FAQ

Q：ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
lmdeploy 0.1.0-git782048c.abi0.dtk2404.torch2.1. requires transformers==4.33.2, but you have transformers 4.43.3 which is incompatible.

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
transformers 4.33.2 requires tokenizers!=0.11.3,<0.14,>=0.11.1, but you have tokenizers 0.15.0 which is incompatible.
vllm 0.3.3+git3380931.abi0.dtk2404.torch2.1 requires transformers>=4.38.0, but you have transformers 4.33.2 which is incompatible.

错误原因：错误一要求安装 transformers==4.33.2，安装该版本后，出现错误二。错误二要求安装 transformers>=4.38.0，与错误一相矛盾。

解决方法：解决该问题的思路，请参考下文的FAQ。

Q：pip._vendor.packaging.version.InvalidVersion: Invalid version: '0.1.0-git782048c.abi0.dtk2404.torch2.1.'

ERROR: Exception:
Traceback (most recent call last):
  File "/opt/conda/envs/llama3/lib/python3.10/site-packages/pip/_internal/cli/base_command.py", line 105, in _run_wrapper
    status = _inner_run()
  File "/opt/conda/envs/llama3/lib/python3.10/site-packages/pip/_internal/cli/base_command.py", line 96, in _inner_run
    return self.run(options, args)
  File "/opt/conda/envs/llama3/lib/python3.10/site-packages/pip/_internal/cli/req_command.py", line 67, in wrapper
    return func(self, options, args)
  File "/opt/conda/envs/llama3/lib/python3.10/site-packages/pip/_internal/commands/install.py", line 483, in run
    installed_versions[distribution.canonical_name] = distribution.version
  File "/opt/conda/envs/llama3/lib/python3.10/site-packages/pip/_internal/metadata/pkg_resources.py", line 192, in version
    return parse_version(self._dist.version)
  File "/opt/conda/envs/llama3/lib/python3.10/site-packages/pip/_vendor/packaging/version.py", line 56, in parse
    return Version(version)
  File "/opt/conda/envs/llama3/lib/python3.10/site-packages/pip/_vendor/packaging/version.py", line 202, in __init__
    raise InvalidVersion(f"Invalid version: '{version}'")
pip._vendor.packaging.version.InvalidVersion: Invalid version: '0.1.0-git782048c.abi0.dtk2404.torch2.1.'

(llama_factory_torch) root@notebook-1813389960667746306-scnlbe5oi5-50216:/public/home/scnlbe5oi5/Downloads/models/LLaMA-Factory# pip install tokenizers==0.13
Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
Collecting tokenizers==0.13
  Downloading https://pypi.tuna.tsinghua.edu.cn/packages/cc/67/4c05eb8cbe8d20e52f5f47a9c591738d8cbc2a29e918813b7fcc431ec3db/tokenizers-0.13.0-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (7.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7.0/7.0 MB 37.4 MB/s eta 0:00:00
WARNING: Error parsing dependencies of lmdeploy: Invalid version: '0.1.0-git782048c.abi0.dtk2404.torch2.1.'
WARNING: Error parsing dependencies of mmcv: Invalid version: '2.0.1-gitc0ccf15.abi0.dtk2404.torch2.1.'
Installing collected packages: tokenizers
  Attempting uninstall: tokenizers
    Found existing installation: tokenizers 0.15.0
    Uninstalling tokenizers-0.15.0:
      Successfully uninstalled tokenizers-0.15.0
ERROR: Exception:
Traceback (most recent call last):
  File "/opt/conda/envs/llama3/lib/python3.10/site-packages/pip/_internal/cli/base_command.py", line 105, in _run_wrapper
    status = _inner_run()
  File "/opt/conda/envs/llama3/lib/python3.10/site-packages/pip/_internal/cli/base_command.py", line 96, in _inner_run
    return self.run(options, args)
  File "/opt/conda/envs/llama3/lib/python3.10/site-packages/pip/_internal/cli/req_command.py", line 67, in wrapper
    return func(self, options, args)
  File "/opt/conda/envs/llama3/lib/python3.10/site-packages/pip/_internal/commands/install.py", line 483, in run
    installed_versions[distribution.canonical_name] = distribution.version
  File "/opt/conda/envs/llama3/lib/python3.10/site-packages/pip/_internal/metadata/pkg_resources.py", line 192, in version
    return parse_version(self._dist.version)
  File "/opt/conda/envs/llama3/lib/python3.10/site-packages/pip/_vendor/packaging/version.py", line 56, in parse
    return Version(version)
  File "/opt/conda/envs/llama3/lib/python3.10/site-packages/pip/_vendor/packaging/version.py", line 202, in __init__
    raise InvalidVersion(f"Invalid version: '{version}'")
pip._vendor.packaging.version.InvalidVersion: Invalid version: '0.1.0-git782048c.abi0.dtk2404.torch2.1.'

错误原因：lmdeploy版本问题。

解决方法：解决该问题的思路，请参考下文的FAQ。

Q：版本匹配问题

(llama_factory) root@notebook-1813389960667746306-scnlbe5oi5-17811:/public/home/scnlbe5oi5/Downloads/models/LLaMA-Facto
ry# pip install -r requirements.txt
...
Installing collected packages: pydub, websockets, urllib3, tomlkit, shtab, semantic-version, scipy, ruff, importlib-resources, ffmpy, docstring-parser, aiofiles, tyro, sse-starlette, tokenizers, gradio-client, transformers, trl, peft, gradio
  Attempting uninstall: websockets
    Found existing installation: websockets 12.0
    Uninstalling websockets-12.0:
      Successfully uninstalled websockets-12.0
  Attempting uninstall: urllib3
    Found existing installation: urllib3 1.26.13
    Uninstalling urllib3-1.26.13:
      Successfully uninstalled urllib3-1.26.13
  Attempting uninstall: tokenizers
    Found existing installation: tokenizers 0.15.0
    Uninstalling tokenizers-0.15.0:
      Successfully uninstalled tokenizers-0.15.0
  Attempting uninstall: transformers
    Found existing installation: transformers 4.38.0
    Uninstalling transformers-4.38.0:
      Successfully uninstalled transformers-4.38.0
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
lmdeploy 0.1.0-git782048c.abi0.dtk2404.torch2.1. requires transformers==4.33.2, but you have transformers 4.43.3 which is incompatible.

错误原因：lmdeploy 0.1.0-git782048c.abi0.dtk2404.torch2.1. 与 transformers 版本冲突，要求 transformers==4.33.2。由于LLaMA-Factory项目要求 transformers>=4.41.2，因此选择升级 lmdeploy 以匹配 transformers 版本。

解决方法：在光合社区中查询并下载安装lmdeploy。以 lmdeploy-0.2.6+das1.1.git6ba90df.abi1.dtk2404.torch2.1.0-cp310-cp310-manylinux_2_31_x86_64.whl 为例，尝试安装 lmdeploy-0.2.6：

root@notebook-1813389960667746306-scnlbe5oi5-17811:~# pip list | grep lmdeploy
lmdeploy                       0.1.0-git782048c.abi0.dtk2404.torch2.1.
(llama_factory) root@notebook-1813389960667746306-scnlbe5oi5-17811:/public/home/scnlbe5oi5/Downloads/cache# pip install  lmdeploy-0.2.6+das1.1.git6ba90df.abi1.dtk2404.torch2.1.0-cp310-cp310-manylinux_2_31_x86_64.whl
...
Installing collected packages: shortuuid, tokenizers, transformers, peft, lmdeploy
  Attempting uninstall: tokenizers
    Found existing installation: tokenizers 0.19.1
    Uninstalling tokenizers-0.19.1:
      Successfully uninstalled tokenizers-0.19.1
  Attempting uninstall: transformers
    Found existing installation: transformers 4.43.3
    Uninstalling transformers-4.43.3:
      Successfully uninstalled transformers-4.43.3
  Attempting uninstall: peft
    Found existing installation: peft 0.12.0
    Uninstalling peft-0.12.0:
      Successfully uninstalled peft-0.12.0
  Attempting uninstall: lmdeploy
    Found existing installation: lmdeploy 0.1.0-git782048c.abi0.dtk2404.torch2.1.
    Uninstalling lmdeploy-0.1.0-git782048c.abi0.dtk2404.torch2.1.:
      Successfully uninstalled lmdeploy-0.1.0-git782048c.abi0.dtk2404.torch2.1.
Successfully installed lmdeploy-0.2.6+das1.1.git6ba90df.abi1.dtk2404.torch2.1.0 peft-0.9.0 shortuuid-1.0.13 tokenizers-0.15.2 transformers-4.38.1

lmdeploy-0.2.6 安装成功，且没有报错，但是transformers版本降低为transformers-4.38.1。

重新启动服务，又出现新问题：

(llama_factory) root@notebook-1813389960667746306-scnlbe5oi5-17811:/public/home/scnlbe5oi5/Downloads/models/LLaMA-Facto
ry# python src/webui.py \
>     --model_name_or_path "/public/home/scnlbe5oi5/Downloads/models/LLaMA-Factory/models/Meta-Llama-3-8B-Instruct" \
>     --template llama3 \
>     --infer_backend vllm \
>     --vllm_enforce_eager
Traceback (most recent call last):
  File "/public/home/scnlbe5oi5/Downloads/models/LLaMA-Factory/src/webui.py", line 17, in <module>
    from llamafactory.webui.interface import create_ui
  File "/public/home/scnlbe5oi5/Downloads/models/LLaMA-Factory/src/llamafactory/__init__.py", line 38, in <module>
    from .cli import VERSION
  File "/public/home/scnlbe5oi5/Downloads/models/LLaMA-Factory/src/llamafactory/cli.py", line 21, in <module>
    from . import launcher
  File "/public/home/scnlbe5oi5/Downloads/models/LLaMA-Factory/src/llamafactory/launcher.py", line 15, in <module>
    from llamafactory.train.tuner import run_exp
  File "/public/home/scnlbe5oi5/Downloads/models/LLaMA-Factory/src/llamafactory/train/tuner.py", line 25, in <module>
    from ..hparams import get_infer_args, get_train_args
  File "/public/home/scnlbe5oi5/Downloads/models/LLaMA-Factory/src/llamafactory/hparams/__init__.py", line 20, in <module>
    from .parser import get_eval_args, get_infer_args, get_train_args
  File "/public/home/scnlbe5oi5/Downloads/models/LLaMA-Factory/src/llamafactory/hparams/parser.py", line 45, in <module>
    check_dependencies()
  File "/public/home/scnlbe5oi5/Downloads/models/LLaMA-Factory/src/llamafactory/extras/misc.py", line 82, in check_dependencies
    require_version("transformers>=4.41.2", "To fix: pip install transformers>=4.41.2")
  File "/opt/conda/envs/llama_factory/lib/python3.10/site-packages/transformers/utils/versions.py", line 111, in require_version
    _compare_versions(op, got_ver, want_ver, requirement, pkg, hint)
  File "/opt/conda/envs/llama_factory/lib/python3.10/site-packages/transformers/utils/versions.py", line 44, in _compare_versions
    raise ImportError(
ImportError: transformers>=4.41.2 is required for a normal functioning of this module, but found transformers==4.38.1.
To fix: pip install transformers>=4.41.2

解决方法：升级 transformers，又出现新问题：

(llama_factory) root@notebook-1813389960667746306-scnlbe5oi5-17811:/public/home/scnlbe5oi5/Downloads/cache# pip install -U transformers
...
Installing collected packages: tokenizers, transformers
  Attempting uninstall: tokenizers
    Found existing installation: tokenizers 0.15.2
    Uninstalling tokenizers-0.15.2:
      Successfully uninstalled tokenizers-0.15.2
  Attempting uninstall: transformers
    Found existing installation: transformers 4.38.1
    Uninstalling transformers-4.38.1:
      Successfully uninstalled transformers-4.38.1
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
lmdeploy 0.2.6+das1.1.git6ba90df.abi1.dtk2404.torch2.1.0 requires transformers<=4.38.1,>=4.33.0, but you have transformers 4.43.3 which is incompatible.
Successfully installed tokenizers-0.19.1 transformers-4.43.3

错误原因：lmdeploy 0.2.6 与 transformers 版本冲突，要求 transformers<=4.38.1,>=4.33.0。由于LLaMA-Factory项目要求 transformers>=4.41.2，因此选择继续升级 lmdeploy 以匹配 transformers 版本。

解决方法：升级 lmdeploy：

(llama_factory) root@notebook-1813389960667746306-scnlbe5oi5-17811:/public/home/scnlbe5oi5/Downloads/cache# pip install -U lmdeploy
...
Installing collected packages: nvidia-nccl-cu12, nvidia-curand-cu12, nvidia-cuda-runtime-cu12, nvidia-cublas-cu12, lmdeploy
  Attempting uninstall: lmdeploy
    Found existing installation: lmdeploy 0.2.6+das1.1.git6ba90df.abi1.dtk2404.torch2.1.0
    Uninstalling lmdeploy-0.2.6+das1.1.git6ba90df.abi1.dtk2404.torch2.1.0:
      Successfully uninstalled lmdeploy-0.2.6+das1.1.git6ba90df.abi1.dtk2404.torch2.1.0
Successfully installed lmdeploy-0.5.2.post1 nvidia-cublas-cu12-12.5.3.2 nvidia-cuda-runtime-cu12-12.5.82 nvidia-curand-cu12-10.3.6.82 nvidia-nccl-cu12-2.22.3

lmdeploy-0.5.2 安装成功，且没有报错。

重新启动服务，又出现新问题：

(llama_factory) root@notebook-1813389960667746306-scnlbe5oi5-17811:/public/home/scnlbe5oi5/Downloads/models/LLaMA-Facto
ry# python src/webui.py     --model_name_or_path "/public/home/scnlbe5oi5/Downloads/models/LLaMA-Factory/models/Meta-Llama-3-8B-Instruct"     --template llama3     --infer_backend vllm     --vllm_enforce_eager
Traceback (most recent call last):
  File "/public/home/scnlbe5oi5/Downloads/models/LLaMA-Factory/src/webui.py", line 17, in <module>
    from llamafactory.webui.interface import create_ui
  File "/public/home/scnlbe5oi5/Downloads/models/LLaMA-Factory/src/llamafactory/__init__.py", line 38, in <module>
    from .cli import VERSION
  File "/public/home/scnlbe5oi5/Downloads/models/LLaMA-Factory/src/llamafactory/cli.py", line 21, in <module>
    from . import launcher
  File "/public/home/scnlbe5oi5/Downloads/models/LLaMA-Factory/src/llamafactory/launcher.py", line 15, in <module>
    from llamafactory.train.tuner import run_exp
  File "/public/home/scnlbe5oi5/Downloads/models/LLaMA-Factory/src/llamafactory/train/tuner.py", line 25, in <module>
    from ..hparams import get_infer_args, get_train_args
  File "/public/home/scnlbe5oi5/Downloads/models/LLaMA-Factory/src/llamafactory/hparams/__init__.py", line 20, in <module>
    from .parser import get_eval_args, get_infer_args, get_train_args
  File "/public/home/scnlbe5oi5/Downloads/models/LLaMA-Factory/src/llamafactory/hparams/parser.py", line 45, in <module>
    check_dependencies()
  File "/public/home/scnlbe5oi5/Downloads/models/LLaMA-Factory/src/llamafactory/extras/misc.py", line 85, in check_dependencies
    require_version("peft>=0.11.1", "To fix: pip install peft>=0.11.1")
  File "/opt/conda/envs/llama_factory/lib/python3.10/site-packages/transformers/utils/versions.py", line 111, in require_version
    _compare_versions(op, got_ver, want_ver, requirement, pkg, hint)
  File "/opt/conda/envs/llama_factory/lib/python3.10/site-packages/transformers/utils/versions.py", line 44, in _compare_versions
    raise ImportError(
ImportError: peft>=0.11.1 is required for a normal functioning of this module, but found peft==0.9.0.
To fix: pip install peft>=0.11.1

解决方法：安装 peft==0.11.1：

(llama_factory) root@notebook-1813389960667746306-scnlbe5oi5-17811:/public/home/scnlbe5oi5/Downloads/cache# pip install peft==0.11.1
...
Installing collected packages: peft
  Attempting uninstall: peft
    Found existing installation: peft 0.12.0
    Uninstalling peft-0.12.0:
      Successfully uninstalled peft-0.12.0
Successfully installed peft-0.11.1

peft==0.11.1 安装成功，且没有报错。

重新启动服务，又出现新问题：

(llama_factory) root@notebook-1813389960667746306-scnlbe5oi5-17811:/public/home/scnlbe5oi5/Downloads/models/LLaMA-Facto
ry# python src/webui.py     --model_name_or_path "/public/home/scnlbe5oi5/Downloads/models/LLaMA-Factory/models/Meta-Llama-3-8B-Instruct"     --template llama3     --infer_backend vllm     --vllm_enforce_eager
[2024-07-31 15:23:04,562] [INFO] [real_accelerator.py:158:get_accelerator] Setting ds_accelerator to cuda (auto detect)
Traceback (most recent call last):
  File "/public/home/scnlbe5oi5/Downloads/models/LLaMA-Factory/src/webui.py", line 17, in <module>
    from llamafactory.webui.interface import create_ui
  File "/public/home/scnlbe5oi5/Downloads/models/LLaMA-Factory/src/llamafactory/__init__.py", line 38, in <module>
    from .cli import VERSION
  File "/public/home/scnlbe5oi5/Downloads/models/LLaMA-Factory/src/llamafactory/cli.py", line 22, in <module>
    from .api.app import run_api
  File "/public/home/scnlbe5oi5/Downloads/models/LLaMA-Factory/src/llamafactory/api/app.py", line 21, in <module>
    from ..chat import ChatModel
  File "/public/home/scnlbe5oi5/Downloads/models/LLaMA-Factory/src/llamafactory/chat/__init__.py", line 16, in <module>
    from .chat_model import ChatModel
  File "/public/home/scnlbe5oi5/Downloads/models/LLaMA-Factory/src/llamafactory/chat/chat_model.py", line 26, in <module>
    from .vllm_engine import VllmEngine
  File "/public/home/scnlbe5oi5/Downloads/models/LLaMA-Factory/src/llamafactory/chat/vllm_engine.py", line 37, in <module>
    from vllm.sequence import MultiModalData
ImportError: cannot import name 'MultiModalData' from 'vllm.sequence' (/opt/conda/envs/llama_factory/lib/python3.10/site-packages/vllm/sequence.py)

该问题的解决方法，请参考下文的FAQ。

Q：ImportError: cannot import name 'MultiModalData' from 'vllm.sequence'

最新取用的代码，运行api.py（或者是webui.py）报错，错误信息均是：ImportError: cannot import name ‘MultiModalData’ from ‘vllm.sequence’ (/usr/local/lib/python3.10/dist-packages/vllm/sequence.py) #3645

ImportError: cannot import name 'MultiModalData' from 'vllm.sequence'

错误原因：vllm版本过高或者版本过低，而LLaMA-Factory项目要求最低版本 vllm==0.4.3。

解决方法：以版本过高为例，将vllm版本从 vllm==0.5.0 降低到 vllm==0.4.3，又出现新问题：

(llama_factory) root@notebook-1813389960667746306-scnlbe5oi5-17811:/public/home/scnlbe5oi5/Downloads/models/LLaMA-Facto
ry# pip install vllm==0.4.3
...
Installing collected packages: nvidia-ml-py, triton, nvidia-nvtx-cu12, nvidia-nvjitlink-cu12, nvidia-nccl-cu12, nvidia-curand-cu12, nvidia-cufft-cu12, nvidia-cuda-runtime-cu12, nvidia-cuda-nvrtc-cu12, nvidia-cuda-cupti-cu12, nvidia-cublas-cu12, llvmlite, lark, joblib, interegular, distro, diskcache, cmake, cloudpickle, nvidia-cusparse-cu12, nvidia-cudnn-cu12, numba, prometheus-fastapi-instrumentator, openai, nvidia-cusolver-cu12, lm-format-enforcer, torch, xformers, vllm-flash-attn, outlines, vllm
  Attempting uninstall: triton
    Found existing installation: triton 2.1.0+git3841f975.abi0.dtk2404
    Uninstalling triton-2.1.0+git3841f975.abi0.dtk2404:
      Successfully uninstalled triton-2.1.0+git3841f975.abi0.dtk2404
  Attempting uninstall: nvidia-nccl-cu12
    Found existing installation: nvidia-nccl-cu12 2.22.3
    Uninstalling nvidia-nccl-cu12-2.22.3:
      Successfully uninstalled nvidia-nccl-cu12-2.22.3
  Attempting uninstall: nvidia-curand-cu12
    Found existing installation: nvidia-curand-cu12 10.3.6.82
    Uninstalling nvidia-curand-cu12-10.3.6.82:
      Successfully uninstalled nvidia-curand-cu12-10.3.6.82
  Attempting uninstall: nvidia-cuda-runtime-cu12
    Found existing installation: nvidia-cuda-runtime-cu12 12.5.82
    Uninstalling nvidia-cuda-runtime-cu12-12.5.82:
      Successfully uninstalled nvidia-cuda-runtime-cu12-12.5.82
  Attempting uninstall: nvidia-cublas-cu12
    Found existing installation: nvidia-cublas-cu12 12.5.3.2
    Uninstalling nvidia-cublas-cu12-12.5.3.2:
      Successfully uninstalled nvidia-cublas-cu12-12.5.3.2
  Attempting uninstall: torch
    Found existing installation: torch 2.1.0+git00661e0.abi0.dtk2404
    Uninstalling torch-2.1.0+git00661e0.abi0.dtk2404:
      Successfully uninstalled torch-2.1.0+git00661e0.abi0.dtk2404
  Attempting uninstall: xformers
    Found existing installation: xformers 0.0.25+gitd11e899.abi0.dtk2404.torch2.1
    Uninstalling xformers-0.0.25+gitd11e899.abi0.dtk2404.torch2.1:
      Successfully uninstalled xformers-0.0.25+gitd11e899.abi0.dtk2404.torch2.1
  Attempting uninstall: vllm
    Found existing installation: vllm 0.3.3+git3380931.abi0.dtk2404.torch2.1
    Uninstalling vllm-0.3.3+git3380931.abi0.dtk2404.torch2.1:
      Successfully uninstalled vllm-0.3.3+git3380931.abi0.dtk2404.torch2.1
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
lmdeploy 0.5.2.post1 requires torch<=2.2.2,>=2.0.0, but you have torch 2.3.0 which is incompatible.
lmdeploy 0.5.2.post1 requires triton<=2.2.0,>=2.1.0; sys_platform == "linux", but you have triton 2.3.0 which is incompatible.
Successfully installed cloudpickle-3.0.0 cmake-3.30.1 diskcache-5.6.3 distro-1.9.0 interegular-0.3.3 joblib-1.4.2 lark-1.1.9 llvmlite-0.43.0 lm-format-enforcer-0.10.1 numba-0.60.0 nvidia-cublas-cu12-12.1.3.1 nvidia-cuda-cupti-cu12-12.1.105 nvidia-cuda-nvrtc-cu12-12.1.105 nvidia-cuda-runtime-cu12-12.1.105 nvidia-cudnn-cu12-8.9.2.26 nvidia-cufft-cu12-11.0.2.54 nvidia-curand-cu12-10.3.2.106 nvidia-cusolver-cu12-11.4.5.107 nvidia-cusparse-cu12-12.1.0.106 nvidia-ml-py-12.555.43 nvidia-nccl-cu12-2.20.5 nvidia-nvjitlink-cu12-12.5.82 nvidia-nvtx-cu12-12.1.105 openai-1.37.1 outlines-0.0.34 prometheus-fastapi-instrumentator-7.0.0 torch-2.3.0 triton-2.3.0 vllm-0.4.3 vllm-flash-attn-2.5.8.post2 xformers-0.0.26.post1

解决方法：将torch版本从 torch 2.3.0 降低到 torch 2.1.0，又出现新问题：

(llama_factory) root@notebook-1813389960667746306-scnlbe5oi5-17811:/public/home/scnlbe5oi5/Downloads/models/LLaMA-Facto
ry# pip install torch==2.1.0
...
Installing collected packages: triton, nvidia-nccl-cu12, torch
  Attempting uninstall: triton
    Found existing installation: triton 2.3.0
    Uninstalling triton-2.3.0:
      Successfully uninstalled triton-2.3.0
  Attempting uninstall: nvidia-nccl-cu12
    Found existing installation: nvidia-nccl-cu12 2.20.5
    Uninstalling nvidia-nccl-cu12-2.20.5:
      Successfully uninstalled nvidia-nccl-cu12-2.20.5
  Attempting uninstall: torch
    Found existing installation: torch 2.3.0
    Uninstalling torch-2.3.0:
      Successfully uninstalled torch-2.3.0
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
vllm 0.4.3 requires torch==2.3.0, but you have torch 2.1.0 which is incompatible.
vllm-flash-attn 2.5.8.post2 requires torch==2.3.0, but you have torch 2.1.0 which is incompatible.
xformers 0.0.26.post1 requires torch==2.3.0, but you have torch 2.1.0 which is incompatible.
Successfully installed nvidia-nccl-cu12-2.18.1 torch-2.1.0 triton-2.1.0

解决方法：将vllm版本从 vllm 0.4.3 降低到 vllm 0.4.2，又出现新问题：

(llama_factory) root@notebook-1813389960667746306-scnlbe5oi5-17811:/public/home/scnlbe5oi5/Downloads/models/LLaMA-Facto
ry# pip install vllm==0.4.2
...
Installing collected packages: vllm-nccl-cu12, triton, nvidia-nccl-cu12, tiktoken, torch, lm-format-enforcer, vllm
  Attempting uninstall: triton
    Found existing installation: triton 2.1.0
    Uninstalling triton-2.1.0:
      Successfully uninstalled triton-2.1.0
  Attempting uninstall: nvidia-nccl-cu12
    Found existing installation: nvidia-nccl-cu12 2.18.1
    Uninstalling nvidia-nccl-cu12-2.18.1:
      Successfully uninstalled nvidia-nccl-cu12-2.18.1
  Attempting uninstall: tiktoken
    Found existing installation: tiktoken 0.7.0
    Uninstalling tiktoken-0.7.0:
      Successfully uninstalled tiktoken-0.7.0
  Attempting uninstall: torch
    Found existing installation: torch 2.1.0
    Uninstalling torch-2.1.0:
      Successfully uninstalled torch-2.1.0
  Attempting uninstall: lm-format-enforcer
    Found existing installation: lm-format-enforcer 0.10.1
    Uninstalling lm-format-enforcer-0.10.1:
      Successfully uninstalled lm-format-enforcer-0.10.1
  Attempting uninstall: vllm
    Found existing installation: vllm 0.4.3
    Uninstalling vllm-0.4.3:
      Successfully uninstalled vllm-0.4.3
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
lmdeploy 0.5.2.post1 requires torch<=2.2.2,>=2.0.0, but you have torch 2.3.0 which is incompatible.
lmdeploy 0.5.2.post1 requires triton<=2.2.0,>=2.1.0; sys_platform == "linux", but you have triton 2.3.0 which is incompatible.
Successfully installed lm-format-enforcer-0.9.8 nvidia-nccl-cu12-2.20.5 tiktoken-0.6.0 torch-2.3.0 triton-2.3.0 vllm-0.4.2 vllm-nccl-cu12-2.18.1.0.4.0

解决方法：将vllm版本从 vllm 0.4.2 降低到 vllm 0.4.1，又出现新问题：

(llama_factory) root@notebook-1813389960667746306-scnlbe5oi5-17811:/public/home/scnlbe5oi5/Downloads/models/LLaMA-Facto
ry# pip install vllm==0.4.1
...
Installing collected packages: triton, nvidia-nccl-cu12, torch, xformers, vllm
  Attempting uninstall: triton
    Found existing installation: triton 2.3.0
    Uninstalling triton-2.3.0:
      Successfully uninstalled triton-2.3.0
  Attempting uninstall: nvidia-nccl-cu12
    Found existing installation: nvidia-nccl-cu12 2.20.5
    Uninstalling nvidia-nccl-cu12-2.20.5:
      Successfully uninstalled nvidia-nccl-cu12-2.20.5
  Attempting uninstall: torch
    Found existing installation: torch 2.3.0
    Uninstalling torch-2.3.0:
      Successfully uninstalled torch-2.3.0
  Attempting uninstall: xformers
    Found existing installation: xformers 0.0.26.post1
    Uninstalling xformers-0.0.26.post1:
      Successfully uninstalled xformers-0.0.26.post1
  Attempting uninstall: vllm
    Found existing installation: vllm 0.4.2
    Uninstalling vllm-0.4.2:
      Successfully uninstalled vllm-0.4.2
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
vllm-flash-attn 2.5.8.post2 requires torch==2.3.0, but you have torch 2.2.1 which is incompatible.
Successfully installed nvidia-nccl-cu12-2.19.3 torch-2.2.1 triton-2.2.0 vllm-0.4.1 xformers-0.0.25

解决方法：将vllm版本从 vllm-flash-attn 2.5.8.post2 降低到 vllm-flash-attn-2.5.6，又出现新问题：

(llama_factory) root@notebook-1813389960667746306-scnlbe5oi5-17811:/public/home/scnlbe5oi5/Downloads/models/LLaMA-Facto
ry# pip install vllm-flash-attn==2.5.6
...
Installing collected packages: triton, nvidia-nccl-cu12, torch, vllm-flash-attn
  Attempting uninstall: triton
    Found existing installation: triton 2.2.0
    Uninstalling triton-2.2.0:
      Successfully uninstalled triton-2.2.0
  Attempting uninstall: nvidia-nccl-cu12
    Found existing installation: nvidia-nccl-cu12 2.19.3
    Uninstalling nvidia-nccl-cu12-2.19.3:
      Successfully uninstalled nvidia-nccl-cu12-2.19.3
  Attempting uninstall: torch
    Found existing installation: torch 2.2.1
    Uninstalling torch-2.2.1:
      Successfully uninstalled torch-2.2.1
  Attempting uninstall: vllm-flash-attn
    Found existing installation: vllm-flash-attn 2.5.8.post2
    Uninstalling vllm-flash-attn-2.5.8.post2:
      Successfully uninstalled vllm-flash-attn-2.5.8.post2
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
vllm 0.4.1 requires torch==2.2.1, but you have torch 2.1.2 which is incompatible.
xformers 0.0.25 requires torch==2.2.1, but you have torch 2.1.2 which is incompatible.
Successfully installed nvidia-nccl-cu12-2.18.1 torch-2.1.2 triton-2.1.0 vllm-flash-attn-2.5.6

解决方法：将vllm版本从 vllm 0.4.1 降低到 vllm 0.4.0：

(llama_factory) root@notebook-1813389960667746306-scnlbe5oi5-17811:/public/home/scnlbe5oi5/Downloads/models/LLaMA-Facto
ry# pip install vllm==0.4.0
...
Installing collected packages: xformers, vllm
  Attempting uninstall: xformers
    Found existing installation: xformers 0.0.25
    Uninstalling xformers-0.0.25:
      Successfully uninstalled xformers-0.0.25
  Attempting uninstall: vllm
    Found existing installation: vllm 0.4.1
    Uninstalling vllm-0.4.1:
      Successfully uninstalled vllm-0.4.1
Successfully installed vllm-0.4.0 xformers-0.0.23.post1

vllm 0.4.0 安装成功，且没有报错。

重新启动服务，又出现新问题：

(llama_factory) root@notebook-1813389960667746306-scnlbe5oi5-17811:/public/home/scnlbe5oi5/Downloads/models/LLaMA-Facto
ry# python src/webui.py \
>     --model_name_or_path "/public/home/scnlbe5oi5/Downloads/models/LLaMA-Factory/models/Meta-Llama-3-8B-Instruct" \
>     --template llama3 \
>     --infer_backend vllm \
>     --vllm_enforce_eager
No ROCm runtime is found, using ROCM_HOME='/opt/dtk'
/opt/conda/envs/llama_factory/lib/python3.10/site-packages/torchvision/io/image.py:13: UserWarning: Failed to load image Python extension: 'libc10_hip.so: cannot open shared object file: No such file or directory'If you don't plan on using image functionality from `torchvision.io`, you can ignore this warning. Otherwise, there might be something wrong with your environment. Did you have `libjpeg` or `libpng` installed before building `torchvision` from source?
  warn(
[2024-07-31 15:52:48,647] [INFO] [real_accelerator.py:158:get_accelerator] Setting ds_accelerator to cuda (auto detect)
Traceback (most recent call last):
  File "/public/home/scnlbe5oi5/Downloads/models/LLaMA-Factory/src/webui.py", line 17, in <module>
    from llamafactory.webui.interface import create_ui
  File "/public/home/scnlbe5oi5/Downloads/models/LLaMA-Factory/src/llamafactory/__init__.py", line 38, in <module>
    from .cli import VERSION
  File "/public/home/scnlbe5oi5/Downloads/models/LLaMA-Factory/src/llamafactory/cli.py", line 22, in <module>
    from .api.app import run_api
  File "/public/home/scnlbe5oi5/Downloads/models/LLaMA-Factory/src/llamafactory/api/app.py", line 21, in <module>
    from ..chat import ChatModel
  File "/public/home/scnlbe5oi5/Downloads/models/LLaMA-Factory/src/llamafactory/chat/__init__.py", line 16, in <module>
    from .chat_model import ChatModel
  File "/public/home/scnlbe5oi5/Downloads/models/LLaMA-Factory/src/llamafactory/chat/chat_model.py", line 26, in <module>
    from .vllm_engine import VllmEngine
  File "/public/home/scnlbe5oi5/Downloads/models/LLaMA-Factory/src/llamafactory/chat/vllm_engine.py", line 29, in <module>
    from vllm import AsyncEngineArgs, AsyncLLMEngine, RequestOutput, SamplingParams
  File "/opt/conda/envs/llama_factory/lib/python3.10/site-packages/vllm/__init__.py", line 4, in <module>
    from vllm.engine.async_llm_engine import AsyncLLMEngine
  File "/opt/conda/envs/llama_factory/lib/python3.10/site-packages/vllm/engine/async_llm_engine.py", line 12, in <module>
    from vllm.engine.llm_engine import LLMEngine
  File "/opt/conda/envs/llama_factory/lib/python3.10/site-packages/vllm/engine/llm_engine.py", line 16, in <module>
    from vllm.model_executor.model_loader import get_architecture_class_name
  File "/opt/conda/envs/llama_factory/lib/python3.10/site-packages/vllm/model_executor/model_loader.py", line 10, in <module>
    from vllm.model_executor.models.llava import LlavaForConditionalGeneration
  File "/opt/conda/envs/llama_factory/lib/python3.10/site-packages/vllm/model_executor/models/llava.py", line 11, in <module>
    from vllm.model_executor.layers.activation import get_act_fn
  File "/opt/conda/envs/llama_factory/lib/python3.10/site-packages/vllm/model_executor/layers/activation.py", line 9, in <module>
    from vllm._C import ops
ImportError: libcuda.so.1: cannot open shared object file: No such file or directory

该问题的解决方法，请参考下文的FAQ。

Q：ImportError: libcuda.so.1: cannot open shared object file: No such file or directory

ImportError: libcuda.so.1: cannot open shared object file: No such file or directory

查找 libcuda.so.1 文件：

(llama_factory) root@notebook-1813389960667746306-scnlbe5oi5-17811:/public/home/scnlbe5oi5/Downloads/models/LLaMA-Facto
ry# find / -name "libcuda.so.1"
find: '/proc/1/map_files': Operation not permitted
find: '/proc/13/map_files': Operation not permitted
find: '/proc/45/map_files': Operation not permitted
find: '/proc/116/map_files': Operation not permitted
find: '/proc/118/map_files': Operation not permitted
find: '/proc/120/map_files': Operation not permitted
find: '/proc/121/map_files': Operation not permitted
find: '/proc/5527/map_files': Operation not permitted
find: '/proc/5529/map_files': Operation not permitted
find: '/proc/5531/map_files': Operation not permitted
find: '/proc/6148/map_files': Operation not permitted
find: '/proc/24592/map_files': Operation not permitted
find: '/proc/24970/map_files': Operation not permitted
find: '/proc/24971/map_files': Operation not permitted

错误原因：没有找到该文件，猜测是vllm的版本问题。

解决方法：重新安装 llvm 0.4.3，又出现新问题：

(llama_factory) root@notebook-1813389960667746306-scnlbe5oi5-17811:/public/home/scnlbe5oi5/Downloads/models/LLaMA-Facto
ry# pip install vllm==0.4.3
...
Installing collected packages: triton, nvidia-nccl-cu12, torch, lm-format-enforcer, xformers, vllm-flash-attn, vllm
  Attempting uninstall: triton
    Found existing installation: triton 2.1.0
    Uninstalling triton-2.1.0:
      Successfully uninstalled triton-2.1.0
  Attempting uninstall: nvidia-nccl-cu12
    Found existing installation: nvidia-nccl-cu12 2.18.1
    Uninstalling nvidia-nccl-cu12-2.18.1:
      Successfully uninstalled nvidia-nccl-cu12-2.18.1
  Attempting uninstall: torch
    Found existing installation: torch 2.1.2
    Uninstalling torch-2.1.2:
      Successfully uninstalled torch-2.1.2
  Attempting uninstall: lm-format-enforcer
    Found existing installation: lm-format-enforcer 0.9.8
    Uninstalling lm-format-enforcer-0.9.8:
      Successfully uninstalled lm-format-enforcer-0.9.8
  Attempting uninstall: xformers
    Found existing installation: xformers 0.0.23.post1
    Uninstalling xformers-0.0.23.post1:
      Successfully uninstalled xformers-0.0.23.post1
  Attempting uninstall: vllm-flash-attn
    Found existing installation: vllm-flash-attn 2.5.6
    Uninstalling vllm-flash-attn-2.5.6:
      Successfully uninstalled vllm-flash-attn-2.5.6
  Attempting uninstall: vllm
    Found existing installation: vllm 0.4.0
    Uninstalling vllm-0.4.0:
      Successfully uninstalled vllm-0.4.0
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
lmdeploy 0.5.2.post1 requires torch<=2.2.2,>=2.0.0, but you have torch 2.3.0 which is incompatible.
lmdeploy 0.5.2.post1 requires triton<=2.2.0,>=2.1.0; sys_platform == "linux", but you have triton 2.3.0 which is incompatible.
Successfully installed lm-format-enforcer-0.10.1 nvidia-nccl-cu12-2.20.5 torch-2.3.0 triton-2.3.0 vllm-0.4.3 vllm-flash-attn-2.5.8.post2 xformers-0.0.26.post1

错误原因：lmdeploy 0.5.2.post1 与 torch 版本冲突，要求 torch<=2.2.2,>=2.0.0，而当前版本为torch 2.3.0。lmdeploy 0.5.2.post1 与 triton 版本冲突，要求 triton<=2.2.0,>=2.1.0，而当前版本为triton 2.3.0。理论上，应该升级lmdeploy 版本以匹配torch版本，但是lmdeploy已经是最新版本了。因此，尝试降低lmdeploy版本。

解决方法：将lmdeploy版本从 lmdeploy 0.5.2.post1 降低到 lmdeploy 0.5.0，又出现新问题：

(llama_factory) root@notebook-1813389960667746306-scnlbe5oi5-17811:/public/home/scnlbe5oi5/Downloads/models/LLaMA-Facto
ry# pip install lmdeploy==0.5.0
...
Installing collected packages: triton, nvidia-nccl-cu12, torch, lmdeploy
  Attempting uninstall: triton
    Found existing installation: triton 2.3.0
    Uninstalling triton-2.3.0:
      Successfully uninstalled triton-2.3.0
  Attempting uninstall: nvidia-nccl-cu12
    Found existing installation: nvidia-nccl-cu12 2.20.5
    Uninstalling nvidia-nccl-cu12-2.20.5:
      Successfully uninstalled nvidia-nccl-cu12-2.20.5
  Attempting uninstall: torch
    Found existing installation: torch 2.3.0
    Uninstalling torch-2.3.0:
      Successfully uninstalled torch-2.3.0
  Attempting uninstall: lmdeploy
    Found existing installation: lmdeploy 0.5.2.post1
    Uninstalling lmdeploy-0.5.2.post1:
      Successfully uninstalled lmdeploy-0.5.2.post1
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
vllm 0.4.3 requires torch==2.3.0, but you have torch 2.2.2 which is incompatible.
vllm-flash-attn 2.5.8.post2 requires torch==2.3.0, but you have torch 2.2.2 which is incompatible.
xformers 0.0.26.post1 requires torch==2.3.0, but you have torch 2.2.2 which is incompatible.

解决方法：将torch版本从 torch 2.2.2 升级到 torch 2.3.0，又出现新问题：

(llama_factory) root@notebook-1813389960667746306-scnlbe5oi5-17811:/public/home/scnlbe5oi5/Downloads/models/LLaMA-Facto
ry# pip install torch==2.3.0
...
Installing collected packages: triton, nvidia-nccl-cu12, torch
  Attempting uninstall: triton
    Found existing installation: triton 2.2.0
    Uninstalling triton-2.2.0:
      Successfully uninstalled triton-2.2.0
  Attempting uninstall: nvidia-nccl-cu12
    Found existing installation: nvidia-nccl-cu12 2.19.3
    Uninstalling nvidia-nccl-cu12-2.19.3:
      Successfully uninstalled nvidia-nccl-cu12-2.19.3
  Attempting uninstall: torch
    Found existing installation: torch 2.2.2
    Uninstalling torch-2.2.2:
      Successfully uninstalled torch-2.2.2
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
lmdeploy 0.5.0 requires torch<=2.2.2,>=2.0.0, but you have torch 2.3.0 which is incompatible.
lmdeploy 0.5.0 requires triton<=2.2.0,>=2.1.0; sys_platform == "linux", but you have triton 2.3.0 which is incompatible.
Successfully installed nvidia-nccl-cu12-2.20.5 torch-2.3.0 triton-2.3.0

解决方法：把lmdeploy版本从 lmdeploy 0.5.0 升级到 lmdeploy 0.5.1。

(llama_factory) root@notebook-1813389960667746306-scnlbe5oi5-17811:/public/home/scnlbe5oi5/Downloads/models/LLaMA-Facto
ry# pip install lmdeploy==0.5.1
...
Installing collected packages: lmdeploy
  Attempting uninstall: lmdeploy
    Found existing installation: lmdeploy 0.5.0
    Uninstalling lmdeploy-0.5.0:
      Successfully uninstalled lmdeploy-0.5.0
Successfully installed lmdeploy-0.5.1

lmdeploy-0.5.1 安装成功，且没有报错。

重新启动服务，又出现新问题：

(llama_factory) root@notebook-1813389960667746306-scnlbe5oi5-17811:/public/home/scnlbe5oi5/Downloads/models/LLaMA-Facto
ry# python src/webui.py \
>     --model_name_or_path "/public/home/scnlbe5oi5/Downloads/models/LLaMA-Factory/models/Meta-Llama-3-8B-Instruct" \
>     --template llama3 \
>     --infer_backend vllm \
>     --vllm_enforce_eager
Traceback (most recent call last):
  File "/public/home/scnlbe5oi5/Downloads/models/LLaMA-Factory/src/webui.py", line 17, in <module>
    from llamafactory.webui.interface import create_ui
  File "/public/home/scnlbe5oi5/Downloads/models/LLaMA-Factory/src/llamafactory/__init__.py", line 38, in <module>
    from .cli import VERSION
  File "/public/home/scnlbe5oi5/Downloads/models/LLaMA-Factory/src/llamafactory/cli.py", line 21, in <module>
    from . import launcher
  File "/public/home/scnlbe5oi5/Downloads/models/LLaMA-Factory/src/llamafactory/launcher.py", line 15, in <module>
    from llamafactory.train.tuner import run_exp
  File "/public/home/scnlbe5oi5/Downloads/models/LLaMA-Factory/src/llamafactory/train/tuner.py", line 25, in <module>
    from ..hparams import get_infer_args, get_train_args
  File "/public/home/scnlbe5oi5/Downloads/models/LLaMA-Factory/src/llamafactory/hparams/__init__.py", line 20, in <module>
    from .parser import get_eval_args, get_infer_args, get_train_args
  File "/public/home/scnlbe5oi5/Downloads/models/LLaMA-Factory/src/llamafactory/hparams/parser.py", line 45, in <module>
    check_dependencies()
  File "/public/home/scnlbe5oi5/Downloads/models/LLaMA-Factory/src/llamafactory/extras/misc.py", line 85, in check_dependencies
    require_version("peft>=0.11.1", "To fix: pip install peft>=0.11.1")
  File "/opt/conda/envs/llama_factory/lib/python3.10/site-packages/transformers/utils/versions.py", line 111, in require_version
    _compare_versions(op, got_ver, want_ver, requirement, pkg, hint)
  File "/opt/conda/envs/llama_factory/lib/python3.10/site-packages/transformers/utils/versions.py", line 44, in _compare_versions
    raise ImportError(
ImportError: peft>=0.11.1 is required for a normal functioning of this module, but found peft==0.9.0.
To fix: pip install peft>=0.11.1

解决方法：升级 peft==0.11.1。

(llama_factory) root@notebook-1813389960667746306-scnlbe5oi5-17811:/public/home/scnlbe5oi5/Downloads/cache# pip install peft==0.11.1
...
Installing collected packages: peft
  Attempting uninstall: peft
    Found existing installation: peft 0.9.0
    Uninstalling peft-0.9.0:
      Successfully uninstalled peft-0.9.0
Successfully installed peft-0.11.1

peft-0.11.1 安装成功，且没有报错。

重新启动服务，又出现新问题：

(llama_factory) root@notebook-1813389960667746306-scnlbe5oi5-17811:/public/home/scnlbe5oi5/Downloads/models/LLaMA-Facto
ry# python src/webui.py     --model_name_or_path "/public/home/scnlbe5oi5/Downloads/models/LLaMA-Factory/models/Meta-Llama-3-8B-Instruct"     --template llama3     --infer_backend vllm     --vllm_enforce_eager
No ROCm runtime is found, using ROCM_HOME='/opt/dtk'
/opt/conda/envs/llama_factory/lib/python3.10/site-packages/torchvision/io/image.py:13: UserWarning: Failed to load image Python extension: 'libc10_hip.so: cannot open shared object file: No such file or directory'If you don't plan on using image functionality from `torchvision.io`, you can ignore this warning. Otherwise, there might be something wrong with your environment. Did you have `libjpeg` or `libpng` installed before building `torchvision` from source?
  warn(
[2024-07-31 16:58:35,443] [INFO] [real_accelerator.py:158:get_accelerator] Setting ds_accelerator to cuda (auto detect)
gradio_share: False
Running on local URL:  http://127.0.0.1:7860

Could not create share link. Missing file: /opt/conda/envs/llama_factory/lib/python3.10/site-packages/gradio/frpc_linux_amd64_v0.2.

Please check your internet connection. This can happen if your antivirus software blocks the download of this file. You can install manually by following these steps:

1. Download this file: https://cdn-media.huggingface.co/frpc-gradio-0.2/frpc_linux_amd64
2. Rename the downloaded file to: frpc_linux_amd64_v0.2
3. Move the file to this location: /opt/conda/envs/llama_factory/lib/python3.10/site-packages/gradio

该问题的解决方法，请参考下文的FAQ。

Q. Could not create share link. Missing file:/PATH/TO/gradio/frpc_linux_amd64_v0.2

【Gradio】Could not create share link

在这里插入图片描述

Could not create share link. Missing file: /opt/conda/envs/llama_factory_torch/lib/python3.11/site-packages/gradio/frpc_linux_amd64_v0.2. 

Please check your internet connection. This can happen if your antivirus software blocks the download of this file. You can install manually by following these steps: 

1. Download this file: https://cdn-media.huggingface.co/frpc-gradio-0.2/frpc_linux_amd64
2. Rename the downloaded file to: frpc_linux_amd64_v0.2
3. Move the file to this location: /opt/conda/envs/llama_factory_torch/lib/python3.11/site-packages/gradio

# 解决方法
1. 下载 https://cdn-media.huggingface.co/frpc-gradio-0.2/frpc_linux_amd64

2. 重命名
mv frpc_linux_amd64 frpc_linux_amd64_v0.2

3. 移动到指定目录
cp frpc_linux_amd64_v0.2 /opt/conda/envs/llama_factory_torch/lib/python3.10/site-packages/gradio

4. 修改权限
chmod +x /opt/conda/envs/llama_factory_torch/lib/python3.10/site-packages/gradio/frpc_linux_amd64_v0.2

Q. Could not create share link.. Please check your internet connection or our status page

Could not create share link. Please check your internet connection or our status page: https://status.gradio.app

解决方法：修改 frpc_linux_amd64_v0.2文件权限。

chmod +x /opt/conda/envs/llama_factory_torch/lib/python3.11/site-packages/gradio/frpc_linux_amd64_v0.2

本文来自互联网用户投稿，该文观点仅代表作者本人，不代表本站立场。本站仅提供信息存储空间服务，不拥有所有权，不承担相关法律责任。如若转载，请注明出处：http://www.coloradmin.cn/o/1986131.html

如若内容造成侵权/违法违规/事实不符，请联系多彩编程网进行投诉反馈，一经查实，立即删除！

相关文章

Android14音频进阶之命令行播放音频(八十)

Android14音频进阶之命令行播放音频(八十)

简介： CSDN博客专家，专注Android/Linux系统，分享多mic语音方案、音视频、编解码等技术，与大家一起成长！ 新书发布：《Android系统多媒体进阶实战》🚀 优质专栏： Audio工程师进阶系列【原创干货持续更新中……】🚀 优质专栏： 多媒体系统工程师系列【原创干货持续更…
阅读更多...
SpringDI（依赖注入）的理解

SpringDI（依赖注入）的理解

目录 一、什么是SpringDI&#xff1f; 二、SpringDI&#xff08;依赖注入&#xff09;的作用是什么&#xff1f; 三、DI的实现方式 3.1 set注入 1.项目结构&#xff1a; 2.set注入对象 2.1applicationContext.xml&#xff1a; 2.2 Student 实体类 2.3 controller: 2.4 da…
阅读更多...
SpringIOC容器对Bean管理

SpringIOC容器对Bean管理

一、bean实例化 1.通过构造方法&#xff08;默认&#xff09; 2.通过工厂方法 3.通过静态工厂方法 项目结构&#xff1a; 1. 通过构造方法&#xff08;默认&#xff09; 1.1 pojo实体类&#xff1a; Student: public class Student {public Student() {System.out.println…
阅读更多...
软件安全测试报告内容和作用简析，软件测试服务供应商推荐

软件安全测试报告内容和作用简析，软件测试服务供应商推荐

在数字化时代&#xff0c;软件安全问题愈发凸显&#xff0c;安全测试显得尤为重要。软件安全测试报告是对软件系统在安全性方面进行评估和分析后的书面文件。该报告通常包含测试过程、测试发现、漏洞描述、风险评估及改进建议等重要信息。报告的目的是为了帮助开发团队及时发现…
阅读更多...
各种排序算法【持续更新中.....】

各种排序算法【持续更新中.....】

1.归并排序 归并排序 &#xff0c;归并排序是采用分治法(Divide and Conquer&#xff09;的一个非常典型的应用&#xff0c;所以我们先来说一下什么是分治法。 分治法 定义 分治&#xff08;英语&#xff1a;Divide and Conquer&#xff09;&#xff0c;字面上的解释是「分…
阅读更多...
C++初阶大全

C++初阶大全

目录 一.命名空间 1.命名空间定义 2.命名空间使用 二.C输入&输出 三.缺省参数 四. 函数重载 五.引用 1.常引用 2.传值、传引用效率比较 3.引用和指针的区别 4.引用和指针的不同点: 小知识点: 六.内联函数 七.auto关键字(C11) 1.auto的使用细则 八.基于范围…
阅读更多...
为何你的进销存系统买来却成了摆设？教你7大招解决！

为何你的进销存系统买来却成了摆设？教你7大招解决！

我做企业数字化系统9年&#xff0c;调研的企业大大小小也有几十家了。我发现个挺扎心的现象&#xff1a;有三成企业买的进销存系统&#xff0c;最后都成了闲置的“高科技装饰品”。为啥这些企业舍得掏腰包&#xff0c;到头来却让它们吃灰呢&#xff1f; 说到底&#xff0c;就一…
阅读更多...
开源免费的wiki知识库

开源免费的wiki知识库

开源的Wiki知识库有多种选择&#xff0c;它们各自具有不同的特点和优势&#xff0c;适用于不同的场景和需求。以下是一些主流的开源Wiki知识库系统&#xff1a; MediaWiki 简介&#xff1a;MediaWiki是使用PHP编写的免费开源Wiki软件包&#xff0c;是Wikipedia和其他Wikimedia…
阅读更多...
达梦数据库的系统视图v$mem_reginfo

达梦数据库的系统视图v$mem_reginfo

达梦数据库的系统视图v$mem_reginfo 达梦数据库的V$MEM_REGINFO视图提供了系统当前已分配但未释放的内存信息。这个视图在MEMORY_LEAK_CHECK参数设置为1时才会注册信息&#xff0c;用于监控内存的申请和释放情况。通过查询V$MEM_REGINFO视图&#xff0c;可以关注REFNUM字段&am…
阅读更多...
跟《经济学人》学英文：2024年08月03日这期 Britain’s railways go from one extreme to another

跟《经济学人》学英文：2024年08月03日这期 Britain’s railways go from one extreme to another

Britain’s railways go from one extreme to another Departing: privatisation. Destination: centralisation 出发:私有化。目的地:集中化 depart&#xff1a;出发 privatisation&#xff1a;美 [ˌpraɪvətaɪ’zeɪʃən] 私有化&#xff1b;民营化&#xff1b;私营化…
阅读更多...
IO进程—进程间的通信—共享内存具体函数操作

IO进程—进程间的通信—共享内存具体函数操作

创建唯一key值—— ftok key_t ftok(const char *pathname, int proj_id); #include <sys/types.h> #include <sys/ipc.h> 功能&#xff1a; ftok函数是用于生成一个键值&#xff08;key_t类型&#xff09;的函数&#xff0c;这个键值通常用于进程间通信&#xff0…
阅读更多...
人像修复-通道磨皮

人像修复-通道磨皮

文章目录 原理步骤建立动作一键操作 原理 皮肤看起来比较脏的原因&#xff1a;光影过渡不均匀 步骤 在通道中&#xff0c;找到明暗反差最大的通道&#xff0c;复制一层通道 在复制通道中&#xff0c;用高反差保留&#xff0c;把画面中反差较大的部分保留出来&#xff0c;半…
阅读更多...
解决使用Navicat连接数据库时，打开数据库表很慢的问题

解决使用Navicat连接数据库时，打开数据库表很慢的问题

今天使用Navicat连接数据库时&#xff0c;发现不管表中数据多少&#xff0c;打开数据库表非常慢。 解决方法&#xff1a; Navicat - 右键编辑数据库连接 - 高级 - 勾选保持连接间隔 - 输入框设置为20 - 点击确定&#xff01; 参考文章&#xff1a;https://51.ruyo.net/14030.…
阅读更多...
【ARM】v8架构programmer guide(2)_处理器及V8基础

【ARM】v8架构programmer guide(2)_处理器及V8基础

目录 2.1 ARMv8-A 3. ARMv8的基础 普通世界&#xff08;Non-secure state&#xff09;的特权组件&#xff1a; 安全世界&#xff08;Secure state&#xff09;的特权组件&#xff1a; 安全监视器&#xff08;Secure Monitor&#xff09;&#xff1a; 3.1 execution state…
阅读更多...
实现高级社会的构想-基于人工智能模式

实现高级社会的构想-基于人工智能模式

实现高级社会(共产主义)的构想 刚过完劳动节,劳动最光荣.在新质生产力思想启发下,我想探讨的是我们当下能否从资本主义或社会主义进入或构建高级社会(共产主义社会)呢?高级社会(共产主义),我的理解可以用平均主义来无限接近,即使政策上无法做到平均分配&#xff0c;也可以通过…
阅读更多...
lvs的nat模式配置

lvs的nat模式配置

lvs的nat模式 准备工作 lvs主机需要两个网络适配器&#xff0c;其中一个为NAT模式&#xff0c;另外一个为仅主机模式&#xff0c;还需要两个server主机&#xff0c;一个为server1还有一个为server2&#xff0c;都为仅主机模式 配置过程 配置ip lvs主机 [rootlvs ~]# cat /…
阅读更多...
项目中引入RedisTemplate和Redisson时RedisTemplate无法使用zset问题（栈溢出stackOverflow）深入源码分析解决

项目中引入RedisTemplate和Redisson时RedisTemplate无法使用zset问题（栈溢出stackOverflow）深入源码分析解决

文章目录 项目中引入RedisTemplate和Redisson时RedisTemplate无法使用zset问题&#xff08;栈溢出stackOverflow&#xff09;深入源码分析解决依赖信息报错信息与分析解决办法 项目中引入RedisTemplate和Redisson时RedisTemplate无法使用zset问题&#xff08;栈溢出stackOverfl…
阅读更多...
utf-8‘ codec can‘t decode byte 0xd3 in position

utf-8‘ codec can‘t decode byte 0xd3 in position

1.背景 安装psycopg2-binary&#xff0c;安装命令“pip install psycopg2-binary2.9.7” 完整报错信息如下&#xff1a; (venv) PS D:\workspace_all\pyCharm\MaxKB-main> pip install psycopg2-binary2.9.7 Looking in indexes: https://mirrors.ustc.edu.cn/pypi/web/s…
阅读更多...
Python语法重温1

Python语法重温1

最近做项目的时候&#xff0c;有时总得愣一下语法。所以决定来温习一下。。。 温故而知新&#xff0c;谁都没办法说自己经历时间的流逝后还能时刻记得&#xff0c;希望能帮到你们 列表的替换 如果超出列表元素则会报错。 列表末尾的追加 插入列表操作 可以超出列表之外插入 删…
阅读更多...
常见中间件漏洞复现之【WebLogic】！

常见中间件漏洞复现之【WebLogic】！

Weblogic介绍 WebLogic是美国Oracle公司出品的⼀个application server&#xff0c;确切的说是⼀个基于JAVAEE架构的中间件&#xff0c;默认端⼝&#xff1a;7001 WebLogic是⽤于开发、集成、部署和管理⼤型分布式Web应⽤、⽹络应⽤和数据库应⽤的Java应⽤服务器。将Java的动态…
阅读更多...
推荐文章
最新文章

Copyright @ 2022~2023