peft / bitsandbytes包windows安装问题
- 环境版本
- 安装peft
- CUDA Setup failed despite GPU being available
- 报错信息
- 解决方法
- ImportError: cannot import name 'is_npu_available' from 'accelerate.utils'
- 报错信息
- 解决方法
- AttributeError: 'NoneType' object has no attribute 'cuDeviceGetCount'
- 报错信息
- 解决方法
- Reference
折腾了一下午这个报错，终于解决了，写一下解决方法下，希望对大家有帮助。
环境版本
- python 3.10
- cuda 118
- windows 10
- torch 2.1.1+cu118
- torchaudio 2.1.1+cu118
- torchvision 0.16.1+cu118
安装peft
pip install https://github.com/jllllll/bitsandbytes-windows-webui/releases/download/wheels/bitsandbytes-0.41.2.post2-py3-none-win_amd64.whl
whl文件来源：https://github.com/jllllll/bitsandbytes-windows-webui/releases/
CUDA Setup failed despite GPU being available
报错信息
>>> import peft
False
===================================BUG REPORT===================================
C:\Language\Anaconda3\envs\llm\lib\site-packages\bitsandbytes\cuda_setup\main.py:166: UserWarning: Welcome to bitsandbytes. For bug reports, please run
python -m bitsandbytes
warn(msg)
================================================================================
The following directories listed in your path were found to be non-existent: {WindowsPath('C'), WindowsPath('/Language/Anaconda3/envs/llm/lib')}
C:\Language\Anaconda3\envs\llm\lib\site-packages\bitsandbytes\cuda_setup\main.py:166: UserWarning: C:\Language\Anaconda3\envs\llm did not contain ['libcudart.so', 'libcudart.so.11.0', 'libcudart.so.12.0'] as expected! Searching further paths...
warn(msg)
CUDA_SETUP: WARNING! libcudart.so not found in any environmental path. Searching in backup paths...
The following directories listed in your path were found to be non-existent: {WindowsPath('/usr/local/cuda/lib64')}
DEBUG: Possible options found for libcudart.so: set()
CUDA SETUP: PyTorch settings found: CUDA_VERSION=118, Highest Compute Capability: 8.6.
CUDA SETUP: To manually override the PyTorch CUDA version please see:https://github.com/TimDettmers/bitsandbytes/blob/main/how_to_use_nonpytorch_cuda.md
CUDA SETUP: Loading binary C:\Language\Anaconda3\envs\llm\lib\site-packages\bitsandbytes\libbitsandbytes_cuda118.so...
argument of type 'WindowsPath' is not iterable
CUDA SETUP: Problem: The main issue seems to be that the main CUDA runtime library was not detected.
CUDA SETUP: Solution 1: To solve the issue the libcudart.so location needs to be added to the LD_LIBRARY_PATH variable
CUDA SETUP: Solution 1a): Find the cuda runtime library via: find / -name libcudart.so 2>/dev/null
CUDA SETUP: Solution 1b): Once the library is found add it to the LD_LIBRARY_PATH: export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:FOUND_PATH_FROM_1a
CUDA SETUP: Solution 1c): For a permanent solution add the export from 1b into your .bashrc file, located at ~/.bashrc
CUDA SETUP: Solution 2: If no library was found in step 1a) you need to install CUDA.
CUDA SETUP: Solution 2a): Download CUDA install script: wget https://github.com/TimDettmers/bitsandbytes/blob/main/cuda_install.sh
CUDA SETUP: Solution 2b): Install desired CUDA version to desired location. The syntax is bash cuda_install.sh CUDA_VERSION PATH_TO_INSTALL_INTO.
CUDA SETUP: Solution 2b): For example, "bash cuda_install.sh 113 ~/local/" will download CUDA 11.3 and install into the folder ~/local
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Language\Anaconda3\envs\llm\lib\site-packages\peft\__init__.py", line 22, in <module>
from .auto import (
File "C:\Language\Anaconda3\envs\llm\lib\site-packages\peft\auto.py", line 31, in <module>
from .mapping import MODEL_TYPE_TO_PEFT_MODEL_MAPPING
File "C:\Language\Anaconda3\envs\llm\lib\site-packages\peft\mapping.py", line 23, in <module>
from .peft_model import (
File "C:\Language\Anaconda3\envs\llm\lib\site-packages\peft\peft_model.py", line 39, in <module>
from .tuners import (
File "C:\Language\Anaconda3\envs\llm\lib\site-packages\peft\tuners\__init__.py", line 21, in <module>
from .lora import LoraConfig, LoraModel
File "C:\Language\Anaconda3\envs\llm\lib\site-packages\peft\tuners\lora\__init__.py", line 21, in <module>
from .model import LoraModel
File "C:\Language\Anaconda3\envs\llm\lib\site-packages\peft\tuners\lora\model.py", line 45, in <module>
import bitsandbytes as bnb
File "C:\Language\Anaconda3\envs\llm\lib\site-packages\bitsandbytes\__init__.py", line 6, in <module>
from . import cuda_setup, utils, research
File "C:\Language\Anaconda3\envs\llm\lib\site-packages\bitsandbytes\research\__init__.py", line 1, in <module>
from . import nn
File "C:\Language\Anaconda3\envs\llm\lib\site-packages\bitsandbytes\research\nn\__init__.py", line 1, in <module>
from .modules import LinearFP8Mixed, LinearFP8Global
File "C:\Language\Anaconda3\envs\llm\lib\site-packages\bitsandbytes\research\nn\modules.py", line 8, in <module>
from bitsandbytes.optim import GlobalOptimManager
File "C:\Language\Anaconda3\envs\llm\lib\site-packages\bitsandbytes\optim\__init__.py", line 6, in <module>
from bitsandbytes.cextension import COMPILED_WITH_CUDA
File "C:\Language\Anaconda3\envs\llm\lib\site-packages\bitsandbytes\cextension.py", line 20, in <module>
raise RuntimeError('''
RuntimeError:
CUDA Setup failed despite GPU being available. Please run the following command to get more information:
python -m bitsandbytes
Inspect the output of the command and see if you can locate CUDA libraries. You might need to add them
to your LD_LIBRARY_PATH. If you suspect a bug, please take the information from python -m bitsandbytes
and open an issue at: https://github.com/TimDettmers/bitsandbytes/issues
解决方法
-
设置环境变量
LD_LIBRARY_PATH=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8\lib\x64BNB_CUDA_VERSION=18
这里写18是因为不知道为什么调用peft的时候会自动拼一个1上去：
-
重新安装
bitsandbytespip uninstall bitsandbytes pip uninstall bitsandbytes-windows pip install https://github.com/jllllll/bitsandbytes-windows-webui/releases/download/wheels/bitsandbytes-0.41.1-py3-none-win_amd64.whl
ImportError: cannot import name ‘is_npu_available’ from ‘accelerate.utils’
报错信息
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ in <module>:1 │
│ │
│ C:\Language\Anaconda3\envs\llm\lib\site-packages\peft\__init__.py:22 in <module> │
│ │
│ 19 │
│ 20 __version__ = "0.6.2" │
│ 21 │
│ ❱ 22 from .auto import ( │
│ 23 │ AutoPeftModel, │
│ 24 │ AutoPeftModelForCausalLM, │
│ 25 │ AutoPeftModelForSequenceClassification, │
│ │
│ C:\Language\Anaconda3\envs\llm\lib\site-packages\peft\auto.py:30 in <module> │
│ │
│ 27 │ AutoModelForTokenClassification, │
│ 28 ) │
│ 29 │
│ ❱ 30 from .config import PeftConfig │
│ 31 from .mapping import MODEL_TYPE_TO_PEFT_MODEL_MAPPING │
│ 32 from .peft_model import ( │
│ 33 │ PeftModel, │
│ │
│ C:\Language\Anaconda3\envs\llm\lib\site-packages\peft\config.py:24 in <module> │
│ │
│ 21 from huggingface_hub import hf_hub_download │
│ 22 from transformers.utils import PushToHubMixin │
│ 23 │
│ ❱ 24 from .utils import CONFIG_NAME, PeftType, TaskType │
│ 25 │
│ 26 │
│ 27 @dataclass │
│ │
│ C:\Language\Anaconda3\envs\llm\lib\site-packages\peft\utils\__init__.py:22 in <module> │
│ │
│ 19 │
│ 20 # from .config import PeftConfig, PeftType, PromptLearningConfig, TaskType │
│ 21 from .peft_types import PeftType, TaskType │
│ ❱ 22 from .other import ( │
│ 23 │ TRANSFORMERS_MODELS_TO_PREFIX_TUNING_POSTPROCESS_MAPPING, │
│ 24 │ TRANSFORMERS_MODELS_TO_LORA_TARGET_MODULES_MAPPING, │
│ 25 │ TRANSFORMERS_MODELS_TO_ADALORA_TARGET_MODULES_MAPPING, │
│ │
│ C:\Language\Anaconda3\envs\llm\lib\site-packages\peft\utils\other.py:23 in <module> │
│ │
│ 20 import accelerate │
│ 21 import torch │
│ 22 from accelerate.hooks import add_hook_to_module, remove_hook_from_module │
│ ❱ 23 from accelerate.utils import is_npu_available, is_xpu_available │
│ 24 from safetensors.torch import storage_ptr, storage_size │
│ 25 │
│ 26 from ..import_utils import is_auto_gptq_available, is_torch_tpu_available │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
ImportError: cannot import name 'is_npu_available' from 'accelerate.utils'
(C:\Language\Anaconda3\envs\llm\lib\site-packages\accelerate\utils\__init__.py)
解决方法
重装accelerate包
pip uninstall accelerate
pip install accelerate
AttributeError: ‘NoneType’ object has no attribute ‘cuDeviceGetCount’
报错信息
===================================BUG REPORT===================================
Welcome to bitsandbytes. For bug reports, please submit your error trace to: https://github.com/TimDettmers/bitsandbytes/issues
================================================================================
binary_path: C:\Language\Anaconda3\envs\llm\lib\site-packages\bitsandbytes\cuda_setup\libbitsandbytes_cuda116.dll
CUDA SETUP: Loading binary C:\Language\Anaconda3\envs\llm\lib\site-packages\bitsandbytes\cuda_setup\libbitsandbytes_cuda116.dll...
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++ DEBUG INFORMATION +++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++ POTENTIALLY LIBRARY-PATH-LIKE ENV VARS ++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
WARNING: Please be sure to sanitize sensible info from any such env vars!
++++++++++++++++++++++++++ OTHER +++++++++++++++++++++++++++
COMPILED_WITH_CUDA = True
Traceback (most recent call last):
File "C:\Language\Anaconda3\envs\llm\lib\runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "C:\Language\Anaconda3\envs\llm\lib\runpy.py", line 86, in _run_code
exec(code, run_globals)
File "C:\Language\Anaconda3\envs\llm\lib\site-packages\bitsandbytes\__main__.py", line 47, in <module>
print(f"COMPUTE_CAPABILITIES_PER_GPU = {get_compute_capabilities(cuda)}")
File "C:\Language\Anaconda3\envs\llm\lib\site-packages\bitsandbytes\cuda_setup\main.py", line 336, in get_compute_capabilities
check_cuda_result(cuda, cuda.cuDeviceGetCount(ct.byref(nGpus)))
AttributeError: 'NoneType' object has no attribute 'cuDeviceGetCount'
解决方法
同问题1
Reference
报错1：https://github.com/TimDettmers/bitsandbytes/issues/748
报错2：https://blog.csdn.net/weixin_43430848/article/details/132759232
