搞大模型训练的工作不可避免地需要很多库，但是非常讨厌的事情是这些库动不动就不兼容。最近在做文本分类训练的时候又遇到了这个问题，为了避免后面再安装包的时候把我之前的环境破坏了，所以特地来记录一下：protobuf、streamlit、transformers、icetk、cpm_kernels版本号 ( venv_net) PS D:\ mydatapro\ myweb> pip show protobuf streamlit transformers icetk cpm_kernels
Name: protobuf
Version: 3.20 .3
Summary: Protocol Buffers
Home-page: https://developers.google.com/protocol-buffers/
Author:
Author-email:
License: BSD-3-Clause
Location: D:\ mydatapro\ venv_net\ Lib\ site-packages
Requires:
Required-by: googleapis-common-protos, onnxruntime, opentelemetry-proto, streamlit
---
Name: streamlit
Version: 1.17 .0
Summary: The fastest way to build data apps in Python
Home-page: https://streamlit.io
Author: Snowflake Inc
Author-email: hello@streamlit.io
License: Apache License 2.0
Location: D:\ mydatapro\ venv_net\ Lib\ site-packages
Requires: altair, blinker, cachetools, click, gitpython, importlib-metadata, numpy, packaging, pandas, pillow, protobuf, pyarrow, pydeck, pympler, python-dateutil, requests, rich, semver, toml, tornado, typing-extensions, tzlocal, validators, watchdog
Required-by:
---
Name: transformers
Version: 4.27 .1
Summary: State-of-the-art Machine Learning for JAX, PyTorch and TensorFlow
Home-page: https://github.com/huggingface/transformers
Author: The Hugging Face team ( past and future) with the help of all our contributors ( https://github.com/huggingface/transformers/graphs/contributors)
Author-email: transformers@huggingface.co
License: Apache 2.0 License
Location: D:\ mydatapro\ venv_net\ Lib\ site-packages
Requires: filelock, huggingface-hub, numpy, packaging, pyyaml, regex, requests, tokenizers, tqdm
Required-by: sentence-transformers
---
Name: icetk
Version: 0.0 .3
Summary: A unified tokenization tool for Images, Chinese and English.
Home-page: https://github.com/THUDM/icetk
Author: Wendi Zheng and Ming Ding
Author-email:
License: MIT license
Location: D:\ mydatapro\ venv_net\ Lib\ site-packages
Requires: requests, sentencepiece, torchvision, tqdm
Required-by:
---
Name: cpm-kernels
Version: 1.0 .11
Summary: CPM CUDA kernels
Home-page: UNKNOWN
Author:
Author-email:
License: Apache 2.0
Location: D:\ mydatapro\ venv_net\ Lib\ site-packages
Requires:
Required-by:
