sed 环境配置

news2024/10/6 16:23:33

参考项目来自这里：
https://github.com/DCASE-REPO/DESED_task/tree/master/recipes/dcase2023_task4_baseline

1. 更新自己的 conda

避免一些包在旧的conda 环境中不存在；

conda update conda

使用conda 指定安装对应版本

# CUDA 11.7
conda install pytorch==1.13.1 torchvision==0.14.1 torchaudio==0.13.1 pytorch-cuda=11.7 -c pytorch -c nvidia

然后安装 Pytorch lightning

conda install -y  ffmpeg pandas numba scipy torchmetrics  tqdm pytorch-lightning=1.9

2. 关于 pip install -e 的问题；

2.1 numpy 问题

一开始执行的时候，此时会出现numpy.core.multiarray 相关问题，


  from sed_scores_eval.base_modules.cy_medfilt import cy_medfilt
  File "sed_scores_eval/base_modules/cy_medfilt.pyx", line 1, in init sed_scores_eval.base_modules.cy_medfilt
ImportError: numpy.core.multiarray failed to import (auto-generated because you didn't call 'numpy.import_array()' after cimporting numpy; use '<void>numpy._import_array' to disable if you are certain you don't need it).

经过搜索后，给出的方案是增加 np.import_array()

import numpy as np
cimport numpy as np
cimport cython
from numpy.math cimport INFINITY

# Initialize the NumPy C API
np.import_array()

然后rebuild Cython

python setup.py build_ext --inplace

该问题可能会与 numpy 版本有关系；

笔者在另外一个环境中使用 numpy 1.24.3 没有出现该问题。

而在当前 numpy 1.22.4 中就出现该情况。
Name: numpy
Version: 1.22.4

2.2 cy_detection 导入的问题

  File "/home/anaconda3/envs/SoundDetection/lib/python3.8/site-packages/sed_scores_eval/intersection_based/intermediate_statistics.py", line 4, in <module>
    from sed_scores_eval.base_modules import statistics
  File "/home/anaconda3/envs/SoundDetection/lib/python3.8/site-packages/sed_scores_eval/base_modules/statistics.py", line 6, in <module>
    from sed_scores_eval.base_modules.cy_detection import onset_offset_curves
ModuleNotFoundError: No module named 'sed_scores_eval.base_modules.cy_detection'

对 setup.py 中配制进行修改，
原始的 setup.py 信息如下：

from setuptools import setup

setup(
    name="desed_task",
    version="0.1.1",
    description="Sound Event Detection and Separation in Domestic Environments.",
    author="DCASE Task 4 Organizers",
    author_email="romain.serizel@loria.fr",
    license="MIT",
    packages=["desed_task"],
    python_requires=">=3.8",
    install_requires=[
        "dcase_util>=0.2.16",
        "psds_eval>=0.4.0",
        "sed_eval>=0.2.1",
        "sed_scores_eval>=0.0.0",
    ],
)

修改后的配置文件如下，主要功能是，

添加对应的扩展
会在当前项目下生成 sed_scores_eval/base_modules 文件夹，执行 pip install -e 之后，
会在该文件下面生成两个 .so 共享文件，将这两个共享文件拷贝到该虚拟环境中的位置，即该路径中
~/anaconda3/envs/SoundDetection/lib/python3.8/site-packages/sed_scores_eval/base_modules

#
# from setuptools import setup, Extension
# from Cython.Build import cythonize
# import numpy as np
# import os
#
# # Ensure the output directory exists
# os.makedirs("sed_scores_eval/base_modules", exist_ok=True)
#
# extensions = [
#     Extension(
#         name="sed_scores_eval.base_modules.cy_detection",
#         sources=["/home/xxxxx/anaconda3/envs/SoundDetection/lib/python3.8/site-packages/sed_scores_eval/base_modules/cy_detection.pyx"],
#         #sources=["sed_scores_eval/base_modules/cy_detection.pyx"],
#         include_dirs=[np.get_include()]
#     ),
#
#     Extension(
#         name="sed_scores_eval.base_modules.cy_medfilt",
#         sources=["/home/xxxxx/anaconda3/envs/SoundDetection/lib/python3.8/site-packages/sed_scores_eval/base_modules/cy_medfilt.pyx"],
#         #sources=["sed_scores_eval/base_modules/cy_medfilt.pyx"],
#         include_dirs=[np.get_include()],
#     )
#
# ]
#
# setup(
#     name="desed_task",
#     version="0.1.1",
#     description="Sound Event Detection and Separation in Domestic Environments.",
#     author="DCASE Task 4 Organizers",
#     author_email="romain.serizel@loria.fr",
#     license="MIT",
#     packages=["desed_task"],
#     python_requires=">=3.8",
#     install_requires=[
#         "dcase_util>=0.2.16",
#         "psds_eval>=0.4.0",
#         "sed_eval>=0.2.1",
#         "sed_scores_eval>=0.0.0",
#     ],
#     ext_modules=cythonize(extensions),
#     zip_safe=False,
# )

3. other problem

AttributeError: 'module' object has no attribute 'ndimage'

You have to import the module:

import scipy.ndimage

pip show  numpy  cython
Name: numpy
Version: 1.22.4
Summary: NumPy is the fundamental package for array computing with Python.
Home-page: https://www.numpy.org
Author: Travis E. Oliphant et al.
Author-email: 
License: BSD
Location: /home/respecting_god/anaconda3/envs/SoundDetection/lib/python3.8/site-packages
Requires: 
Required-by: Bottleneck, contourpy, dcase-util, desed, h5py, jams, lazy-dataset, librosa, matplotlib, mir-eval, mkl-fft, mkl-random, musdb, museval, numba, numexpr, pandas, psds-eval, pyloudnorm, pytorch-lightning, resampy, scaper, scikit-learn, scipy, sed-eval, sed_scores_eval, sox, soxr, stempeg, tensorboard, torchmetrics, torchvision
---
Name: Cython
Version: 3.0.11
Summary: The Cython compiler for writing C extensions in the Python language.
Home-page: https://cython.org/
Author: Robert Bradshaw, Stefan Behnel, Dag Seljebotn, Greg Ewing, et al.
Author-email: cython-devel@python.org
License: Apache-2.0
Location: /home/respecting_god/anaconda3/envs/SoundDetection/lib/python3.8/site-packages
Requires: 
Required-by: sed_scores_eval