1. 背景介绍
一直有在关注文生视频方面的进展信息。比较知名的可能还是sora这类文生视频模型。但今天要讲的是另外的文生视频项目,也是偶然间在git上看到的,项目名称叫MoneyPrinterTurbo,是基于MoneyPrinter项目增加了中文支持,虽然使用方式和sora这类可能类似,通过大模型生成视频的脚本信息,然后通过模型进行生成。但又有其不同点,它是通过视频描述信息生成关键词,然后基于关键词对图像视频网站进行检索出相关视频片段后,进行视频合并,并配音,生成最终的视频,原理类似于视频剪辑,并不复杂。
实际操作下来,发现虽然MoneyPrinterTurbo支持了中文,但是由于视频素材网站还是以外国网站为主,所以基于中文关键词翻译成英文,再去检索的效果比较差。实际体验下来用英文生成视频的效果远好于中文,但质量也不要太报过高的期待,感觉可用但还不够,字幕描述和视频画面一致性还有待提升。由于网络质量问题,执行过程中,从视频素材网站下载内容,可能会出现多次错误,需要频繁尝试才行。假如能利用国内的视频素材,这个项目的生成质量还是可以的。
2. 实验案例展示
2.1 案例1
页面配置:
视频主题:
health and fitness
视频提示语:
Incorporating a healthy lifestyle is essential for overall well-being. Regular exercise, a balanced diet, and adequate sleep are the cornerstones of a healthy routine. Exercise not only helps in maintaining a healthy weight but also improves cardiovascular health, boosts mood, and strengthens muscles and bones. A balanced diet rich in fruits, vegetables, lean proteins, and whole grains provides essential nutrients for the body to function optimally. Additionally, getting enough sleep is crucial for physical and mental health, as it allows the body to recover and rejuvenate. By making these small changes, one can significantly improve their health and fitness.
关键词:
Health and Fitness Lifestyle, Exercise and Nutrition, Cardiovascular Health Benefits, Balanced Diet Importance, Muscle and Bone Strength
执行日志:
2024-07-28 15:06:36.803 | INFO | __main__:<module>:783 - 开始生成视频
2024-07-28 15:06:36.804 | INFO | __main__:<module>:784 - {
"video_subject": "health and fitness",
"video_script": "Incorporating a healthy lifestyle is essential for overall well-being. Regular exercise, a balanced diet, and adequate sleep are the cornerstones of a healthy routine. Exercise not only helps in maintaining a healthy weight but also improves cardiovascular health, boosts mood, and strengthens muscles and bones. A balanced diet rich in fruits, vegetables, lean proteins, and whole grains provides essential nutrients for the body to function optimally. Additionally, getting enough sleep is crucial for physical and mental health, as it allows the body to recover and rejuvenate. By making these small changes, one can significantly improve their health and fitness.",
"video_terms": "Health and Fitness Lifestyle, Exercise and Nutrition, Cardiovascular Health Benefits, Balanced Diet Importance, Muscle and Bone Strength",
"video_aspect": "9:16",
"video_concat_mode": "random",
"video_clip_duration": 5,
"video_count": 1,
"video_source": "pexels",
"video_materials": null,
"video_language": "",
"voice_name": "en-US-BrianNeural-Male",
"voice_volume": 1.0,
"voice_rate": 1.0,
"bgm_type": "random",
"bgm_file": "",
"bgm_volume": 0.2,
"subtitle_enabled": true,
"subtitle_position": "bottom",
"custom_position": 70.0,
"font_name": "MicrosoftYaHeiBold.ttc",
"text_fore_color": "#FFFFFF",
"text_background_color": "transparent",
"font_size": 60,
"stroke_color": "#000000",
"stroke_width": 1.5,
"n_threads": 2,
"paragraph_number": 1
}
2024-07-28 15:06:36.805 | INFO | app.services.task:start:210 - start task: 9e3da794-f3b0-446f-8c9e-576774525370, stop_at: video
2024-07-28 15:06:36.805 | INFO | app.services.task:generate_script:18 -
## generating video script
2024-07-28 15:06:36.806 | DEBUG | app.services.task:generate_script:27 - video script:
Incorporating a healthy lifestyle is essential for overall well-being. Regular exercise, a balanced diet, and adequate sleep are the cornerstones of a healthy routine. Exercise not only helps in maintaining a healthy weight but also improves cardiovascular health, boosts mood, and strengthens muscles and bones. A balanced diet rich in fruits, vegetables, lean proteins, and whole grains provides essential nutrients for the body to function optimally. Additionally, getting enough sleep is crucial for physical and mental health, as it allows the body to recover and rejuvenate. By making these small changes, one can significantly improve their health and fitness.
2024-07-28 15:06:36.806 | INFO | app.services.task:generate_terms:38 -
## generating video terms
2024-07-28 15:06:36.807 | DEBUG | app.services.task:generate_terms:52 - video terms: [
"Health and Fitness Lifestyle",
"Exercise and Nutrition",
"Cardiovascular Health Benefits",
"Balanced Diet Importance",
"Muscle and Bone Strength"
]
2024-07-28 15:06:36.814 | INFO | app.services.task:generate_audio:75 -
## generating audio
2024-07-28 15:06:36.816 | INFO | app.services.voice:azure_tts_v1:1057 - start, voice name: en-US-BrianNeural, try: 1
2024-07-28 15:06:47.004 | INFO | app.services.voice:azure_tts_v1:1077 - completed, output file: ./storage/tasks/9e3da794-f3b0-446f-8c9e-576774525370/audio.mp3
2024-07-28 15:06:47.008 | INFO | app.services.task:generate_subtitle:103 -
## generating subtitle, provider: edge
2024-07-28 15:06:47.015 | INFO | app.services.voice:create_subtitle:1262 - completed, subtitle file created: ./storage/tasks/9e3da794-f3b0-446f-8c9e-576774525370/subtitle.srt, duration: 41.712
2024-07-28 15:06:47.017 | INFO | app.services.task:get_video_materials:141 -
## downloading videos from pexels
2024-07-28 15:06:47.018 | INFO | app.services.material:search_videos_pexels:47 - searching videos: https://api.pexels.com/videos/search?query=Health+and+Fitness+Lifestyle&per_page=20&orientation=portrait, with proxies: {}
2024-07-28 15:06:52.041 | ERROR | app.services.material:search_videos_pexels:83 - search videos failed: HTTPSConnectionPool(host='api.pexels.com', port=443): Max retries exceeded with url: /videos/search?query=Health+and+Fitness+Lifestyle&per_page=20&orientation=portrait (Caused by SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:1007)')))
2024-07-28 15:06:52.044 | INFO | app.services.material:download_videos:208 - found 0 videos for 'Health and Fitness Lifestyle'
2024-07-28 15:06:52.046 | INFO | app.services.material:search_videos_pexels:47 - searching videos: https://api.pexels.com/videos/search?query=Exercise+and+Nutrition&per_page=20&orientation=portrait, with proxies: {}
2024-07-28 15:06:53.399 | INFO | app.services.material:download_videos:208 - found 18 videos for 'Exercise and Nutrition'
2024-07-28 15:06:53.401 | INFO | app.services.material:search_videos_pexels:47 - searching videos: https://api.pexels.com/videos/search?query=Cardiovascular+Health+Benefits&per_page=20&orientation=portrait, with proxies: {}
2024-07-28 15:06:54.544 | INFO | app.services.material:download_videos:208 - found 18 videos for 'Cardiovascular Health Benefits'
2024-07-28 15:06:54.547 | INFO | app.services.material:search_videos_pexels:47 - searching videos: https://api.pexels.com/videos/search?query=Balanced+Diet+Importance&per_page=20&orientation=portrait, with proxies: {}
2024-07-28 15:06:59.567 | ERROR | app.services.material:search_videos_pexels:83 - search videos failed: HTTPSConnectionPool(host='api.pexels.com', port=443): Max retries exceeded with url: /videos/search?query=Balanced+Diet+Importance&per_page=20&orientation=portrait (Caused by SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:1007)')))
2024-07-28 15:06:59.576 | INFO | app.services.material:download_videos:208 - found 0 videos for 'Balanced Diet Importance'
2024-07-28 15:06:59.578 | INFO | app.services.material:search_videos_pexels:47 - searching videos: https://api.pexels.com/videos/search?query=Muscle+and+Bone+Strength&per_page=20&orientation=portrait, with proxies: {}
2024-07-28 15:07:04.596 | ERROR | app.services.material:search_videos_pexels:83 - search videos failed: HTTPSConnectionPool(host='api.pexels.com', port=443): Max retries exceeded with url: /videos/search?query=Muscle+and+Bone+Strength&per_page=20&orientation=portrait (Caused by SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:1007)')))
2024-07-28 15:07:04.600 | INFO | app.services.material:download_videos:208 - found 0 videos for 'Muscle and Bone Strength'
2024-07-28 15:07:04.601 | INFO | app.services.material:download_videos:216 - found total videos: 25, required duration: 42 seconds, found duration: 479.0 seconds
2024-07-28 15:07:04.603 | INFO | app.services.material:download_videos:233 - downloading video: https://videos.pexels.com/video-files/5528012/5528012-hd_1080_1920_25fps.mp4
2024-07-28 15:07:09.299 | INFO | app.services.material:download_videos:238 - video saved: ./storage/cache_videos/vid-cfd843b3fd33e3532131e7a7e78116d4.mp4
2024-07-28 15:07:09.300 | INFO | app.services.material:download_videos:233 - downloading video: https://videos.pexels.com/video-files/5310962/5310962-hd_1080_1920_25fps.mp4
2024-07-28 15:07:09.301 | INFO | app.services.material:save_video:158 - video already exists: ./storage/cache_videos/vid-d0a022f903340954ed6723349c4d7218.mp4
2024-07-28 15:07:09.302 | INFO | app.services.material:download_videos:238 - video saved: ./storage/cache_videos/vid-d0a022f903340954ed6723349c4d7218.mp4
2024-07-28 15:07:09.302 | INFO | app.services.material:download_videos:233 - downloading video: https://videos.pexels.com/video-files/6740718/6740718-hd_1080_1920_25fps.mp4
2024-07-28 15:07:15.488 | INFO | app.services.material:download_videos:238 - video saved: ./storage/cache_videos/vid-d491e5f68e92df5d1c82df611fee789a.mp4
2024-07-28 15:07:15.492 | INFO | app.services.material:download_videos:233 - downloading video: https://videos.pexels.com/video-files/5310966/5310966-hd_1080_1920_25fps.mp4
2024-07-28 15:07:15.495 | INFO | app.services.material:save_video:158 - video already exists: ./storage/cache_videos/vid-d1fd2830013c4c68eea71fc1bcb71325.mp4
2024-07-28 15:07:15.495 | INFO | app.services.material:download_videos:238 - video saved: ./storage/cache_videos/vid-d1fd2830013c4c68eea71fc1bcb71325.mp4
2024-07-28 15:07:15.496 | INFO | app.services.material:download_videos:233 - downloading video: https://videos.pexels.com/video-files/6893205/6893205-hd_1080_1920_25fps.mp4
2024-07-28 15:07:19.030 | INFO | app.services.material:download_videos:238 - video saved: ./storage/cache_videos/vid-37eb1326ec50872f829b98a80159a201.mp4
2024-07-28 15:07:19.031 | INFO | app.services.material:download_videos:233 - downloading video: https://videos.pexels.com/video-files/6707359/6707359-hd_1080_1920_25fps.mp4
2024-07-28 15:07:23.727 | INFO | app.services.material:download_videos:238 - video saved: ./storage/cache_videos/vid-49bf808f0b13a74e93673a230c2d40da.mp4
2024-07-28 15:07:23.728 | INFO | app.services.material:download_videos:233 - downloading video: https://videos.pexels.com/video-files/7017803/7017803-hd_1080_1920_30fps.mp4
2024-07-28 15:07:23.729 | INFO | app.services.material:save_video:158 - video already exists: ./storage/cache_videos/vid-dde20b6d271f06194124dd94bd6f24ff.mp4
2024-07-28 15:07:23.729 | INFO | app.services.material:download_videos:238 - video saved: ./storage/cache_videos/vid-dde20b6d271f06194124dd94bd6f24ff.mp4
2024-07-28 15:07:23.730 | INFO | app.services.material:download_videos:233 - downloading video: https://videos.pexels.com/video-files/5319426/5319426-hd_1080_1920_25fps.mp4
2024-07-28 15:07:28.742 | ERROR | app.services.material:download_videos:248 - failed to download video: {
"provider": "pexels",
"url": "https://videos.pexels.com/video-files/5319426/5319426-hd_1080_1920_25fps.mp4",
"duration": 19
} => HTTPSConnectionPool(host='videos.pexels.com', port=443): Max retries exceeded with url: /video-files/5319426/5319426-hd_1080_1920_25fps.mp4 (Caused by SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:1007)')))
2024-07-28 15:07:28.748 | INFO | app.services.material:download_videos:233 - downloading video: https://videos.pexels.com/video-files/5645107/5645107-hd_1080_1920_25fps.mp4
2024-07-28 15:07:33.356 | INFO | app.services.material:download_videos:238 - video saved: ./storage/cache_videos/vid-25a6036bfe6839e5378354e84b63ae62.mp4
2024-07-28 15:07:33.359 | INFO | app.services.material:download_videos:233 - downloading video: https://videos.pexels.com/video-files/3694919/3694919-hd_1080_1920_30fps.mp4
2024-07-28 15:07:38.374 | ERROR | app.services.material:download_videos:248 - failed to download video: {
"provider": "pexels",
"url": "https://videos.pexels.com/video-files/3694919/3694919-hd_1080_1920_30fps.mp4",
"duration": 23
} => HTTPSConnectionPool(host='videos.pexels.com', port=443): Max retries exceeded with url: /video-files/3694919/3694919-hd_1080_1920_30fps.mp4 (Caused by SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:1007)')))
2024-07-28 15:07:38.377 | INFO | app.services.material:download_videos:233 - downloading video: https://videos.pexels.com/video-files/5320011/5320011-hd_1080_1920_25fps.mp4
2024-07-28 15:07:38.380 | INFO | app.services.material:save_video:158 - video already exists: ./storage/cache_videos/vid-81294f94e5ac183e7de4a0582ff48144.mp4
2024-07-28 15:07:38.382 | INFO | app.services.material:download_videos:238 - video saved: ./storage/cache_videos/vid-81294f94e5ac183e7de4a0582ff48144.mp4
2024-07-28 15:07:38.383 | INFO | app.services.material:download_videos:243 - total duration of downloaded videos: 45.0 seconds, skip downloading more
2024-07-28 15:07:38.385 | SUCCESS | app.services.material:download_videos:249 - downloaded 9 videos
2024-07-28 15:07:38.387 | INFO | app.services.task:generate_final_videos:175 -
## combining video: 1 => ./storage/tasks/9e3da794-f3b0-446f-8c9e-576774525370/combined-1.mp4
2024-07-28 15:07:38.423 | INFO | app.services.video:combine_videos:42 - max duration of audio: 42.1 seconds
2024-07-28 15:07:38.424 | INFO | app.services.video:combine_videos:46 - each clip will be maximum 5 seconds long
2024-07-28 15:07:45.101 | INFO | app.services.video:combine_videos:117 - resizing video to 1080 x 1920, clip size: 720 x 1280
2024-07-28 15:07:45.152 | INFO | app.services.video:combine_videos:117 - resizing video to 1080 x 1920, clip size: 720 x 1280
2024-07-28 15:07:48.307 | INFO | app.services.video:combine_videos:117 - resizing video to 1080 x 1920, clip size: 720 x 1280
2024-07-28 15:07:49.312 | INFO | app.services.video:combine_videos:117 - resizing video to 1080 x 1920, clip size: 720 x 1280
2024-07-28 15:07:49.817 | INFO | app.services.video:combine_videos:129 - writing
2024-07-28 15:08:54.208 | SUCCESS | app.services.video:combine_videos:140 - completed
2024-07-28 15:08:54.213 | INFO | app.services.task:generate_final_videos:191 -
## generating video: 1 => ./storage/tasks/9e3da794-f3b0-446f-8c9e-576774525370/final-1.mp4
2024-07-28 15:08:54.213 | INFO | app.services.video:generate_video:212 - start, video size: 1080 x 1920
2024-07-28 15:08:54.214 | INFO | app.services.video:generate_video:213 - ① video: ./storage/tasks/9e3da794-f3b0-446f-8c9e-576774525370/combined-1.mp4
2024-07-28 15:08:54.214 | INFO | app.services.video:generate_video:214 - ② audio: ./storage/tasks/9e3da794-f3b0-446f-8c9e-576774525370/audio.mp3
2024-07-28 15:08:54.215 | INFO | app.services.video:generate_video:215 - ③ subtitle: ./storage/tasks/9e3da794-f3b0-446f-8c9e-576774525370/subtitle.srt
2024-07-28 15:08:54.215 | INFO | app.services.video:generate_video:216 - ④ output: ./storage/tasks/9e3da794-f3b0-446f-8c9e-576774525370/final-1.mp4
2024-07-28 15:08:54.216 | INFO | app.services.video:generate_video:231 - using font: ./resource/fonts/MicrosoftYaHeiBold.ttc
2024-07-28 15:09:56.414 | SUCCESS | app.services.video:generate_video:302 - completed
2024-07-28 15:09:56.432 | SUCCESS | app.services.task:start:307 - task 9e3da794-f3b0-446f-8c9e-576774525370 finished, generated 1 videos.
2024-07-28 15:09:56.455 | INFO | __main__:<module>:805 - 视频生成完成
日志主要步骤:
## generating video script
## generating video terms
## generating audio
## generating subtitle, provider: edge
## downloading videos from pexels
## combining video
## generating video
生成视频结果:
AIGC文生视频-health and fitness
3. 项目安装流程
项目地址:
MoneyPrinterTurbo
项目功能:
提供一个视频主题,就可以全自动生成视频文案、视频素材、视频字幕、视频背景音乐,然后合成一个高清的短视频。
资源要求:
- 建议最低 CPU 4核或以上,内存 8G 或以上,显卡非必须
- Windows 10 或 MacOS 11.0 以上系统
安装流程:
1. 下载项目
git clone https://github.com/harry0703/MoneyPrinterTurbo.git
2.修改配置文件
- 将
config.example.toml
文件复制一份,命名为config.toml
- 按照
config.toml
文件中的说明,配置好pexels_api_keys
和llm_provider
,并根据 llm_provider 对应的服务商,配置相关的 API Key这里我使用的
llm_provider
是月之暗面,其实这种中文的生成,对于现有绝大多数大模型来说,都不是什么大问题。
3.安装docker并启动
cd MoneyPrinterTurbodocker compose up
4. 访问Web界面
打开浏览器,访问 http://0.0.0.0:8501
就可以看到刚才示例中的编辑页面了