一、前言

SMPL是一种3D人体建模方法.在数字人或者人物角色三维重建领域有着广泛应用
支持人体的各种形状及动作
可以简单理解为通过训练获取的人物模型
常用的模型有 smpl(身体模型),mano(手部模型),smplh(身体+手部),flame(脸部),smplx(身体+手部+脸部) 官网:https://smpl-x.is.tue.mpg.de/index.html https://smpl.is.tue.mpg.de

SMPL 输入是：姿态shape vector $\beta$ + 影响动作Pose的参数 $\theta$
输出是： 各种变化后vertex顶点的数据
作者：Loper, Matthew and Mahmood, Naureen and Romero, Javier and Pons-Moll, Gerard and Black, Michael J.

【Paper】 > 【Code】 > 【Project】
SMPL 是基于蒙皮和混合形状的真实人体 3D 模型，是从数千个 3D 人体扫描中学习得到的。

a. T-pose姿态.常用的模版Pose有T-Pose,A-Pose,Da-Pose $颜色表示身体每个部位受到身体关节点影响的权重$
b. 改变β参数的姿态,J关节点的变化 β参数可以理解为影响人体高矮胖瘦的一个参数
c. 改变β跟θ参数的姿态,此时还没有发生变化
d. 加上W权重通过W的LBS蒙皮算法的姿态

SMPL-X

【Paper】 > 【Code】 > 【Project】
感兴趣的可以看看原文

环境搭建

1. 代码下载

git clone https://github.com/vchoutas/smplx.git 
cd smplx

2. conda 虚拟环境搭建

conda create -n smplx python=3.8 -y 
conda activate smplx

3. torch

conda install -y pytorch==1.13.1 torchvision==0.14.1 torchaudio==0.13.1 pytorch-cuda=11.7 -c pytorch -c nvidia

pip install smplx[all]

4. matplotlib y轴朝上

ax.view_init(azim=-90, elev=100)

5. 下载models，先要注册登录
   SMPL下载：
   
   
   
   将这三个模型上传到项目中，需要改名，KaTeX parse error: Double subscript at position 13: basicmodel_f_̲lbs_10_207_0_v1…，其余类似
   SMPL-X 下载：
   
   
   
   将所需6个文件上传到项目中

models
├── smpl
│   ├── SMPL_FEMALE.pkl
│   └── SMPL_MALE.pkl
│   └── SMPL_NEUTRAL.pkl
├── smplh
│   ├── SMPLH_FEMALE.pkl
│   └── SMPLH_MALE.pkl
├── mano
|   ├── MANO_RIGHT.pkl
|   └── MANO_LEFT.pkl
└── smplx
    ├── SMPLX_FEMALE.npz
    ├── SMPLX_FEMALE.pkl
    ├── SMPLX_MALE.npz
    ├── SMPLX_MALE.pkl
    ├── SMPLX_NEUTRAL.npz
    └── SMPLX_NEUTRAL.pkl

运行demo.py

安装 smplx 包并下载模型参数后，您应该能够运行 demo.py 脚本来可视化结果。对于此步骤，您必须安装pyrender 和trimesh 软件包。

python examples/demo.py --model-folder $SMPLX_FOLDER --plot-joints=True --gender="neutral"

我们可以先用 --plotting-module matplotlib 来先用matplotlib渲染。pip install matplotlib
可以得到对应的输出，远程环境可以保存为png图像查看

elif plotting_module == 'matplotlib':
        from matplotlib import pyplot as plt
        from mpl_toolkits.mplot3d import Axes3D
        from mpl_toolkits.mplot3d.art3d import Poly3DCollection

        fig = plt.figure()
        ax = fig.add_subplot(111, projection='3d')

        mesh = Poly3DCollection(vertices[model.faces], alpha=0.1)
        face_color = (1.0, 1.0, 0.9)
        edge_color = (0, 0, 0)
        mesh.set_edgecolor(edge_color)
        mesh.set_facecolor(face_color)
        ax.add_collection3d(mesh)
        ax.scatter(joints[:, 0], joints[:, 1], joints[:, 2], color='r')

        ax.view_init(azim=-90, elev=100) #  y轴朝上
        if plot_joints:
            ax.scatter(joints[:, 0], joints[:, 1], joints[:, 2], alpha=0.1)
        plt.show()
        plt.savefig("./outputs_zzk/ok"+".png") # 保存观看
        plt.close()

图像是：(y轴朝上)

没改代码之前是：

pyrender 离线渲染包安装
然后就可以跑demo.py了，改为–plotting-module pyrender
需要改demo.py的代码，加上renderer.py，在examples下面：【直接复制过去即可】结果在最下面
demo.py

# -*- coding: utf-8 -*-
import os
# Max-Planck-Gesellschaft zur Förderung der Wissenschaften e.V. (MPG) is
# holder of all proprietary rights on this computer program.
# You can only use this computer program if you have closed
# a license agreement with MPG or you get the right to use the computer
# program from someone who is authorized to grant you that right.
# Any use of the computer program without a valid license is prohibited and
# liable to prosecution.
#
# Copyright©2019 Max-Planck-Gesellschaft zur Förderung
# der Wissenschaften e.V. (MPG). acting on behalf of its Max Planck Institute
# for Intelligent Systems. All rights reserved.
#
# Contact: ps-license@tuebingen.mpg.de

import os.path as osp
import argparse

import numpy as np
import torch

import smplx
from renderer import Renderer
import cv2

def main(model_folder,
         model_type='smplx',
         ext='npz',
         gender='neutral',
         plot_joints=False,
         num_betas=10,
         sample_shape=True,
         sample_expression=True,
         num_expression_coeffs=10,
         plotting_module='pyrender',
         use_face_contour=False):

    model = smplx.create(model_folder, model_type=model_type,
                         gender=gender, use_face_contour=use_face_contour,
                         num_betas=num_betas,
                         num_expression_coeffs=num_expression_coeffs,
                         ext=ext)
    print(model)

    betas, expression = None, None
    if sample_shape:
        betas = torch.randn([1, model.num_betas], dtype=torch.float32)
    if sample_expression:
        expression = torch.randn(
            [1, model.num_expression_coeffs], dtype=torch.float32)

    pose = torch.zeros([1,21*3], dtype=torch.float32)
    pose[:,2] = 1
    pose[:,5] = -1
    output = model(betas=betas, expression=expression, body_pose=pose,
                   return_verts=True)
    # output = model(betas=betas, expression=expression,
    #                return_verts=True)
    vertices = output.vertices.detach().cpu().numpy().squeeze()
    joints = output.joints.detach().cpu().numpy().squeeze()

    print('Vertices shape =', vertices.shape)
    print('Joints shape =', joints.shape)

    if plotting_module == 'pyrender':
        import pyrender
        import trimesh

        renderer = Renderer(resolution=(512, 512), orig_img=True, faces=model.faces)
        img = renderer.render(
            vertices,
        )
        cv2.imwrite('demo.jpg',img)
    elif plotting_module == 'matplotlib':
        from matplotlib import pyplot as plt
        from mpl_toolkits.mplot3d import Axes3D
        from mpl_toolkits.mplot3d.art3d import Poly3DCollection

        fig = plt.figure()
        ax = fig.add_subplot(111, projection='3d')

        mesh = Poly3DCollection(vertices[model.faces], alpha=0.1)
        face_color = (1.0, 1.0, 0.9)
        edge_color = (0, 0, 0)
        mesh.set_edgecolor(edge_color)
        mesh.set_facecolor(face_color)
        ax.add_collection3d(mesh)
        ax.scatter(joints[:, 0], joints[:, 1], joints[:, 2], color='r')
        ax.view_init(azim=-90, elev=100)
        if plot_joints:
            ax.scatter(joints[:, 0], joints[:, 1], joints[:, 2], alpha=0.1)
        plt.show()
    elif plotting_module == 'open3d':
        import open3d as o3d

        mesh = o3d.geometry.TriangleMesh()
        mesh.vertices = o3d.utility.Vector3dVector(
            vertices)
        mesh.triangles = o3d.utility.Vector3iVector(model.faces)
        mesh.compute_vertex_normals()
        mesh.paint_uniform_color([0.3, 0.3, 0.3])

        geometry = [mesh]
        if plot_joints:
            joints_pcl = o3d.geometry.PointCloud()
            joints_pcl.points = o3d.utility.Vector3dVector(joints)
            joints_pcl.paint_uniform_color([0.7, 0.3, 0.3])
            geometry.append(joints_pcl)

        o3d.visualization.draw_geometries(geometry)
    else:
        raise ValueError('Unknown plotting_module: {}'.format(plotting_module))


if __name__ == '__main__':
    parser = argparse.ArgumentParser(description='SMPL-X Demo')

    parser.add_argument('--model-folder', required=True, type=str,
                        help='The path to the model folder')
    parser.add_argument('--model-type', default='smplx', type=str,
                        choices=['smpl', 'smplh', 'smplx', 'mano', 'flame'],
                        help='The type of model to load')
    parser.add_argument('--gender', type=str, default='neutral',
                        help='The gender of the model')
    parser.add_argument('--num-betas', default=10, type=int,
                        dest='num_betas',
                        help='Number of shape coefficients.')
    parser.add_argument('--num-expression-coeffs', default=10, type=int,
                        dest='num_expression_coeffs',
                        help='Number of expression coefficients.')
    parser.add_argument('--plotting-module', type=str, default='pyrender',
                        dest='plotting_module',
                        choices=['pyrender', 'matplotlib', 'open3d'],
                        help='The module to use for plotting the result')
    parser.add_argument('--ext', type=str, default='npz',
                        help='Which extension to use for loading')
    parser.add_argument('--plot-joints', default=False,
                        type=lambda arg: arg.lower() in ['true', '1'],
                        help='The path to the model folder')
    parser.add_argument('--sample-shape', default=True,
                        dest='sample_shape',
                        type=lambda arg: arg.lower() in ['true', '1'],
                        help='Sample a random shape')
    parser.add_argument('--sample-expression', default=True,
                        dest='sample_expression',
                        type=lambda arg: arg.lower() in ['true', '1'],
                        help='Sample a random expression')
    parser.add_argument('--use-face-contour', default=False,
                        type=lambda arg: arg.lower() in ['true', '1'],
                        help='Compute the contour of the face')

    args = parser.parse_args()

    model_folder = osp.expanduser(osp.expandvars(args.model_folder))
    model_type = args.model_type
    plot_joints = args.plot_joints
    use_face_contour = args.use_face_contour
    gender = args.gender
    ext = args.ext
    plotting_module = args.plotting_module
    num_betas = args.num_betas
    num_expression_coeffs = args.num_expression_coeffs
    sample_shape = args.sample_shape
    sample_expression = args.sample_expression

    main(model_folder, model_type, ext=ext,
         gender=gender, plot_joints=plot_joints,
         num_betas=num_betas,
         num_expression_coeffs=num_expression_coeffs,
         sample_shape=sample_shape,
         sample_expression=sample_expression,
         plotting_module=plotting_module,
         use_face_contour=use_face_contour)

renderer.py

import os
#os.environ['PYOPENGL_PLATFORM'] = 'egl'
import math
import trimesh
import pyrender
import numpy as np
from pyrender.constants import RenderFlags


class WeakPerspectiveCamera(pyrender.Camera):
    def __init__(self,
                 scale,
                 translation,
                 znear=pyrender.camera.DEFAULT_Z_NEAR,
                 zfar=None,
                 name=None):
        super(WeakPerspectiveCamera, self).__init__(
            znear=znear,
            zfar=zfar,
            name=name,
        )
        self.scale = scale
        self.translation = translation

    def get_projection_matrix(self, width=None, height=None):
        P = np.eye(4)
        P[0, 0] = self.scale[0]
        P[1, 1] = self.scale[1]
        P[0, 3] = self.translation[0] * self.scale[0]
        P[1, 3] = -self.translation[1] * self.scale[1]
        P[2, 2] = -1
        return P


class PerspectiveCamera(pyrender.Camera):
    def __init__(self,
                 yfov=None,
                 aspectRatio=1):
        super(PerspectiveCamera, self).__init__(
            znear=0.05,
            zfar=100.0
        )
        self.znear = 0.05
        self.zfar = 100.0
        self.yfov = yfov
        self.aspectRatio = aspectRatio

    def get_projection_matrix(self, width=None, height=None):
        znear = 0.05
        zfar = 100.0
        fovX = self.yfov
        tanHalfFovY = math.tan((self.yfov / 2))
        tanHalfFovX = math.tan((fovX / 2))

        top = tanHalfFovY * znear
        bottom = -top
        right = tanHalfFovX * znear
        left = -right

        P = np.zeros((4, 4))

        z_sign = 1.0

        P[0, 0] = 2.0 * znear / (right - left)
        P[1, 1] = 2.0 * znear / (top - bottom)
        P[0, 2] = (right + left) / (right - left)
        P[1, 2] = (top + bottom) / (top - bottom)
        P[3, 2] = -z_sign
        P[2, 2] = z_sign * zfar / (znear-zfar)
        P[2, 3] = 2*(zfar * znear) / (znear-zfar)
        return P





class Renderer:
    def __init__(self, resolution=(224,224), orig_img=False, wireframe=False, faces=None):
        self.resolution = resolution

        self.faces = faces
        self.orig_img = orig_img
        self.wireframe = wireframe
        self.renderer = pyrender.OffscreenRenderer(
            viewport_width=self.resolution[0],
            viewport_height=self.resolution[1],
            point_size=1.0
        )

        # set the scene
        self.scene = pyrender.Scene(bg_color=[0.0, 0.0, 0.0, 0.0], ambient_light=(0.3, 0.3, 0.3))

        # light = pyrender.PointLight(color=[1.0, 1.0, 1.0], intensity=0.8)
        light = pyrender.DirectionalLight(color=[1.0, 1.0, 1.0], intensity=0.8)

        light_pose = np.eye(4)
        light_pose[:3, 3] = [0, -1, 1]
        self.scene.add(light, pose=light_pose)

        light_pose[:3, 3] = [0, 1, 1]
        self.scene.add(light, pose=light_pose)

        light_pose[:3, 3] = [1, 1, 2]
        self.scene.add(light, pose=light_pose)

    def render(self, verts, cam=None, angle=None, axis=None, mesh_filename=None, color=[1.0, 1.0, 0.9], rotate=False):
        mesh = trimesh.Trimesh(vertices=verts, faces=self.faces, process=False)

        Rx = trimesh.transformations.rotation_matrix(math.radians(180), [1, 0, 0])
        mesh.apply_transform(Rx)

        if rotate:
            rot = trimesh.transformations.rotation_matrix(
                np.radians(60), [0, 1, 0])
            mesh.apply_transform(rot)

        if mesh_filename is not None:
            mesh.export(mesh_filename)
        angle = 180
        axis = [1, 0, 0]
        if angle and axis:
            R = trimesh.transformations.rotation_matrix(math.radians(angle), axis)
            mesh.apply_transform(R)



        camera = PerspectiveCamera(yfov=0.5, aspectRatio=1)

        material = pyrender.MetallicRoughnessMaterial(
            metallicFactor=0.0,
            alphaMode='OPAQUE',
            smooth=True,
            wireframe=True,
            roughnessFactor=1.0,
            emissiveFactor=(0.1, 0.1, 0.1),
            baseColorFactor=(color[0], color[1], color[2], 1.0)
        )

        mesh = pyrender.Mesh.from_trimesh(mesh, material=material)

        mesh_node = self.scene.add(mesh, 'mesh')

        camera_pose = np.eye(4)
        camera_pose[:3, 3] = np.array([0, 0, 5])
        cam_node = self.scene.add(camera, pose=camera_pose)

        if self.wireframe:
            render_flags = RenderFlags.RGBA | RenderFlags.ALL_WIREFRAME
        else:
            render_flags = RenderFlags.RGBA

        rgb, _ = self.renderer.render(self.scene, flags=render_flags)
        # valid_mask = (rgb[:, :, -1] > 0)[:, :, np.newaxis]
        # #output_img = rgb[:, :, :-1] * valid_mask + (1 - valid_mask) * img
        # output_img = rgb * valid_mask + (1 - valid_mask) * img
        # image = output_img.astype(np.uint8)

        self.scene.remove_node(mesh_node)
        self.scene.remove_node(cam_node)

        return rgb

参考链接

1. https://www.bilibili.com/video/BV1jkVeeJEDn/?spm_id_from=333.788&vd_source=5413f4289a5882463411525768a1ee27
2. https://github.com/heawon-yoon/smpl_gaussian_tutorial/blob/main/smpl.ipynb