文章目录
- 基础介绍
- Open3D读取文件
- `Voxel`降采样
- 裁剪点云
- `Open3D`点云和`numpy`数组的转换
- 点云的`bounding box`
欢迎访问个人网络日志🌹🌹知行空间🌹🌹
基础介绍
Open3D
是英特尔公司于2015
年发布的开源3D
视觉算法库,截至2023年03月已经更新到了0.17.0
版本。基于MIT
协议开源许可。
其后端使用C++11
实现,经过了高度优化,使用OpenMP
并行运算优化。通过Python Pybinding
,其提供前端Python API
。
Open3D
的介绍论文发布在http://www.open3d.org/wordpress/wp-content/paper.pdf。更多详细的介绍可以参考这里。
值得一提的是在另外一个大名鼎鼎的3D
视觉算法库是Point Cloud Library(PCL)
,PCL
由Willow Garage
实验室开源于2011
年,与Robot Operating System(ROS)
同出一源。关于PCL
的介绍可以参考这里https://pointclouds.org/assets/pdf/pcl_icra2011.pdf。在Open3D
的介绍论文中,作者指出,PCL
作为较早出现的3D
视觉算法库,经过一段时间的开源维护后,代码变的臃肿,且更新维护频率比较低。
Open3D的更新历史https://github.com/isl-org/Open3D/tags
PCL的更新历史https://github.com/PointCloudLibrary/pcl/tags
作为Open3D
和PCL
的使用者,客观的讲Open3D
的Python
接口更好用,维护更新做的也更好。
Open3D读取文件
使用的测试数据可以从这里下载。
数据下载地址http://graphics.stanford.edu/data/3Dscanrep/
物体的3D
表示可以使用点云/Mesh/Model。
Open3D
支持的Mesh
类型有:
Open3D
支持的点云类型有:
- 读取
ply
格式的Mesh
filename = "dragon_recon/dragon_vrip_res2.ply"
dragon_mesh = o3d.io.read_triangle_mesh(filename)
print(dragon_mesh)
dragon_mesh.compute_vertex_normals()
print(np.asarray(dragon_mesh.triangles).shape)
print(np.asarray(dragon_mesh.vertices).shape)
o3d.visualization.draw_geometries([dragon_mesh]) # 可视化Mesh
# TriangleMesh with 100250 points and 202520 triangles.
# (202520, 3)
# (100250, 3)
- 读取
pcd
格式的点云 - 保存点云:
write_point_cloud(filename, pointcloud, write_ascii=False, compressed=False, print_progress=False)
Function to write PointCloud to file - 可视化点云:
-
draw_geometries(geometry_list, window_name='Open3D', width=1920, height=1080, left=50, top=50, point_show_normal=False, mesh_show_wireframe=False, mesh_show_back_face=False)
Function to draw a list of geometry.Geometry objects。
使用OpenGL
进行渲染。
- 2.
draw_geometries_with_key_callbacks
参考自callback
-
dragon_pc = dragon_mesh.sample_points_uniformly(number_of_points=20000)
save_file = "dragon_recon/dragon_vrip_res2.pcd"
o3d.io.write_point_cloud(save_file, dragon_pc)
o3d.visualization.draw_geometries([dragon_pc]) # 可视化点云 +/-可调点云的大小
def rotate_callback(vis):
ctr = vis.get_view_control()
ctr.rotate(10.0, 0.0)
return False
key_to_callback = dict()
key_to_callback[ord("r")] = rotate_callback
o3d.visualization.draw_geometries_with_key_callbacks([dragon_pc], key_to_callback)
Voxel
降采样
刚刚随机采样生成点云时,number_of_points设置的是20000
,点太多了的时候,可以使用体素Voxel
来降采样。
voxel_down_sample(self, voxel_size)
Function to downsample input pointcloud into output pointcloud with a voxel. Normals and colors are averaged if they exist.
-
o3d.visualization.read_selection_polygon_volume 通过空间多边形和最大最小距离来裁剪点云数据
-
compute_convex_hull(…) method of open3d.cpu.pybind.geometry.PointCloud instance,计算点云的凸包
-
estimate_normals 计算点云的法向量
max_bound = dragon_pc.get_max_bound()
min_bound = dragon_pc.get_min_bound()
dx, dy, dz = max_bound - min_bound
voxel_size = min([dx, dy, dz]) / 20
print(dragon_pc)
dragon_pc_downsampled = dragon_pc.voxel_down_sample(voxel_size=voxel_size)
print(dragon_pc_downsampled.points)
dragon_pc_downsampled.estimate_normals(o3d.geometry.KDTreeSearchParamHybrid(radius=0.1, max_nn=32))
o3d.visualization.draw_geometries([dragon_pc_downsampled], point_show_normal=True)
dragon_normals = dragon_pc_downsampled.normals
print(dragon_normals[0])
# PointCloud with 20000 points.
# std::vector<Eigen::Vector3d> with 4179 elements.
# Use numpy.asarray() to access data.
# [-0.39450818 0.35314528 0.84832288]
裁剪点云
demo_crop_data = o3d.data.DemoCropPointCloud()
pcd = o3d.io.read_point_cloud(demo_crop_data.point_cloud_path)
vol = o3d.visualization.read_selection_polygon_volume(demo_crop_data.cropped_json_path)
chair = vol.crop_point_cloud(pcd)
o3d.visualization.draw_geometries([chair],
zoom=0.7,
front=[0.5439, -0.2333, -0.8060],
lookat=[2.4615, 2.1331, 1.338],
up=[-0.1781, -0.9708, 0.1608])
解析打印出来裁剪使用json
文件中的内容,如下
import json
import pprint
pp = pprint.PrettyPrinter(2)
with open(demo_crop_data.cropped_json_path) as f:
d = json.load(f)
pp.pprint(d)
bounding_polygon = np.array(d["bounding_polygon"])
bounding_polygon_data = o3d.geometry.PointCloud()
bounding_polygon_data.points = o3d.utility.Vector3dVector(bounding_polygon)
bounding_polygon_data.paint_uniform_color([1, 0.6, 0])
chair_dat = chair.random_down_sample(0.6)
hull, _ = chair_dat.compute_convex_hull()
hull_ls = o3d.geometry.LineSet.create_from_triangle_mesh(hull)
hull_ls.paint_uniform_color((1, 0, 0))
o3d.visualization.draw_geometries([bounding_polygon_data, hull_ls, chair_dat, o3d.geometry.TriangleMesh.create_coordinate_frame()])
json
文件中的内容为:
{ 'axis_max': 4.022921085357666,
'axis_min': -0.763413667678833,
'bounding_polygon': [ [2. 6509309513852526, 0.0, 1.6834473132326844],
[2.578642824691715, 0.0, 1.6892074266735244],
[2.4625790337552154, 0.0, 1.6665777078297999],
[2.2228544982251655, 0.0, 1.6168160446813649],
[2.166993206001413, 0.0, 1.6115495157201662],
[2.1167895865303286, 0.0, 1.6257706054969348],
[2.0634657721747383, 0.0, 1.623021658624539],
[2.0568612343437236, 0.0, 1.5853892911207643],
[2.1605399001237027, 0.0, 0.9622899325508302],
[2.1956669387205228, 0.0, 0.9557274604978507],
[2.2191318790575583, 0.0, 0.8873444998210875],
[2.248488184792592, 0.0, 0.8704280726701363],
[2.6891234157295827, 0.0, 0.941406779889676],
[2.7328692490470647, 0.0, 0.9877574067484025],
[2.7129337547575547, 0.0, 1.0398850034649203],
[2.7592174072415405, 0.0, 1.0692940558509485],
[2.768921641945343, 0.0, 1.0953914441371593],
[2.685145562545567, 0.0, 1.6307334122162018],
[2.671477609998124, 0.0, 1.675524657088997],
[2.6579576128816544, 0.0, 1.6819127849749496]],
'class_name': 'SelectionPolygonVolume',
'orthogonal_axis': 'Y',
'version_major': 1,
'version_minor': 0}
bounding_polygon
字段定义了空间中的多边形,axis_max/axis_min
定义了与多边形垂直的轴上的最大最小距离。
裁剪mesh的方法:
filename = "dragon_recon/dragon_vrip_res2.ply"
dragon_mesh = o3d.io.read_triangle_mesh(filename)
dragon_mesh.triangles = o3d.utility.Vector3iVector(
np.asarray(dragon_mesh.triangles)[:len(dragon_mesh.triangles) // 2, :])
dragon_mesh.triangle_normals = o3d.utility.Vector3dVector(
np.asarray(dragon_mesh.triangle_normals)[:len(dragon_mesh.triangle_normals) // 2, :])
dragon_mesh.paint_uniform_color([1, 0.7,0])
print(dragon_mesh.triangles)
o3d.visualization.draw_geometries([dragon_mesh])
裁剪的结果,如下:
Open3D
点云和numpy
数组的转换
Open3D
点云的后端表示是std::vector
和Eigen::Vector
,因此,
- 将
numpy.array
转换成Open3D
数组时,需要使用o3d.utility.Vector3dVector
- 将
PointCloud.points/colors
转换成numpy.array
时需要使用numpy.asarray
printr_points = np.array([[0, 0, 0], [0, 1, 1], [1, 1, 0], [0, 0, 1]])
four_pc = o3d.geometry.PointCloud()
four_pc.points = o3d.utility.Vector3dVector(four_points)
four_pc.paint_uniform_color([1, 0.2, 0.4])
aabb = four_pc.get_axis_aligned_bounding_box()
aabb.color = (1, 0, 0)
obb = four_pc.get_oriented_bounding_box()
obb.color = (0, 1, 0)
o3d.visualization.draw_geometries([four_pc, aabb, obb, o3d.geometry.TriangleMesh.create_coordinate_frame()]) # 可视化点云
点云的bounding box
open3d.geometry.PointCloud
类中有get_axis_aligned_bounding_box
和get_oriented_bounding_box
两个方法可以获取当前点云的最小包围框。
其区别通过一个例子来介绍,
有点[0, 0, 0], [0, 1, 1], [1, 1, 0], [0, 0, 1]
- get_axis_aligned_bounding_box 如上图,红色部分包围框就是此函数求得的结果,包围矩形的相互平行的四条边都与坐标轴平行,类似
2D
图像中的水平包围框 - get_oriented_bounding_box 如上图,绿色部分的包围框就是此函数求得的结果,给出的是空间中的最小包围矩形,类似
2D
图像中的旋转包围框
printr_points = np.array([[0, 0, 0], [0, 1, 1], [1, 1, 0], [0, 0, 1]])
four_pc = o3d.geometry.PointCloud()
four_pc.points = o3d.utility.Vector3dVector(four_points)
four_pc.paint_uniform_color([1, 0.2, 0.4])
aabb = four_pc.get_axis_aligned_bounding_box()
aabb.color = (1, 0, 0)
obb = four_pc.get_oriented_bounding_box()
obb.color = (0, 1, 0)
o3d.visualization.draw_geometries([four_pc, aabb, obb, o3d.geometry.TriangleMesh.create_coordinate_frame()]) # 可视化点云
完整的例子见仓库https://gitee.com/lx_r/object_detection_task/tree/main/detection3d/open3d
欢迎访问个人网络日志🌹🌹知行空间🌹🌹
- 1.https://github.com/isl-org/Open3D/blob/master/examples/python/visualization/draw.py