What is open3D

Open3D is an open-source library that supports rapid development of software that deals with 3D data. The Open3D frontend exposes a set of carefully selected data structures and algorithms in both C++ and Python. The backend is highly optimized and is set up for parallelization. We welcome contributions from the open-source community.

open3D 核心功能包括：

• 3D data structures 3维数据结构
• 3D data processing algorithms 3维数据处理算法
• Scene reconstruction 场景重建
• Surface alignment 表面对齐
• 3D visualization 三维可视化
• Physically based rendering (PBR) 物理渲染
• 3D machine learning support with PyTorch and TensorFlow 机器学习
• GPU acceleration for core 3D operations GPU加速
• Available in C++ and Python

python quick start

# Install
pip install open3d

# Verify installation
python -c "import open3d as o3d; print(o3d.__version__)"

0.15.2

# Python API
python -c "import open3d as o3d; \
           mesh = o3d.geometry.TriangleMesh.create_sphere(); \
           mesh.compute_vertex_normals(); \
           o3d.visualization.draw(mesh, raw_mode=True)"

# Open3D CLI
open3d example visualization/draw

***************************************************
* Open3D: A Modern Library for 3D Data Processing *
*                                                 *
* Version 0.15.2                                  *
* Docs    http://www.open3d.org/docs              *
* Code    https://github.com/isl-org/Open3D       *
***************************************************

交互指令

-- Mouse view control --
  Left button + drag         : Rotate.
  Ctrl + left button + drag  : Translate.
  Wheel button + drag        : Translate.
  Shift + left button + drag : Roll.
  Wheel                      : Zoom in/out.

-- Keyboard view control --
  [/]          : Increase/decrease field of view.
  R            : Reset view point.
  Ctrl/Cmd + C : Copy current view status into the clipboard.
  Ctrl/Cmd + V : Paste view status from clipboard.

-- General control --
  Q, Esc       : Exit window.
  H            : Print help message.
  P, PrtScn    : Take a screen capture.
  D            : Take a depth capture.
  O            : Take a capture of current rendering settings.

显示点云

open3d.io.read_point_cloud(filename, format='auto', remove_nan_points=False, \
remove_infinite_points=False, print_progress=False)

Example:
    
import open3d as o3d
pcd=o3d.io.read_point_cloud(r"Cloud.pcd")
print(pcd)

read_point_cloud Parameters

filename (str) – 文件路径
format (str,optional,default=‘auto’) – 文件的格式，默认是auto，将影响如何读取文件
remove_nan_points (bool*,* optional*,* default=False) – 是否移除值为nan的点
remove_infinite_points (bool*,* optional*,* default=False) – 是否移除值为inf的点
print_progress (bool*,* optional*,* default=False) – 当该值为True时，将会在可视化时出现一个过程条

Return

open3d.geometry.PointCloud对象

format参数的可选参数为：

格式	描述
xyz	每一行包含[x,y,z]
xyzn	每一行包含[x,y,z,nx,ny,nz]
xyzrgb	每一行包括[x,y,z,r,g,b] rgb为[0,1]之间的float类型
pts	第一行表示点数，之后每行包括[x,y,z,i,r,g,b] rgb为unit8类型
ply	ply文件
pcd	pcd文件

读取文本点云文件，通过numpy转换

import numpy as np
import open3d as o3d

# 读取到ndarray
data=np.genfromtxt(r'randy\three.txt',delimiter=",")
# 创建PointCloud类
pcd=o3d.geometry.PointCloud()
pcd.points=o3d.utility.Vector3dVector(data[:,:3])
print(pcd)

PointCloud的属性：

• colors: 颜色信息，在可视化时能为几何体赋予视觉信息
• covariances: 协方差
• normal: 法向量
• points: 位置信息

加载ply点云：

    print("Load a ply point cloud, print it, and render it")  # 加载PLY点云，打印它并显示它
    ply_point_cloud = o3d.data.PLYPointCloud()  # 获取点云数据，如果下载不成功，可以删除这行代码
    pcd = o3d.io.read_point_cloud(ply_point_cloud.path)  # ply_point_cloud.path，可以换成自己的点云文件的地址
    print(pcd)
    print(np.asarray(pcd.points))
    o3d.visualization.draw_geometries([pcd],
                                      zoom=0.3412,
                                      front=[0.4257, -0.2125, -0.8795],
                                      lookat=[2.6172, 2.0475, 1.532],
                                      up=[-0.0694, -0.9768, 0.2024])

显示单帧点云：

def draw_point_cloud(pcd):
    vis = o3d.visualization.Visualizer()
    vis.create_window()
    vis.add_geometry(pcd)
    render_option = vis.get_render_option()
    render_option.point_size = 1
    render_option.background_color = np.asarray([0, 0, 0])
    vis.run()
    vis.destroy_window()

批量单帧显示

import open3d as o3d
import os

pcds = os.listdir("/home/user/Documents/reconstruction/0715_file/0806_reconstruction/")
# pcds = [i for i in pcds if i.split(".")[-1] == "pcd"]
pcds.sort()
for pcd in pcds:
    print(pcd)
    source = o3d.io.read_point_cloud("/home/qiancj/pcd" + pcd)  # source 为需要配准的点云

    vis = o3d.visualization.Visualizer()
    vis.create_window()
    # 将两个点云放入visualizer
    vis.add_geometry(source)
    # vis.add_geometry(target)
    vis.get_render_option().point_size = 2
    # 让visualizer渲染点云
    vis.update_geometry()
    vis.poll_events()
    vis.update_renderer()

    vis.run()

点云可视化

draw_geometries(geometry_list, window_name=’Randy’, width=1920,\
 height=1080, left=50, top=50, point_show_normal=False,\
  mesh_show_wireframe=False, mesh_show_back_face=False,\
   lookat, up, front, zoom)

Retun:
    None
    
call function：
o3d.visualization.draw_geometries([pcd])

draw_geometries Parameters含义

parameter	含义
geometry_list (List[open3d.geometry.Geometry])	需要可视化的几何体列表
window_name (str, optional, default=‘Open3D’)	窗口名称
width (int, optional, default=1920)	窗口宽度
height (int, optional, default=1080)	窗口高度
left (int, optional, default=50)	窗口左边界
top (int, optional, default=50)	窗口顶部边界
point_show_normal (bool, optional, default=False)	是否展示法向量
mesh_show_wireframe (bool, optional, default=False)	是否可视化网格线框
mesh_show_back_face (bool, optional, default=False)	同时可视化格网三角形背部
lookat (numpy.ndarray[float64[3,1]])	相机注视向量
up (numpy.ndarray[float64[3,1]])	相机的上方向向量
front (numpy.ndarray[float64[3,1]])	相机的前矢量
zoom (float)	相机缩放倍数

Demos:

def custom_draw_geometry_with_key_callback(pcd):
    def change_background_to_black(vis):
        opt = vis.get_render_option()
        opt.background_color = np.asarray([0, 0, 0])
        return False
    def load_render_option(vis):
        vis.get_render_option().load_from_json(
                "../../TestData/renderoption.json")
        return False
    def capture_depth(vis):
        depth = vis.capture_depth_float_buffer()
        plt.imshow(np.asarray(depth))
        plt.show()
        return False
    def capture_image(vis):
        image = vis.capture_screen_float_buffer()
        plt.imshow(np.asarray(image))
        plt.show()
        return False
    key_to_callback = {}
    key_to_callback[ord("K")] = change_background_to_black
    key_to_callback[ord("R")] = load_render_option
    key_to_callback[ord(",")] = capture_depth
    key_to_callback[ord(".")] = capture_image
    draw_geometries_with_key_callbacks([pcd], key_to_callback)

custom_draw_geometry_with_key_callback(point_cloud)

显示法向量

pcd.normals=o3d.utility.Vector3dVector(data[:,3:])
o3d.visualization.draw_geometries([pcd],window_name="o3d",width=1920,height=1080,
                                  left=50,top=50,point_show_normal=True)

# 自动计算法向量
radius=0.01 # 搜索半径
max_nn=25 # 邻域内用于估算法线的最大点数
# 执行KD树搜索
pcd_bin.estimate_normals(search_param=o3d.geometry.KDTreeSearchParamHybrid(radius,max_nn))
o3d.visualization.draw_geometries([pcd_bin],window_name="o3d",width=1920,height=1080,
                                  left=50,top=50,point_show_normal=True)

点云增加颜色，可自己配置

colors = np.ndarray(shape=(npts, 3), dtype=int)
count = 0
for z in data_new[:, 2]:
    colors[count] = convert_data8(z)
    count = count + 1

pcd_bin.colors = o3d.utility.Vector3dVector(colors/255.0)
o3d.visualization.draw_geometries([pcd_bin], window_name="o3d", width=1920, height=1080,
left=50, top=50, point_show_normal=True)

多个geometry同时显示

path=r'\randy'
pcd1=read_txt(path+r"\three_1.txt")
pcd2=read_txt(path+r"\three_2.txt")
o3d.visualization.draw_geometries([pcd1,pcd2],window_name="Sesame",width=1920,height=1080,
            left=50,top=50,mesh_show_back_face=True)

同时显示多个点云

def draw_point_clouds_simultaneously(base_fold, is_down_sample = False, arg_voxel_size = 0.2):
    if base_fold == "":
        print("\033[1;31mBase fold is EMPTY. Please check!\033[0m")
    pcds = []
    pcds_path = []
    pcd_total = o3d.geometry.PointCloud()
    files = os.listdir(base_fold)
    for f in files:
        pcd_path = os.path.join(base_fold + f)
        pcds_path.append(pcd_path)
        pcd = o3d.io.read_point_cloud(pcd_path)
        # 降采样
        if is_down_sample:
            pcd_down = pcd.voxel_down_sample(voxel_size=arg_voxel_size)
            pcds.append(pcd_down)
        else:
            pcds.append(pcd)

    o3d.visualization.draw_geometries(pcds, window_name="Simultaneous display point clouds",
                                      width=1024, height=768,
                                      left=50, top=50,
                                      mesh_show_back_face=False)

保存点云

open3d.io.write_point_cloud(filename, pointcloud, write_ascii=False, compressed=False, print_progress=False)

Example: 
o3d.io.write_point_cloud("02.pcd",pcd2,write_ascii=True)

Return 
   bool

Parameters	含义
filename (str)	文件路径
pointcloud (open3d.geometry.PointCloud)	点云对象
write_ascii (bool，optional,default=False)	该参数为True时，将会写入ASCII码，否则一般写入二进制文件
compressed (bool,optional,default=False)	是否以压缩格式进行输出
print_progress (bool,optional,default=False)	是否在控制台打印一个进度条

合并点云

def merge_point_clouds(base_fold, save_path):
    if base_fold == "":
        print("\033[1;31mBase fold is EMPTY. Please check!\033[0m")
    if base_fold == "":
        print("\033[1;33mBase fold is EMPTY. Set base fold as target fold!\033[0m")
        now_time = datetime.datetime.now()
        cur_date = datetime.datetime.strptime(now_time, '%Y%m%d')
        tar_pcd = "merge_pcd_{}.pcd".format(cur_date)
        save_path = os.path.join(base_fold, tar_pcd)
    pcds_path = []
    files = os.listdir(base_fold)
    for f in files:
        pcd_path = os.path.join(base_fold + f)
        pcds_path.append(pcd_path)
    for i in range(len(pcds_path)):
        if i == 0:
            pcd0 = o3d.io.read_point_cloud(pcds_path[0])
            o3d.io.write_point_cloud(save_path, pcd0)
        else:
            pcd1 = o3d.io.read_point_cloud(save_path)
            pcd2 = pcd1 + o3d.io.read_point_cloud(pcds_path[i])
            o3d.io.write_point_cloud(save_path, pcd2)
    print("save merged point cloud file: ", save_path)

点云操作

体素下采样

down_sample_pcd = pcd_randy.voxel_down_sample(voxel_size=0.05)
o3d.visualization.draw_geometries([down_sample_pcd])
print("The number of point cloud is : ",pcd_randy)
print("The number of down_sample_PC is : ",down_sample_pcd)

点云正态估计

点云正态估计通过指定算法参数估测每个点可能的法向量，estimate_normals查找指定搜索半径内的临近点，通过这些临近点的协方差计算其主轴，从而估计法向量。正常情况下会产生两个方向相反的法向量，在不知道几何体的全局结构下，两者都可以是正确的。Open3D会尝试调整法线的方向，使其与原始法线对齐。

down_sample_pcd.estimate_normals(
    search_param=o3d.geometry.KDTreeSearchParamHybrid(radius=0.1, max_nn=30))
o3d.visualization.draw_geometries([down_sample_pcd],
                                  zoom=0.3488,
               front=[0.4288, -0.2188, -0.8788],
               lookat=[2.6188, 2.0488, 1.588],
               up=[-0.0688, -0.9788, 0.2088],
               point_show_normal=True)

# 访问顶点法线
print("Print a normal vector of the 8th point")
print(down_sample_pcd.normals[8])

print("Print the normal vectors of the first 8 points")
print(np.asarray(down_sample_pcd.normals)[:8, :])

点云裁剪

Open3D的点云裁剪需要通过read_selection_polygon_volume读取多边形选择区域的json文件，接着通过.crop_point_cloud()方法过滤出点。

print("Load a polygon volume and use it to crop the original point cloud")
demo_crop_data = o3d.data.DemoCropPointCloud()
pcd = o3d.io.read_point_cloud(demo_crop_data.point_cloud_path)
vol = o3d.visualization.read_selection_polygon_volume(demo_crop_data.cropped_json_path)
chair = vol.crop_point_cloud(pcd)
o3d.visualization.draw_geometries([chair],
                                  zoom=0.7,
               front=[0.4288, -0.2188, -0.8330],
               lookat=[2.6188, 2.0488, 1.588],
               up=[-0.0699, -0.9799, 0.2099])

绘制点云

paint_uniform_color可以将点云颜色绘制成同一的色彩。注意颜色是在[0,1]之间的float类型。

print("Paint desk")
chair.paint_uniform_color([0.8, 0.806, 0])
o3d.visualization.draw_geometries([desk],
                                  zoom=0.8,
          front=[0.5434, -0.2388, -0.8888],
          lookat=[2.4688, 2.1388, 1.388],
          up=[-0.1788, -0.9788, 0.1688])

### 

选择点云

Open3D中，可以通过点云索引来进行筛选。select_by_index也可以通过修改invert方法进行反向选取。

inner=pcd_randy.select_by_index([i for i in range(len(pcd_randy.points)) if i%2==0])
outer=pcd_randy.select_by_index([i for i in range(10)],invert=True)
o3d.visualization.draw_geometries([pcd_randy])
o3d.visualization.draw_geometries([inner])
o3d.visualization.draw_geometries([outer])

点云距离

Open3D提供了compute_point_cloud_distance方法，能够计算源点云到目标点云的最近距离，该方法也能用于计算两点云之间的切角距离。

demo_crop_data = o3d.data.DemoCropPointCloud()
pcd = o3d.io.read_point_cloud(demo_crop_data.point_cloud_path)
vol = o3d.visualization.read_selection_polygon_volume(demo_crop_data.cropped_json_path)
chair = vol.crop_point_cloud(pcd)

#从原始图像到裁剪图像中最近点的距离
dists=pcd.compute_point_cloud_distance(chair)
dists=np.asarray(dists)
index=np.where(dists>0.1)[0]
pcd_without_chair = pcd.select_by_index(index)
o3d.visualization.draw_geometries([pcd_without_chair],
                                  zoom=0.3412,
          front=[0.5434, -0.2388, -0.8888],
          lookat=[2.4688, 2.1388, 1.388],
          up=[-0.1788, -0.9788, 0.1688])

边界体积

与其几何类型相似，PointCloud也具有边界体积。

aabb = chair.get_axis_aligned_bounding_box()
aabb.color = (1, 0, 0)
obb = chair.get_oriented_bounding_box()
obb.color = (0, 1, 0)
o3d.visualization.draw_geometries([chair, aabb, obb],
                                  zoom=0.7,
          front=[0.5434, -0.2388, -0.8888],
          lookat=[2.4688, 2.1388, 1.388],
          up=[-0.1788, -0.9788, 0.1688])

凸包计算

点云凸包是包含所有点的最小凸集，在Open3D中，可采用compute_convex_hull计算。

bunny = o3d.data.BunnyMesh()
mesh = o3d.io.read_triangle_mesh(bunny.path)
mesh.compute_vertex_normals()

pcl = mesh.sample_points_poisson_disk(number_of_points=2000)
hull, _ = pcl.compute_convex_hull()
hull_ls = o3d.geometry.LineSet.create_from_triangle_mesh(hull)
hull_ls.paint_uniform_color((1, 0, 0))
o3d.visualization.draw_geometries([pcl, hull_ls])

DBSCAN聚类

DBSCAN是Ester在1996年提出的一种聚类算法，Open3D中也提供了该算法的APIpc.cluster_dbscan(eps,min_points,print_progress)，eps定义了簇的半径距离，而min_points定义形成簇的最小点数量。返回是一个标签对象，若值为-1则表示噪声。

import matplotlib.pyplot as plt
ply_point_cloud = o3d.data.PLYPointCloud()
pcd = o3d.io.read_point_cloud(ply_point_cloud.path)

with o3d.utility.VerbosityContextManager(
        o3d.utility.VerbosityLevel.Debug) as cm:
    labels = np.array(
        pcd.cluster_dbscan(eps=0.02, min_points=10, print_progress=True))

max_label = labels.max()
print(f"point cloud has {max_label + 1} clusters")
colors = plt.get_cmap("DaTou")(labels / (max_label if max_label > 0 else 1))
colors[labels < 0] = 0
pcd.colors = o3d.utility.Vector3dVector(colors[:, :3])
o3d.visualization.draw_geometries([pcd],
                                  zoom=0.453,
                 front=[-0.4999, -0.1633, -0.8499],
                 lookat=[2.1833, 2.0633, 2.0999],
                 up=[0.1233, -0.9833, 0.1

平面分割

Open3D支持使用RANSAC方法从点云中分割几何基元(geometric primitives)。通过segment_plane方法，可以找到点云中的最大支持平面(the plane with the largest support`)。该方法提供了三个参数：

• distance_threshold：定义了一个点可被视为内嵌点的估计平面的最大距离
• ransac_n：定义用来估计平面的随机抽样点数量
• num_iterations：定义了随机平面抽样和验证的频率

消隐点

当我们从给定视角渲染点云时，由于前方没有遮挡，可能会有背面的点渗入到前景中。Katz提出了一种消隐算法(Hidden point removal),可以从给定的视图中近似地获得点云的可见性，而无需表面重建或正常的估计。

print("Convert mesh to a point cloud and estimate dimensions")
armadillo = o3d.data.ArmadilloMesh()
mesh = o3d.io.read_triangle_mesh(armadillo.path)
mesh.compute_vertex_normals()

pcd = mesh.sample_points_poisson_disk(5000)
diameter = np.linalg.norm(
    np.asarray(pcd.get_max_bound()) - np.asarray(pcd.get_min_bound()))
o3d.visualization.draw_geometries([pcd])

print("Define parameters used for hidden_point_removal")
camera = [0, 0, diameter]
radius = diameter * 100

print("Get all points that are visible from given view point")
_, pt_map = pcd.hidden_point_removal(camera, radius)

print("Visualize result")
pcd = pcd.select_by_index(pt_map)
o3d.visualization.draw_geometries([pcd])

参考

• Open3D 官网
  http://www.open3d.org/docs/release/index.html

• Open3D点云处理
  https://blog.csdn.net/qq_45957458/article/details/124282087

• Python: 用open3D库，连续多帧显示点云（查看localization pose的好坏）
  https://blog.csdn.net/melally/article/details/126116894

• 使用open3d可视化
  https://blog.csdn.net/suyunzzz/article/details/105183824#t4)

• Open3D之键盘切换上下帧显示点云
  https://zhuanlan.zhihu.com/p/539697926

• 点云可视化工具open3d的使用
  https://blog.csdn.net/mingshili/article/details/124714290