This chapter mainly introduces how to adjust vision parameters according to actual scenarios for ordered loading/unloading, unordered picking, and positioning assembly of general workpieces.

Getting Started:

Background Introduction

Build the Project

(1) Create a new general workpiece ordered loading/unloading / general workpiece unordered picking / general workpiece positioning assembly project (the project name and project path can be customized, but the project name cannot contain Chinese characters)

Workpiece type: general workpiece (not circular, cylindrical, or quadrilateral, and with relatively large differences between the front and back sides)

(2) Camera and Robot configuration

(3) Add the workpiece

• Workpiece Information

The workpiece name can be customized. The workpiece type defaults to standard workpiece and cannot be changed. The workpiece ID can be customized and is used to automatically switch workpieces during robot picking.

Point cloud file: the workpiece point cloud, created in Point Cloud Template Creation

Fine matching point cloud template: used for fine matching

Custom keypoints: general workpieces require uploading the workpiece keypoint file

Camera parameters: not required

• Model Information

Vision model: the 2D recognition solution used for general workpiece applications is CAD-based synthetic data training (One-click Connection), and the 3D matching solution is Rigid Transform. The vision model for different general workpiece applications must be obtained through one-click connection training.

Mesh file: normally upload the workpiece CAD. To remove some noise, a standardized mesh file is required. The mesh file can also be standardized in Point Cloud Template Creation.

Workpiece attributes: elongated, symmetrical, highly reflective, and low-solidity

task environment: enter the environment file. In one-click connection, the environment used for data generation is automatically replaced with the entered environment to improve recognition results.

Workpiece texture: enter the workpiece texture. During model training in one-click connection, the entered workpiece texture is used for data augmentation to improve recognition results.

Maximum number of model recognitions: the default is 20; modify it according to scene requirements.

• Pick Point: configure the Pick Point according to the workpiece

Absolute coordinate system: uses the initial point as the origin. The initial point is built into the workpiece point cloud and CAD.

Pick Point coordinate system (offset): uses the current Pick Point as the origin.

(4) Add Tool, eye-hand calibration, and ROI

(5) Optional functional options: instance optimization, collision detection, collision detection (new), visual classification, recognition types, front/back recognition (via point cloud templates), and local feature recognition

Instance optimization: optimizes instances generated by the model and processes the instance masks.

Collision detection (new): this function is used to detect collisions between the Tool and the container, and to filter out picking poses that may collide. Collision Detection User Guide

Visual classification: used to identify features such as different textures and different orientations of the same workpiece. Visual Classification User Guide

Recognition types: used to identify multiple types of workpieces. Recognition Types User Guide

Local feature recognition: locates workpieces through local features of the workpiece. Local Feature Recognition User Guide

Front/back recognition (via point cloud templates): you can import point cloud templates for the front and back sides of the workpiece to match the front or back side of the picked workpiece, and configure Pick Points separately for the front and back sides. Front/Back Recognition (via Point Cloud Templates) User Guide

(6) Test data (historical data is provided for subsequent practice. You can use the 2D images and 3D point clouds in the foreground\input folder`{=tex} historical data instead of camera image acquisition to configure the ROI)

Vision Parameters

• 2D recognition: recognize and segment instances from the actual scene

Preprocessing: process the 2D image before instance segmentation (commonly used: fill holes in the depth map & edge enhancement & extract the top-layer texture & remove image background outside ROI 3D)

Instance segmentation: segment instances (scaling ratio & lower Confidence threshold & auto enhancement). To accelerate processing, you can clear the checkbox for Return Mask.

Point cloud generation: the method for generating instance point clouds — generate instance point clouds using segmented instance masks or bounding boxes / generate instance point clouds using filtered instance masks or bounding boxes

Instance filtering: filter the segmented instances

Instance sorting: sort instances

• 3D computation: calculate the pose of the instance in the Camera coordinate system and generate Pick Points

Preprocessing: preprocess the 3D point cloud before calculating Pick Points

Pose estimation: calculate the pose of the instance in the Camera coordinate system (coarse matching and fine matching) and generate Pick Points

• Pick Point processing: filter, adjust, and sort Pick Points

Pick Point filtering: filter Pick Points

Pick Point adjustment: adjust Pick Points

Pick Point sorting: sort Pick Points

---

1. 2D识别

本节主要对影响2D图像识别结果的预处理、实例分割、实例过滤、实例排序相关函数进行说明和调参建议

1.1 预处理

2D识别的预处理是在实例分割之前对2D图像进行处理

1.1.1 双边滤波

• 功能

基于双边滤波的图像平滑功能

• 参数说明

参数	说明	默认值	取值范围
最大深度差值	双边过滤的最大深度差值	0.03	[0.01, 1]
过滤核大小	双边过滤卷积核大小	7	[1, 3000]

1.1.2 深度转法向量图

• 功能

通过深度图计算像素法向量，并把图片转换成法向量图

1.1.3 图像增强

• 功能

常用图像增强，如色彩饱和度、对比度、亮度、锐利度

• 参数说明

参数	说明	默认值	取值范围
图像增强类型	对图像的某个元素进行增强	对比度	色彩饱和度、对比度、亮度、锐利度
图片增强阈值	对图像的某个元素增强多少	1.5	[0.1, 100]

1.1.4 直方图均衡

• 功能

提高图像的对比度

• 参数说明

参数	说明	默认值	取值范围
局部模式	局域或全局直方图均衡，勾选则局域直方图均衡，取消勾选则全局直方图均衡	勾选	/
对比度阈值	对比度阈值	3	[1,1000]

1.1.5 通过颜色过滤深度图

• 功能

根据颜色值过滤深度图

• 参数说明

参数	说明	默认值	取值范围
填充核大小	颜色填充的大小	3	[1,99]
根据hsv过滤深度-色域最大值	最大颜色值	[180,255,255]	[[0,0,0],[255,255,255]]
根据hsv过滤深度-色域最小值	最小颜色值	[0,0,0]	[[0,0,0],[255,255,255]]
保存颜色范围内的区域	勾选则保存颜色范围内的区域，不勾选则保存颜色范围外的区域	/	/

1.1.6 伽马图片校正

• 功能

gamma校正改变图片亮度

• 参数说明

参数	说明	默认值	取值范围
gamma补偿系数	该值小于1，图片变暗该值大于1，图片变量	1	[0.1,100]
gamma校正系数	该值小于1，图片变暗，适用于亮度过高的图片该值大于1，图片变亮，适用于亮度过低的图片	2.2	[0.1,100]

1.1.7 填充深度图空洞

• 功能

对深度图中的空洞区域进行填充，并对填充后的深度图进行平滑处理

• 使用场景

因工件本身结构遮挡、光照不均匀等问题，深度图可能缺失工件的部分区域

• 参数说明

参数	说明	默认值	取值范围
填充核大小	空洞填充的大小	3	[1,99]

填充核大小只能填入奇数

• 调参

根据检测结果调整，如果填充过度，应调小参数；如果填充不足，应调大参数

• 示例

1.1.8 边缘增强

• 功能

把图像中纹理的边缘部分置为背景色或者与背景色相差较大的颜色，以凸显工件的边缘信息

• 使用场景

工件相互遮挡或重叠导致边缘不清晰

• 参数说明

参数	说明	默认值	参数范围	调参建议
法向z方向过滤阈值	深度图每个点对应的法向量与相机坐标系Z轴正方向的角度过滤阈值，若点的法向量与相机坐标系的Z轴正方向的角度大于此阈值，则2D图中该点对应位置的颜色将会被置为背景色或者与背景色相差较大的颜色	30	[0,180]	对于平整工件表面，该阈值可以小一些，曲面工件根据表面倾斜程度适当增大
背景色	背景色的RGB颜色阈值	128	[0,255]
自动调节反差背景	勾选 自动调节反差背景后，将2D图中角度大于过滤阈值的点的颜色设置为与背景色相差较大的颜色不勾选 自动调节反差背景，将2D图中角度大于过滤阈值的点的颜色设置为背景色对应的颜色	不勾选	/

• 示例

1.1.9 提取最上层纹理

• 功能

提取最上层或最底层的工件纹理，而把其他区域置为背景色或者与背景颜色相差较大的颜色。

• 使用场景

光照条件不佳、颜色纹理相近、紧密堆叠、交错堆叠或遮挡等因素可能导致模型难以区分上层和下层工件的纹理差异，因此容易误检测。

• 参数说明

参数	说明	默认值	参数范围	单位	调参建议
距离阈值(mm)	点与最上层平面（最底层平面）的距离低于该阈值，则被认为是最上层平面（最底层平面）内的点，应当保留，否则认为是下层（上层）的点，将下层（上层）的点的颜色置为背景色或与背景色相差较大的颜色	50	[0.1, 1000]	mm	一般调整为工件高度的1/2
聚类点云数量	期望参与聚类的点数量，在ROI 3D区域内采样点云的数量	10000	[1，10000000]	/	聚类点云数量越多，模型推理速度下降而精度提升；聚类点云数量越少，模型推理速度提升而精度下降
类别点最小数量	用于过滤类别的最小点数	1000	[1, 10000000]	/	/
自动计算反差背景	勾选 自动计算反差背景后，将2D图中最上层（最底层）外的其他区域设置为与背景色阈值相差较大的颜色不勾选 自动计算反差背景，将2D图中最上层（最底层）外的其他区域设置为背景色阈值对应的颜色	勾选	/	/	/
背景色阈值	背景色RGB颜色阈值	128	[0, 255]	/	/

• 示例

1.1.10 去除roi3d外的图片背景

• 功能

去除2D图像中除了ROI3D区域以外的背景

• 使用场景

图像背景噪声较多影响检测结果

• 参数说明

参数名	说明	默认值	取值范围
填充核大小	空洞填充的大小	5	[1,99]
迭代次数	图片膨胀的迭代次数	1	[1,99]
自动计算反差背景	勾选 自动计算反差背景后，将2D图中roi以外的区域设置为与背景色阈值相差较大的颜色不勾选 自动计算反差背景，将2D图中roi以外的区域设置为背景色阈值对应的颜色	勾选	/
背景色阈值	背景色的RGB颜色阈值	128	[0,255]

填充核大小只能填入奇数

• 调参

如果需要去除图像中更多背景噪声，应当调小填充核大小

• 示例

---

1.2 实例分割

1.2.1 缩放比例

• 功能

通过等比放缩原始图像后再推理以提升2D识别的准确率与召回率。

• 使用场景

检测结果不佳（如未检测到实例、漏识别、包围盒框到多个实例或框不满实例)应调整该函数。

• 参数说明

默认值：1.0

取值范围：[0.01, 3.00]

步长：0.01

• 调参

  • 使用默认值运行，在可视化视窗查看检测结果，若出现未检测到实例、漏识别、包围盒框到多个实例或框不满实例的情况，应调整该函数。

• 2D识别中实例的百分数为置信度分数，数字为实例ID（实例的识别顺序）。

• 2D识别中实例上有颜色的阴影是掩膜，包围实例的矩形框是包围盒，有颜色的点是关键点。

如下图所示，缩放比例为0.83时检测结果显著提高，因此可以确定0.83为缩放比例范围的下限

缩放比例为1.5时检测结果显著降低，因此可以确定1.5为缩放比例范围的上限

---

1.2.2 置信度下限阈值

• 功能

仅保留深度学习模型识别结果分数高于置信度下限阈值的识别结果

• 使用场景

检测结果框取的实例不符预期时，可调整该函数

• 参数说明

默认值：0.5

取值范围：[0.01, 1.00]

• 调参

  • 如果模型检测出的实例较少，应当调小该阈值；取值过小，可能会影响图像识别的准确度。

• 若因为置信度下限阈值过小导致检测出错误的实例，而需要去除这些错误的实例，应当调大该阈值；取值过大，可能会导致保留的检测结果为零，没有结果输出。

---

1.2.3 启用自动增强

• 功能

将输入的缩放比例和旋转角度中所有的值进行组合后推理，返回组合后大于设定置信度下限阈值的所有结果，可以提升模型推理精度, 但会增加耗时。

• 使用场景

单个缩放比例无法满足实际场景需求导致检测不完整或物体摆放倾斜度较高。

• 示例

如果自动增强-缩放比例设置为 [0.8, 0.9, 1.0] ，自动增强-旋转角度设置为 [0, 90.0] ，那么将缩放比例和旋转角度中的值两两组合，模型内部会自动生成6种图片进行推理，最后将这6种推理结果统一到一起，输出大于置信度下限阈值的结果。

自动增强-缩放比例

• 功能

对原始图像进行多次缩放并进行多次推理，输出综合的推理结果

• 使用场景

单个缩放比例无法满足实际场景需求导致检测不完整

• 参数说明

默认值：[1.0]

取值范围：每个缩放比例的范围为[0.1, 3.0]

可设置多个缩放比例，每个缩放比例之间用英文逗号隔开

• 调参

填入多个 1.2.1 缩放比例 获得的检测效果较好的缩放比例

自动增强-旋转角度

• 功能

对原始图像多次旋转并进行多次推理，输出综合的推理结果

• 使用场景

物体摆放偏离坐标轴较多时使用

• 参数说明

默认值：[0.0]

取值范围：每个旋转角度的取值范围为[0, 360]

可设置多个旋转角度，每个旋转角度之间用英文逗号隔开

• 调参

根据实际场景中的物体角度调整自动增强-旋转角度，可根据麻袋的图案和袋口形状、纸箱的棱边和品牌标志判断倾斜角度

1.3 点云生成

实例点云生成形式	掩膜形式（分割后）	—	使用分割后的实例掩膜生成点云
	包围盒形式（分割后）	包围盒缩放比例（分割后）	使用分割后的实例包围盒生成点云
		生成点云是否需要颜色（分割后）	生成的实例点云是否需要附着颜色
	掩膜形式（过滤后）	—	使用过滤后的实例掩膜生成点云
	包围盒形式（过滤后）	包围盒缩放比例（过滤后）	使用过滤后的实例包围盒生成点云
		生成点云是否需要颜色（过滤后）	生成的实例点云是否需要附着颜色

如果不需要加速，无需使用实例过滤函数，使用掩膜形式（分割后）或包围盒形式（分割后）生成实例点云，可在 项目存储文件夹\Project Name\data\PickLight\Historical Data Timestamp\Builder\pose\input folder to view the generated instance point cloud；

如果需要加速，可使用实例过滤函数对实例进行过滤，使用掩膜形式（过滤后）或包围盒形式（过滤后）生成实例点云，可在 项目存储文件夹\Project Name\data\PickLight\Historical Data Timestamp\Builder\pose\input folder to view the generated instance point cloud

1.4 实例过滤

1.4.1 基于包围盒面积过滤

• 功能介绍

根据检测出实例的包围盒的像素面积来进行过滤。

• 使用场景

适用于实例包围盒面积相差较大的场景，通过设置包围盒面积的上限和下限来过滤图像中的噪声，提升图像识别的准确度，避免噪声给后续处理增加耗时。

• 参数说明

参数	说明	默认值	参数范围	单位
最小面积（像素）	该参数用于设置包围盒的最小过滤面积，包围盒面积低于这个值的实例会被过滤	1	[1, 10000000]	像素点
最大面积（像素）	该参数用于设置包围盒的最大过滤面积，包围盒面积高于这个值的实例会被过滤	10000000	[2, 10000000]	像素点

• 示例

按默认值运行，可在日志中查看每个实例的包围盒面积，如下图所示。

根据每个实例的包围盒面积调整 最小面积 和 最大面积，如将最小面积设置为20000，将最大面积设置为30000，即可将像素面积为小于20000或大于30000的实例过滤掉，可在日志中查看实例过滤过程。

1.4.2 基于包围盒长宽比过滤

• 功能介绍

包围盒长宽比在指定范围外的实例将被过滤掉

• 使用场景

适用于实例的包围盒长宽比相差较大的场景

• 参数说明

参数	说明	默认值	参数范围
最小长宽比	包围盒长宽比的最小值，包围盒长宽比低于该值的实例会被过滤	0	[0, 10000000]
最大长宽比	包围盒长宽比的最大值，包围盒长宽比高于该值的实例会被过滤	10000000	[0, 10000000]
使用X/Y轴边长作长宽比	默认不勾选，使用包围盒的较长边/较短边的长度比值作为长宽比，适用于包围盒的长短边长度相差大的情况； 勾选后，则使用像素坐标系下包围盒在X轴/Y轴上的边的长度比值作为长宽比，适用于大部分正常实例包围盒的长边/短边比值近似，但部分异常识别的实例包围盒在X轴上的长度/在Y轴上的长度的比值相差较大的情况。	不勾选	/

1.4.3 基于类别ID过滤实例

• 功能介绍

根据实例类别过滤

• 使用场景

适用于来料有多种类型工件的场景

• 参数说明

参数	说明	默认值
保留的类别ID	保留类别ID在列表内的实例，类别ID不在列表内的实例将被过滤	[0]

• 示例

1.4.4 基于实例点云的边长过滤

• 功能介绍

根据实例点云的长边和短边过滤

• 使用场景

适用于实例点云在x轴或y轴的距离相差较大的场景，通过设置实例点云的距离范围来过滤图像中的噪声，提升图像识别的准确度，避免噪声给后续处理增加耗时。

• 参数说明

参数	说明	默认值	参数范围	单位
短边长度范围（mm）	点云短边的边长范围	[0, 10000]	[0, 10000]	mm
长边长度范围（mm）	点云长边的边长范围	[0, 10000]	[0, 10000]	mm
边缘去噪下限（%）	提取实例点云中X/Y值（相机坐标系）的百分比下限，去除上下限外的点云，避免噪点影响长度计算	5	[0, 100]	/
边缘去噪上限（%）	提取实例点云中X/Y值（相机坐标系）的百分比上限，去除上下限外的点云，避免噪点影响长度计算	95	[0, 100]	/
边长类型	按实例点云的长边、短边过滤，长边、短边的长度不在范围内的实例将被过滤	实例点云短边	实例点云短边；实例点云长边；实例点云长边和短边	/

• 示例

1.4.5 基于分类器的类别ID过滤

• 功能介绍

基于分类器的类别 ID 过滤实例，不在参考类别内的实例将被过滤。

• 使用场景

在多类工件场景中，视觉模型可能会检测出多种类型的工件，但实际作业可能仅需其中某一种类别的工件，此时就可以使用该函数过滤掉不需要的工件

• 参数说明

默认值为[0]，即默认保留类别 ID 为 0 的实例，类别 ID 不在列表内的实例将被过滤。

1.4.6 基于三通道颜色过滤

• 功能介绍

可通过三通道颜色阈值（HSV或者RGB）过滤掉实例。

• 使用场景

错误实例和正确实例颜色有明显区分的情况。

• 参数说明

参数	说明	默认值	取值范围
色域最大值	最大颜色值	[180,255,255]	[[0,0,0],[255,255,255]]
色域最小值	最小颜色值	[0,0,0]	[[0,0,0],[255,255,255]]
过滤百分比阈值	颜色通过率阈值	0.05	[0,1]
反向过滤	勾选则剔除颜色范围外的比例低于阈值的实例，不勾选则剔除实例图像中颜色范围内的比例低于阈值的实例	不勾选	/
颜色模式	颜色过滤中选择的颜色空间	HSV色彩空间	RGB色彩空间HSV色彩空间

• 示例

1.4.7 基于置信度过滤

• 功能介绍

根据实例的置信度分数过滤

• 使用场景

适用于实例的置信度相差较大的场景

• 参数说明

参数	说明	默认值	参数范围
参考置信度度	保留置信度大于阈值的实例，过滤置信度小于阈值的实例。	0.5	[0,1]
反转过滤结果	反转后，保留可见度置信度小于阈值的实例，过滤置信度大于阈值的实例。	不勾选	/

• 示例

1.4.8 基于点云数量过滤

• 功能介绍

根据降采样后的实例点云数量过滤

• 使用场景

实例点云带有较多噪声

• 参数说明

参数	说明	默认值	参数范围
点云数量最小值	点云数量的最小值	3500	[1, 10000000]
点云数量最大值	点云数量的最大值	8500	[2, 10000000]
过滤数量在区间内的实例	勾选则过滤点云数量在最小值和最大值区间内的实例，不勾选则过滤点云数量不在区间内的实例	不勾选	/

1.4.9 基于掩膜面积过滤

• 功能介绍

根据检测出实例的掩膜像素和（即像素面积）过滤图像掩膜。

• 使用场景

适用于实例掩膜面积相差较大的场景，通过设置掩膜的面积上限和下限来过滤图像掩膜中的噪声，提升图像识别的准确度，避免噪声给后续处理增加耗时。

• 参数设置说明

参数名	说明	默认值	参数范围	单位
参考最小面积	该参数用于设置掩膜的最小过滤面积，掩膜面积低于这个值的实例会被过滤	1	[1, 10000000]	像素点
参考最大面积	该参数用于设置掩膜的最大过滤面积，掩膜面积高于这个值的实例会被过滤	10000000	[2, 10000000]	像素点

• 示例

1.4.10 基于可见度过滤

• 功能介绍

根据实例的可见度分数过滤

• 使用场景

适用于实例的可见度相差较大的场景

• 参数说明

参数	说明	默认值	参数范围
参考可见度阈值	保留可见度大于阈值的实例，过滤可见度小于阈值的实例。可见度用于判断图像中的实例可见的程度，工件被遮挡越多，可见度越低。	0.5	[0,1]
反转过滤结果	反转后，保留可见度小于阈值的实例，过滤可见度大于阈值的实例。	不勾选	/

1.4.11 过滤包围盒重叠的实例

• 功能介绍

过滤包围盒交叉重叠的实例

• 使用场景

适用于实例的包围盒相互交叉的场景

• 参数说明

参数	说明	默认值	参数范围
包围盒重叠比例阈值	包围盒交叉的面积与实例包围盒的面积的占比阈值	0.05	[0, 1]
过滤包围盒面积较大的实例	勾选则过滤两个包围盒交叉的实例中面积较大的实例，不勾选则过滤两个包围盒交叉的实例中面积较小的实例	勾选	/

• 示例

新增 过滤被包围实例，以默认值运行，在日志中查看实例包围盒交叉的情况，实例过滤后剩余2个实例

由日志可知，12个实例因为包围盒交叉被过滤掉，剩余2个包围盒没有交叉的实例

将 包围盒重叠比例阈值 设置为0.1，勾选 是否过滤较大的实例，在日志中查看实例例过滤过程，9个实例因为包围盒交叉面积与实例包围盒面积的占比大于0.1被过滤掉，3个实例因为包围盒交叉面积与实例包围盒面积的占比小于0.1被保留，2个实例包围盒没有交叉。

将 包围盒重叠比例阈值 设置为0.1，取消勾选 是否过滤较大的实例，在日志中查看实例例过滤过程，9个实例的包围盒交叉面积与实例包围盒面积的占比大于0.1，但其中2个实例因为包围盒面积小于与其交叉的实例被保留，因此7个实例被过滤，3个实例因为包围盒交叉面积与实例包围盒面积的占比小于0.1被保留，2个实例包围盒没有交叉。

1.4.12 【大师】基于掩膜/掩膜外接多边形面积比，过滤掩膜凹凸的实例

• 功能介绍

计算掩膜/掩膜外接多边形的面积比值，若小于设置的阈值则会过滤掉实例

• 使用场景

适用于工件掩膜存在锯齿/凹凸的情况。

• 参数说明

参数	说明	默认值	取值范围
面积比阈值	掩膜/凸包面积比阈值，若小于设置的阈值则会过滤掉实例。	0.1	[0,1]

1.4.13 【大师】基于点云平均距离过滤

• 功能介绍

基于点云中点到拟合平面的距离的平均值进行过滤，剔除不平整的实例点云

• 使用场景

适用于平面型工件点云弯曲的场景

• 参数说明

参数	说明	默认值	参数范围	单位
平面分割距离阈值（mm）	在弯曲的实例点云中提取一个平面，与平面的距离小于该阈值的点视为该平面的点	10	[-1000, 1000]	mm
平均距离阈值（mm）	实例点云中的点到提取平面的距离的平均值	20	[-1000, 1000]	mm
剔除平均距离小于阈值的实例	勾选则过滤点到提取平面的平均距离小于平均距离阈值的实例，不勾选则过滤点到提取平面的平均距离大于平均距离阈值的实例	不勾选	/	/

1.4.14 【大师】基于掩膜/包围盒面积比，过滤被遮挡的实例

• 功能介绍

计算掩膜/包围盒面积比值，比值不在最大最小范围内的实例将被过滤

• 使用场景

用于过滤被遮挡工件的实例

• 参数说明

，相反，代表可能被遮挡。

参数	说明	默认值	取值范围
最小面积比	掩膜/包围盒面积比例范围下限，比值越小，说明实例被遮挡程度越高	0.1	[0,1]
最大面积比	掩膜/包围盒面积比例范围上限，比值越接近1，说明实例被遮挡程度低	1.0	[0,1]

1.4.15 【大师】判断最上层实例是否全量检出

• 功能介绍

防呆机制之一，判断最上层的实例是否全部被检出，若存在没有被检出的最上层实例则会报错并结束工作流

• 使用场景

适用于拍一次抓多次或者必须要按顺序进行抓取的场景，防止因实例检出不完整造成漏抓影响后续作业

• 参数说明

参数	说明	默认值	参数范围	单位	调参
距离阈值	用于判断最上层的工件，点与工件点云最高点的距离小于距离阈值，则认为这个点是最上层点云，否则认为这个点不是最上层点云。	5	[0.1, 1000]	mm	应当小于工件的高度

1.5 实例排序

• 功能介绍

根据选择的策略对实例进行分组、排序、提取

• 使用场景

拆垛、无序抓取、有序上下料场景通用

如果不需要排序，可以不配置具体的策略。

1.5.1 基准坐标系

• 功能介绍

为所有实例设定一个统一的坐标系，进行实例的分组排序

• 使用场景

拆垛场景、无序抓取场景、有序上下料场景通用

使用坐标相关的策略应当先设置基准坐标系

• 参数说明

参数	说明	图示
相机坐标系	坐标系原点在物体上方，Z轴正方向朝下；XYZ取值是物体中心点在该坐标系下的值
ROI坐标系	坐标系原点大致在垛中心，Z轴正方向朝上；XYZ取值是物体中心点在该坐标系下的值
机械臂坐标系	坐标系原点在机械臂自身，Z轴正方向一般朝上；XYZ取值是物体中心点在该坐标系下的值
像素坐标系	坐标系原点在RGB图的左顶点，是二维平面坐标系；X、Y取值是bbox识别框的x值、bbox识别框的y值，Z是0

1.5.2 通用抓取策略

• 参数说明

参数	说明	默认值
策略	选择依据哪个值进行分组排序以及如何排序，包括实例点云中心XYZ坐标值、包围盒长宽比、实例点云中心距离ROI中心等可叠加多条，按顺序依次执行	实例点云中心X坐标值从小到大(mm)
分组步长	依据选择的策略、按照步长将实例划分为若干组，分组步长即两组实例之间的间隔如策略选择”实例点云中心Z坐标值从大到小(mm)“，则将所有实例点云中心的Z坐标从大到小排序，然后按照步长把Z坐标分组，相应的实例也划分为若干组	/
提取前几组	分组排序之后，需要保留多少组实例	10000

策略名*	说明	分组步长		提取前几组
		默认值	取值范围	默认值
实例点云中心XYZ坐标值从大到小/从小到大(mm)	使用每个实例的点云中心的XYZ坐标值来进行分组排序 使用该策略进行排序前应当先设置基准坐标系	200.000	(0, 10000000]	10000
从实例点云中心XY坐标轴向的中间到两侧/从实例点云中心XY坐标轴的两侧到中间(mm)	使用每个实例的点云中心的 XY 坐标值，按照 “中间到两侧” 或 “两侧到中间” 的方向进行分组排序 使用该策略进行排序前应当先设置基准坐标系	200.000	(0, 10000000]	10000
包围盒中心XY坐标值从大到小/从小到大(mm)	使用像素坐标系下，每个实例的包围盒中心点的XY 坐标值进行分组排序	200.000	(0, 10000000]	10000
包围盒长宽比从大到小/从小到大	使用包围盒的长边/宽边的比值进行分组排序	1	(0, 10000]	10000
从包围盒中心XY坐标轴向的中间到两侧/两侧到中间(mm)	使用包围盒的中心点的XY坐标值，按照 “中间到两侧” 或 “两侧到中间” 的方向进行分组排序	200.000	(0, 10000000]	10000
工件类型ID从大到小/从小到大	使用工件类型的ID进行分组排序，适用于多类工件场景	1	[1, 10000]	10000
局部特征ID从大到小/从小到大	使用局部特征的ID进行分组排序	1	[1, 10000]	10000
置信度从大到小/从小到大	使用每个实例的置信度进行分组排序	1	(0, 1]	10000
可见度从小到大/从大到小	使用每个实例的可见度进行分组排序	1	(0, 0.1]	10000
掩膜面积从大到小/从小到大	使用每个实例的掩膜面积进行分组排序	10000	[1, 10000000]	10000
实例点云中心距离ROI中心近到远/远到近(mm)	使用每个实例的点云中心与ROI坐标系的中心的距离进行分组排序	200.000	(0, 10000000]	10000
实例点云中心距离机器人坐标原点近到远/远到近(mm)	使用每个实例的点云中心与机器人坐标系的原点的距离进行分组排序	200.000	(0, 10000000]	10000

• 示例

1.5.3 自定义抓取策略

（1）功能说明

将Grasping Strategy切换为Custom Grasping Strategy，点击 Add 可增加一条自定义抓取策略。

• 自定义每个工件按照什么顺序抓取，使用通用抓取策略很难实现抓取或者因为点云噪点等问题很难调到合适的参数，可以考虑使用自定义抓取策略

• 自定义抓取策略适用于拆垛场景、有序上下料场景，无序抓取场景不适用，因为自定义抓取策略的工件必须是有序的（即工件的顺序固定）

• 自定义抓取策略只能和单个通用抓取策略组合使用，且策略只能选择Z坐标从小到大

（2）参数说明

参数	说明	默认值	取值范围	调参
IOU阈值	表示标注的bbox框和检测出来的bbox框的重叠度阈值，通过重叠度来确定当前工件实例排序时应该选择哪一张图片上的排序方式。	0.7	[0,1]	阈值越大，匹配越严格，抗干扰性会越差，微小的形状或位置变化都可能导致匹配失败，可能匹配到错误的自定义策略，按错误的顺序进行排序
像素距离阈值	表示可以匹配上的bbox框和检测出来的bbox框在尺寸上的差异性。	100	[0,1000]	阈值越小，匹配越严格，抗干扰性也会更好。如果不同层之间的工件摆放比较相似，也可能误匹配自定义策略，导致排序顺序错误。

（3）选择基准坐标系

使用自定义抓取策略，只能选择相机坐标系或像素坐标系

如果有多层工件，则选择相机坐标系；如果只有一层工件，则选择像素坐标系

（4）策略、分组步长、提取前几组

参数	说明	默认值
策略	只能选择实例点云中心Z坐标值从大到小/从小到大(mm)	/
分组步长	依据Z坐标从小到大策略，将实例的Z坐标从小到大排序，按照步长将实例划分为若干组	10000
提取前几组	分组排序之后，需要保留多少组实例	10000

（5）拍照取图/添加本地图像

点击拍照取图从当前连接的相机获取图像，或点击添加本地图像从本地导入图像，有多少层或有多少种工件的不同摆放形式，就需要拍照取图或添加本地图像得到多少张图片，如果每一层相同，只需要一张即可。鼠标右键点击图像可删除。

在获取的图像上长按拖动鼠标左键标注bbox框，DELETE键可逐步删除标注的bbox框。

2. 3D计算

本节主要对影响抓取点生成的相关函数进行说明和调参建议

2.1 预处理

3D计算的预处理是在对实例进行姿态估计、生成抓取点之前对3D点云进行处理

2.1.1 点云聚类去噪

• 功能

通过点云聚类方式去除噪声

• 使用场景

实例点云中噪声较多

点云聚类去噪可能因为工件点云和噪声点云粘连而把部分工件点云过滤

• 参数说明

参数名	说明	默认值	取值范围	单位	调参建议
点云聚类的距离阈值(mm)	判断空间中的点云是否为同一类别，点云之间的距离低于该阈值则为同一类别	5	[0.1, 1000]	mm	一般不需更改大于工件点云的点间距，小于工件点云与噪声点云的最小距离
最小点数阈值	小于该点数的点云簇将被过滤	100	[1,10000000]	/	一般不需更改根据实例点云中的噪声大小，调大 最小点数阈值
最大点数阈值	大于该点数的点云簇将被过滤	100000	[1,10000000]	/	一般不需更改如果工件点云数量大于100000，调大 最大点数阈值
选取ROI顶部点云	勾选则计算ROI坐标系下同一类别点云的Z坐标平均值并排序，保留Z坐标平均值最大的点云类别（顶部点云），不勾选则保留全部满足条件的点云	不勾选	/	/	如果工件点云在噪声点云之上，勾选则保留工件点云；如果工件点云在噪声点云之下，勾选的同时应当调整ROI坐标系的Z轴朝下，才能保留工件点云
可视化过程数据	勾选则保存去噪后的点云，可在C:_data	不勾选	/	/	调试模式下，如需保存可视化数据可勾选

• 示例

（1）不使用点云聚类去噪，生成的实例点云如下图所示，实例0、1、2都有较多背景噪声

（2）勾选点云聚类去噪，使用默认值运行，如果聚类成功，实例点云中点云数量大于100、小于100000的类别都会保留，点云数量小于100或大于100000的类别会被过滤。查看日志可知，实例4、5、9聚类后点云数量均大于100小于100000，不过滤点云。

（3）调大最小点数阈值为400，实例4、5、9的噪声去除只保留第0类点云

（4）勾选 选取ROI顶部点云，只保留ROI坐标系下Z坐标平均值最大的点云类别。

如下图，实例7聚类后有5类点云，如果把最小点数阈值设置为400，则实例7的第3、4类点云被过滤，第0、1、2类点云被保留。

勾选 选取ROI顶部点云，实例7只保留第0类点云

如果噪声在上工件点云在下，应当在可视化视窗将ROI的Z轴调整向下，让工件点云的Z坐标平均值最大，保留工件点云，否则保留的顶部点云是噪声

（5）如果聚类失败，日志报错如下图，应当调大点云聚类的距离阈值，点云聚类的距离阈值应当大于工件点云的点间距，小于工件点云与噪声点云之间的最小距离

使用meshlab测量生成的实例点云中工件点云的点间距、工件点云与噪声点云的最小距离，如下图所示

如果工件点云数量大于最大点数阈值，那么可能因为点云数量大于100000全部被过滤，导致聚类失败，应当调大最大点数阈值

2.1.2 点云降采样

• 功能

按照指定的点间距来采样点云，以减少计算点的数量，提高模型推理速度，但精度可能下降

• 使用场景

实际场景点云数量过多，勾选点云降采样。

• 参数说明

参数	说明	默认值	参数范围	单位
降采样的点间距(mm)	按照指定的点间距来采样点云	5.0	[0.1, 1000]	mm

• 调参

  • 降采样的点间距取值越大，降采样后的点云数量越少，抓取点计算速度会加快，但精度可能下降

  • 降采样的点间距取值越小，降采样后的点云数量越多，抓取点计算速度下降，精度提升

2.1.3 计算法向量

• 功能

计算点云法向量，供后续处理点云使用

• 参数说明

参数名	说明	默认值	取值范围
固定法向量朝向	计算点云法向量时是否固定朝向，启用后法向量由朝向参考向量决定	勾选	/
法向量计算邻域点数	数值越大，参考的邻域点越多，但可能导致局部变化被忽略，数值越小则相反	30	[1,200]
朝向参考向量	法向量计算朝向参考向量	[0,0,1]	/

• 调参

不能更改

2.1.4 点云轮廓提取

• 功能

从实例点云中提取工件轮廓

• 使用场景

使用 **2.2.2 物体姿态修正 **中的 **使用轮廓模式 **应同时勾选 点云轮廓提取

• 参数说明

参数名	说明	默认值	取值范围	单位	调参建议
参考半径(mm)	在实例点云中提取轮廓的搜索半径	10	[0.1,10000000000]	mm	参考半径建议设置为 2.1.2点云降采样中降采样的点间距的1/2，且必须大于点云间距
点云轮廓搜索模式	搜索点云轮廓的模式	普通模式	普通模式；平面模式	/	一般选择普通模式，平面工件选择平面模式

• 示例

2.1.5 根据色域hsv(色调、饱和度、亮度)过滤点云

• 功能

根据点云图中的色调、饱和度、亮度过滤点云，筛选出与目标范围匹配的点云区域

• 参数说明

参数名	说明	默认值	取值范围
根据hsv过滤深度-色域最大值	过滤点云的最大颜色值	[0.9,0.9,0.9]	[[0,0,0],[1,1,1]]
根据hsv过滤深度-色域最小值	过滤点云的最小颜色值	[0.0,0.0,0.0]	[[0,0,0],[1,1,1]]

• 示例

2.1.6 根据三通颜色过滤点云

• 功能

根据三通颜色过滤点云，筛选出与目标范围匹配的点云区域

• 参数说明

参数名	说明	默认值	取值范围
根据三通颜色过滤点云-颜色最大值	过滤点云的最大颜色值	[0.9,0.9,0.9]	[[0,0,0],[1,1,1]]
根据三通颜色过滤深度-颜色最小值	过滤点云的最小颜色值	[0.0,0.0,0.0]	[[0,0,0],[1,1,1]]

• 示例

2.1.7 选取ROI区域内点云

• 功能

从实例点云中选取ROI 3D区域内点云，默认函数不可删除

• 示例

2.1.8 移除法线大于角度阈值的点

• 功能

去除法向量与标准法向量轴向的角度大于法向量角度阈值的点云

• 使用场景

面型工件上下料（物料相互孤立）

• 参数说明

参数名	说明	默认值	取值范围	单位
角度阈值	大于该角度阈值的点云认为是不同实例	15	[-360, 360]
标准法向量轴向	点云的法向量与标准法向量轴向形成的角度	Z轴	X/Y/Z轴	/
是否使用ROI坐标系	勾选则计算法向量与ROI坐标系的轴之间的角度，不勾选则计算法向量与相机坐标系的轴之间的角度	不勾选	/	/

2.1.9 点云平面分割

• 功能

保留或去除实例点云中点云数量最大的平面

• 使用场景

实例点云中带有噪声平面

• 参数说明

参数	说明	默认值	取值范围	单位	调参建议
平面拟合的参考距离(mm)	点与平面的距离低于参考距离则被认为是平面内的点，否则认为是平面外的点	3	[0.001，10000]	mm	一般不更改
去除平面	勾选则去除点云数量最大的平面，不勾选则保留点云数量最大的平面	不勾选	/	/	如果点云数量最大的平面是工件，则保留平面，不勾选；如果点云数量最大的平面是噪声，则去除平面，应勾选

2.1.10 点云去除离群点

• 功能

识别和去除点云中的离群点噪声，提高点云质量

• 使用场景

实例点云带有较多离群点噪声

• 参数说明

参数名	说明	默认值	取值范围
参考邻域点数	点云中每个点相邻的点的数量，即邻域大小稠密的点云，即使邻域较小也足以反映出工件的特征，因此可以取值较小；较稀疏的点云，需要较大的邻域才能反映出工件的特征，因此应当取值较大。	30	[1, 10000000]
标准差倍数	用于识别离群点噪声，如果点的坐标与实例点云的坐标平均值的偏差超过标准差倍数，则该点被认为是离群点。取值越小，更多的点被认为是离群点，去除的离群点也越多，但可能导致误判而去除了工件的重要特征；取值越大，更少的点被认为是离群点，去除的离群点也越少，但可能保留了一些离群点而影响工件识别准确度。	0.005	[0.0001, 2]

• 调参

一般不更改，若点云去除离群点后点云过于稀少，应当增大标准差倍数

• 示例

2.1.11 滤除物体距离超限的点云

• 功能

滤除指定方向上的点云，去除噪点，提升图像识别的准确度

• 参数说明

参数	说明	默认值	参数范围	单位	调参建议
指定轴	点云指定轴，用于滤除指定方向的点云	Z轴	X/Y/Z轴	/	指定轴一般无需更改
阈值(mm)	在指定轴方向，若下层的点云与工件点云的距离大于该阈值，则下层点云会被滤除；若下层的点云与工件点云的距离小于该阈值，则下层点云会被保留	750	[0, 1000]	mm	根据实际场景调整阈值，阈值越大，滤除的点云越少；阈值越小，滤除的点云越多
选择坐标系	在选择的坐标系下滤除点云	ROI坐标系	相机坐标系；ROI坐标系；物体自身坐标系	/

• 示例

2.1.12 根据点云优化掩膜

• 功能

基于ROI 3D内的点云，剔除掩膜中不在ROI 3D内的点云，提高掩膜的精确度

2.2 姿态估计

2.2.1 粗匹配评判阈值(mm)

• 功能

在粗匹配过程中，模板点云的关键点与模型对实例点云预测的关键点进行匹配，若匹配后两组关键点的平均距离低于评判阈值，则认为两组关键点能够匹配，若匹配后两组关键点的平均距离高于评判阈值则认为两组关键点不能匹配。

• 使用场景

通用工件有序上下料、通用工件无序抓取、通用工件定位装配场景

• 参数说明

默认值：10

取值范围：[1, 1000]

单位：mm

• 调参

通常不更改

• 示例

可在可视化视窗查看使用关键点进行粗匹配的结果，如下图所示。

---

2.2.2 物体姿态修正

精匹配搜索半径(mm)

• 功能

在精匹配过程中，模板点云与实例点云进行匹配，模板点云中每个点都需要在实例点云中搜索最邻近点。精匹配搜索半径既表示在实例点云中的搜索半径，又表示模板点云中的每个点与实例点云中的最邻近点的距离阈值。若点与最邻近点的距离小于精匹配搜索半径，则认为这两个点能够匹配，否则认为两个点不能匹配。

• 使用场景

通用工件有序上下料、通用工件无序抓取、通用工件定位装配场景

• 参数说明

默认值：10

取值范围：[1, 500]

单位：mm

• 调参

通常不更改

• 示例

可在可视化视窗查看精匹配结果，如下图所示。

精匹配搜索模式

• 功能

在精匹配过程中，模板点云在实例点云中搜索最邻近点的方式

• 使用场景

模板点云与实例点云的精匹配结果不佳，应调整该函数

• 参数说明

参数	说明
点到点	模板点云中的每个点在实例点云中搜索最邻近点（搜索半径内直线距离最短的点）适用于所有工件
点到面	模板点云中的每个点沿着其法向量在实例点云中搜索最邻近点适用于几何特征明显的工件
点到点和点到面两种方式结合	先采用点对点模式优化实例点云中的工件姿态，再采用点对面模式优化实例点云中的工件姿态适用于几何特征明显的工件 使用该方法会增加节拍

---

使用轮廓模式

• 功能

提取模板点云和实例点云中的轮廓点云进行粗匹配

• 使用场景

通用工件有序上下料、通用工件无序抓取、通用工件定位装配场景，若使用关键点进行粗匹配的结果不佳，应当勾选该函数使用轮廓点云再次进行粗匹配

• 调参

粗匹配的结果会影响精匹配结果，如果精匹配结果不佳，可勾选 使用使用轮廓

可在可视化视窗查看精匹配结果，如下图所示。

轮廓搜索范围(mm)

• 功能

在模板点云和实例点云中提取轮廓点云的搜索半径

• 使用场景

通用工件有序上下料、通用工件无序抓取、通用工件定位装配场景

• 参数说明

默认值：5

取值范围：[0.1, 500]

单位：m

• 调参

取值较小，搜索轮廓点云的半径较小，适合提取细致的工件轮廓，但提取的轮廓可能包含离群点噪声；

取值较大，搜索轮廓点云的半径较大，适合提取较宽的工件轮廓，但提取的轮廓可能会忽略一些细节特征。

保存姿态估计[精匹配]数据

• 功能

勾选则保存精匹配数据

• 使用场景

通用工件有序上下料、通用工件无序抓取、通用工件定位装配场景

• 示例

精匹配数据保存在项目保存路径\Project Folder\data\PickLight\Historical Data Timestamp\Builder\pose\output folder。

2.2.3 基于轴旋转的姿态调整

• 功能

将实例点云基于工件的第一个抓取点围绕给定轴旋转、平移，计算每次旋转、平移后的实例点云与模板点云的匹配分数，选择匹配分数最高的实例点云作为旋转对称工件的最终姿态。

• 使用场景

旋转对称工件的实例点云与模板点云进行匹配时出现偏差，需要旋转一定角度才能完全匹配

不能与功能选项的识别类型、识别正反（通过点云模板）、识别局部特征联用

• 参数说明

参数	说明	默认值	参数范围	调参建议
旋转角度间隔	对实例点云进行等间隔角度的旋转，相邻两次旋转之间的角度差值。比如第一次旋转30°、第二次旋转60°、第三次旋转90°，则旋转角度间隔为30°。	5	\[1, 180\]	若工件特征较多匹配较难，追求高精度匹配，可设置较小的角度间隔，进行更多次匹配，但计算量会增加。若工件形状简单特征较少，可设置较大的角度间隔，提高计算效率。
旋转角度范围	实例点云从初始状态开始，能旋转多大的角度范围。	90	\[1, 180\]	点云旋转范围是取值的正负区间，比如角度范围取值为120°，则点云旋转范围为\[-120°,120°\]，实例点云将按角度间隔从-120°旋转至120°，旋转次数=旋转角度范围 / 旋转角度间隔。 旋转角度范围一般取值为180、120、90，按旋转对称工件的特征角度设置。 比如正三角形工件旋转 120° 后能与自身重合，那么旋转角度范围应当≥120°，若设置为180°会旋转更多次，更多旋转后的实例点云与模板点云进行匹配。 比如正四边形工件旋转 90° 能与自身重合，那么旋转角度范围应当≥90°。 若填写了位姿角度先验，已知工件大致朝向，那么可根据其可能的偏差范围缩小旋转角度范围，减少计算量。
模板点云文件路径	上传工件的模板点云文件，不上传则使用在工件界面上传的点云模板	/	/
评估模式	从不同角度评估匹配结果好坏	Iso-target	Iso-target；Iso-source；Average；Strict；Loose；Fast	Iso - target：从模板点云的角度评估匹配结果好坏，看模板点云的特征是否被很好地匹配。 Iso - source：从实例点云的角度评估匹配结果好坏，看匹配中的实例点云变化和匹配后实例点云的特征是否合理。 Average：综合考虑模板点云和实例点云在匹配过程中的误差、贴合程度等因素，给出一个综合的评估结果。 Strict：采用较为严格的方式来评估匹配结果，只有当点云匹配度非常高时，才会认为匹配结果良好，适用于工件特征较多、对匹配精度要求较高的场景，如精密零件。 Loose：评估方式相对宽松，允许一定范围内的匹配误差，对匹配精度要求没那么高，适用于对匹配精度要求不高，更希望快速得到匹配结果的场景。 Fast：强调快速评估，适用于对匹配速度要求较高，对匹配精度要求较低的场景。
ICP阈值	判断配准是否成功的标准，匹配结果误差小于该阈值则匹配成功，匹配结果误差大于该阈值则匹配不成功	0.005	\[0.000001, 1\]	一般不更改若实际场景对匹配精度要求较高，则调小该阈值；若实际场景对匹配速度要求较高而对精度要求较低，则调大该阈值。若工件点云质量较好，可调小该阈值；若工件点云质量较差，可调大该阈值。
旋转轴选择	实例点云围绕该轴旋转	Z轴	X/Y/Z轴	一般不更改若旋转对称工件的某条轴对识别工件姿态很重要，可将旋转轴设置为该特征轴
保存可视化数据	是否保存可视化数据	不勾选	/
使用边缘进行优化	使用工件的边缘轮廓优化匹配，减少匹配结果误差，使实例点云和模板点云匹配更精细	不勾选	/	若工件的边缘轮廓具有独特的几何特征，勾选该函数提升匹配精度，比如形状复杂的工件，边缘轮廓差异大，使用边缘轮廓的点云可以更准确地识别工件姿态。
边缘点云文件路径	上传工件的边缘点云文件，不上传则从工件界面上传的点云模板中提取边缘点云	/	/
优化旋转结果	匹配过程中找到最佳姿态后再次优化，减少匹配结果误差，使实例点云和模板点云匹配更精细	勾选	/	默认勾选以提高匹配精度，一般不更改
优化模式	优化匹配结果的模式	Point	Point；Plane；Full	Point（点模式）：基于点与点之间的对应关系进行优化，计算点间距离误差并迭代调整。适用于点云简单、特征少的工件。 Plane（面模式）：考虑点云的平面特征，将点与平面的关系纳入配准计算。适用于有较多平面的工件，如板状工件。 Full（全面模式）：综合考虑点、线、面等多种几何特征进行配准优化，能更全面利用点云信息，但计算量相对较大。适用于点云复杂、几何特征丰富，对精度要求高的工件。
优化阈值	在优化过程中，判断配准是否达到预期精度的标准。配准误差小于该阈值，则优化成功；配准误差大于该阈值，则优化失败需要继续迭代。	0.002	\[0.0001, 1\]	一般不更改对配准精度要求较高的场景，调小该阈值；对配准精度要求较低的场景，调大该阈值。
启用中心移动模式	开启则会同时加入平移模式，对实例点云旋转的同时叠加平移	不勾选		若配准结果存在沿轴向的偏移时，可开启中心移动模式优化配准结果 注意启用中心移动模式需要配合设置评估模式为快速评估
x 轴移动范围	中心移动模式沿抓取点 x 轴移动的范围(mm)	0	\[0,100\]	默认数值设置为0，表示评估时 x 轴不移动；数值不为 0 则表示启用 x 轴的移动 注意轴向设置指定的轴为抓取点方向调整前的方向，抓取点处理函数中的函数可能改变轴向，可关闭抓取点处理函数后查看正确的轴向
x 轴移动步长	中心移动模式沿抓取点 x 轴移动的步长(mm)	2	\[0.01,10\]	步长需根据偏移情况以及精度合理设置，设置过小的步长将导致节拍增加较多
y 轴移动范围	中心移动模式沿抓取点 y 轴移动的范围(mm)	0	\[0,100\]	默认数值设置为0，表示评估时 y 轴不移动；数值不为 0 则表示启用 y 轴的移动 注意轴向设置指定的轴为抓取点方向调整前的方向，抓取点处理函数中的函数可能改变轴向，可关闭抓取点处理函数后查看正确的轴向
y 轴移动步长	中心移动模式沿抓取点 y 轴移动的步长(mm)	2	\[0.01,10\]	步长需根据偏移情况以及精度合理设置，设置过小的步长将导致节拍增加较多
z 轴移动范围	中心移动模式沿抓取点 z 轴移动的范围(mm)	0	\[0,100\]	默认数值设置为0，表示评估时 z 轴不移动；数值不为 0 则表示启用 z 轴的移动 注意轴向设置指定的轴为抓取点方向调整前的方向，抓取点处理函数中的函数可能改变轴向，可关闭抓取点处理函数后查看正确的轴向
z 轴移动步长	中心移动模式沿抓取点 z 轴移动的步长(mm)	2	\[0.01,10\]	步长需根据偏移情况以及精度合理设置，设置过小的步长将导致节拍增加较多

• 示例

轴向匹配偏移

增加中心移动模式之后

• 基于轴旋转的姿态调整所评估的姿态数目说明

总的姿态计算数目 = x 轴移动的点位数目 * y 轴移动的点位数目 * z 轴移动的点位数目 * 角度的点位数目

移动范围过大或步长设置过小将导致需要评估的姿态总数目增多，节拍变长，需合理设置范围以及步长参数

2.3 空ROI判断

• 功能

判断ROI 3D内是否还有工件（点云）剩余，如果ROI 3D内的3D点的数量小于该值，表示没有工件点云剩余，此时不返回点云

• 参数说明

默认值：1000

取值范围：[0, 100000]

• 使用流程

设置ROI 3D最小点数判断阈值，小于该阈值即ROI 3D中工件点云不足，从而判断为无工件在ROI 3D中；

机器人配置中，新增视觉状态码，便于后续机器人进行信号处理。

---

3. Pick Point Processing

This section mainly explains functions related to Pick Point filtering and adjustment, along with Parameter tuning recommendations.

3.1 Pick Point Adjustment

3.1.1 Rotate the Picking Pose when it is outside the angle range

• Function description

When the Picking Pose is outside the configured angle range, it is rotated counterclockwise by a certain angle around a fixed axis. If it is still outside the configured angle range after rotation, a warning is issued.

• Usage scenario

This function is only applicable to depalletizing scenarios. It can keep the robot's approach direction stable during picking and prevent the end effector from repeatedly rotating during the picking process. In 180° cases, it can prevent exceptions such as cable twisting.

• Parameter description

Parameter	Description	Default	Range	Unit
Fixed axis	An axis of the Picking Pose. The pose is rotated counterclockwise around this fixed axis	Z-axis	X/Y/Z-axis	/
Rotation angle	The angle by which the pose is rotated counterclockwise around the fixed axis. Adjust this angle so the Picking Pose satisfies the angle range	0	[-360,360]	degree
Angle range	The angle range of the Picking Pose. Set the angle range according to factors such as material placement, end effector type, and cycle time	[0,180]	[-180,180]	degree
Use current robot Euler Angles	By default, pose calculation uses Euler Angles "XYZ". When selected, the Euler Angles configured for the current robot are used so the pose remains consistent with the robot teach pendant.	Unchecked	/	/
Custom coordinate system	The coordinate system of the Picking Pose	Robot arm coordinate system	Default coordinate system; camera coordinate system; ROI coordinate system; robot arm coordinate system	/

• Example

Without using this function, the generated Pick Points are shown below.

When this function is used with the default values, the RZ angles of the Picking Poses for instances 0, 1, and 2 are all within the angle range [0,180], so no processing is performed. The RZ angle of the Picking Pose for instance 4 is -90°, which is outside the angle range [0,180], so the Picking Pose of instance 4 is rotated by 0° around the fixed Z-axis.

If you want to adjust the RZ angle of the Picking Pose for instance 4 into the angle range, you can change the rotation angle to 180 and rotate the Picking Pose of instance 4 by 180° around the fixed Z-axis.

---

3.1.2 Rotate the Picking Pose so the rotation axis direction matches the target axis direction

• Function description

Rotate the Picking Pose once around the fixed axis so that the direction of the rotation axis (determined by the right-hand rule) matches the positive or negative direction of the target axis in the target coordinate system.

• Usage scenario

Avoid collisions between the robot end effector and the bin.

• Parameter description

Parameter	Description	Default	Range
Rotation axis	An axis of the Picking Pose. Determined by the right-hand rule, the Picking Pose is rotated counterclockwise once around the fixed axis so that the direction of the rotation axis matches the positive or negative direction of the target axis in the target coordinate system	X-axis	X/Y/Z-axis
Fixed axis	The Picking Pose is rotated counterclockwise once around the fixed axis so that the direction of the rotation axis matches the positive or negative direction of the target axis in the target coordinate system	Z-axis	X/Y/Z-axis
Target axis	An axis of the target coordinate system. The Picking Pose is rotated counterclockwise once around the fixed axis so that the direction of the rotation axis matches the positive or negative direction of the target axis in the target coordinate system	X-axis	X/Y/Z-axis
Negative target axis direction	If selected, the direction of the rotation axis is aligned with the negative direction of the target axis in the target coordinate system; otherwise, it is aligned with the positive direction of the target axis in the target coordinate system	Unchecked	/
Custom coordinate system	The coordinate system of the Picking Pose	Default coordinate system	Default coordinate system; camera coordinate system; ROI coordinate system; robot arm coordinate system

• Example

3.1.3 Rotate the Picking Pose so the angle between the rotation axis and the target axis is minimized

• Function description

Rotate the Picking Pose around the fixed axis by 0, 90, 180, and 270 degrees respectively, calculate the angle between the rotated rotation axis and the positive or negative direction of the target axis in the camera coordinate system, and finally output the Picking Pose with the smallest angle after rotation.

• Usage scenario

Avoid collisions between the robot end effector and the bin.

• Parameter description

Parameter	Description	Default	Range
Fixed axis	An axis of the Picking Pose. Rotate the pose counterclockwise around this fixed axis	Z-axis	X/Y/Z-axis
Rotation axis	An axis of the Picking Pose. When rotating the pose, calculate the angle between this rotation axis and the positive or negative direction of the target axis	X-axis	X/Y/Z-axis
Target axis	An axis of the camera coordinate system. When rotating the pose, calculate the angle between the rotation axis and the positive or negative direction of this target axis	X-axis	X/Y/Z-axis
Negative target axis direction	If selected, calculate the angle between the rotation axis and the negative direction of the target axis; otherwise, calculate the angle between the rotation axis and the positive direction of the target axis	Selected	/
Custom coordinate system	The coordinate system of the Picking Pose	Default coordinate system	Default coordinate system; camera coordinate system; ROI coordinate system; robot arm coordinate system

• Example

---

3.1.4 Flip the Picking Pose so the angle between the rotation axis and the target axis is minimized

• Function description

Rotate the Picking Pose once around the fixed axis so that the angle formed between the rotation axis and the positive or negative direction of the target axis in the ROI coordinate system is acute.

• Usage scenario

Avoid collisions between the robot end effector and the bin.

• Parameter description

Parameter	Description	Default	Range
Fixed axis	An axis of the Picking Pose. Rotate the Picking Pose counterclockwise around this fixed axis	Z-axis	X/Y/Z-axis
Rotation axis	An axis of the Picking Pose. Rotate the Picking Pose so that the direction of this rotation axis matches the positive or negative direction of the target axis	X-axis	X/Y/Z-axis
Target axis	An axis in the ROI coordinate system. Rotate the Picking Pose so that the direction of the rotation axis matches the positive or negative direction of this target axis	X-axis	X/Y/Z-axis
Negative target axis direction	If selected, rotate the Picking Pose so that the direction of the rotation axis matches the negative direction of the target axis; otherwise, rotate the Picking Pose so that the direction of the rotation axis matches the positive direction of the target axis	Selected	/
Custom coordinate system	The coordinate system of the Picking Pose	Default coordinate system	Default coordinate system; camera coordinate system; ROI coordinate system; robot arm coordinate system

• Example

3.1.5 Point a Picking Pose axis toward the ROI center

• Function

Rotate the Picking Pose around a fixed axis so that the pointing axis of the Picking Pose points to the ROI center.

• Usage scenario

Avoid collisions between the robot end effector and the bin.

• Parameter description

Parameter	Description	Default	Range
Pointing axis	The axis in the Picking Pose that needs to be adjusted	X-axis	X/Y/Z-axis
Fixed axis	The axis that remains unchanged during rotation	Z-axis	X/Y/Z-axis
Reverse align	If selected, reverse-align the pointing axis to the ROI center; otherwise, align the pointing axis to the ROI center	Selected	/
Strict pointing	If selected, force the Picking Pose to rotate so the pointing axis points to the ROI center	Unchecked	/
Custom coordinate system	The coordinate system of the Picking Pose	Default coordinate system	Default coordinate system; camera coordinate system; ROI coordinate system; robot arm coordinate system

• Example

---

3.1.6 Rotate the Picking Pose so the Z-axis direction matches the Z-axis of the target coordinate system

• Function description

Rotate the Picking Pose so that its Z-axis direction matches the Z-axis of the target coordinate system.

• Usage scenario

Usually this is used by default only in depalletizing scenarios and cannot be deleted. It is used to make the Z-axis of the Picking Pose perpendicular to the Z-axis of the ROI coordinate system (4-axis) or consistent with the direction of the Target Object surface (6-axis).

• Parameter description

Parameter	Description	Default	Range
Robot configuration	Set according to the on-site robot configuration. You can choose 4-axis or 6-axis. If a 6-axis robot is actually used as a 4-axis robot, it should be set to 4-axis	4-axis	4-axis/6-axis
Use ROI Z-axis as target direction	When the robot configuration is set to 4-axis, if selected, the pose is rotated around the X-axis so that the Z-axis direction of the rotated pose matches the positive direction of the ROI Z-axis ; if not selected, the pose is rotated around the X-axis so that the Z-axis direction of the rotated pose matches the positive direction of the Z-axis of the camera coordinate system . When the robot configuration is set to 6-axis, regardless of whether it is selected, the pose is rotated around the X-axis so that the Z-axis direction of the rotated pose matches the positive direction of the Z-axis of the object's own coordinate system	Unchecked	/
Custom coordinate system	The coordinate system of the Picking Pose	Camera coordinate system	Default coordinate system; camera coordinate system; ROI coordinate system; robot arm coordinate system

• Example

3.1.7 Rotate the Picking Pose around a fixed axis

• Function description

Rotate the Picking Pose by a certain angle around a fixed axis.

• Usage scenario

Avoid collisions between the robot end effector and the bin.

• Parameter description

Parameter	Description	Default	Range	Unit
Rotation angle	The angle by which the pose is rotated counterclockwise around the fixed axis	90	[-360, 360]	degree°
Fixed axis	An axis of the Picking Pose. Rotate the pose counterclockwise around this fixed axis	Z-axis	X/Y/Z-axis	/
Custom coordinate system	The coordinate system of the Picking Pose	Default coordinate system	Default coordinate system; camera coordinate system; ROI coordinate system; robot arm coordinate system	/

• Example

3.1.8 Translate the Picking Pose

• Function description

Move the Picking Pose by a certain distance along the translation axis.

• Usage scenario

Avoid collisions between the robot end effector and the bin.

• Parameter description

Parameter	Description	Default	Range	Unit
Translation amount (mm)	The distance the Picking Pose moves along the translation axis. A positive translation amount means translating in the positive direction of the translation axis, and a negative translation amount means translating in the negative direction of the translation axis	0	[-1000, 1000]	mm
Translation axis	The direction in which the Picking Pose moves	X-axis	X/Y/Z-axis	/
Custom coordinate system	The coordinate system of the Picking Pose	Robot arm coordinate system	Default coordinate system; camera coordinate system; ROI coordinate system; robot arm coordinate system	/

• Example

3.1.9 Pick Point Teaching

• Function description

Record the Pick Point coordinates generated by the software and the Pick Point coordinates taught under the current operating condition, then output the transformed Picking Pose based on the offset between the two.

• Usage scenario

When the Pick Points generated by the vision system have an obvious systematic offset and the robot TCP coordinate accuracy is limited or difficult to calibrate, this method can be used to directly map the same offset pattern to subsequent Pick Points, thereby avoiding robot TCP calibration.

• Parameter description

Parameter	Description	Default	Range
Vision Pose	Pick coordinates of the detection result
X(mm)	X coordinate of the Vision Pose	0.00	±10000000, meaning no limit.
Y(mm)	Y coordinate of the Vision Pose	0.00	±10000000, meaning no limit.
Z(mm)	Z coordinate of the Vision Pose	0.00	±10000000, meaning no limit.
RX(°)	X-axis rotation amount of the Vision Pose	0.00	±180
RY(°)	Y-axis rotation amount of the Vision Pose	0.00	±180
RZ(°)	Z-axis rotation amount of the Vision Pose	0.00	±180
Picking Pose	Manually taught Pick Point
X(mm)	X coordinate of the Picking Pose	0.00	±10000000, meaning no limit.
Y(mm)	Y coordinate of the Picking Pose	0.00	±10000000, meaning no limit.
Z(mm)	Z coordinate of the Picking Pose	0.00	±10000000, meaning no limit.
RX(°)	X-axis rotation amount of the Picking Pose	0.00	±180
RY(°)	Y-axis rotation amount of the Picking Pose	0.00	±180
RZ(°)	Z-axis rotation amount of the Picking Pose	0.00	±180

3.1.10 Refine Object Pose based on plane Normal

• Function description

Correct the Object Pose by fitting the plane Normal so that the Z-axis direction of the Object Pose remains consistent with the direction of the plane Normal of the Target Object.

• Usage scenario

When the Target Object contains a plane and there is a tilt deviation in the plane when the template Point Cloud is matched with the actual Point Cloud, use this function to fine-tune the Target Object plane and improve picking accuracy.

Not applicable to depalletizing scenarios

• Parameter description

Parameter	Description	Default	Range	Unit
Distance Threshold	Distance Threshold for fitting a plane from the Point Cloud	10	[-1000, 1000]	mm
Save visualization data	If selected, the visualization data will be saved under the historical data timestamp	Selected	/	/
Custom coordinate system	The coordinate system of the Picking Pose	Camera coordinate system	Default coordinate system; camera coordinate system; ROI coordinate system; robot arm coordinate system	/

• Example

3.1.11 Sort Pick Points by inter-axis angle

• Function

Sort Pick Points according to the angle between an axis of the Picking Pose and the target axis of the ROI.

• Parameter description

Parameter	Description	Default	Range
Axis selection	An axis of the Picking Pose	Z-axis	X/Y/Z-axis
Target axis selection	An axis of the ROI coordinate system	Z-axis	X/Y/Z-axis
Select reverse direction	If selected, calculate the angle with the negative direction of the target axis; otherwise, calculate the angle with the positive direction of the target axis	Unchecked	/
Select descending order	If selected, sort Pick Points from small to large by angle; otherwise, sort Pick Points from large to small by angle	Unchecked	/

3.1.12 [Advanced] Rotate the Picking Pose and automatically compensate for excessive angles to the specified axis

• Function description

Determine whether the angle formed between the specified axis of the Picking Pose and the target axis is within the specified range. If not, adjust the Picking Pose into the specified range.

• Usage scenario

Avoid collisions between the robot end effector and the bin.

• Parameter description

Parameter	Description	Default	Range	Unit
Angle range	Adjust the Picking Pose into the angle range	30	[0, 180]	degree°
Specified axis	An axis of the Picking Pose. Adjust this axis so that it falls within the angle range relative to the target axis of the ROI coordinate system	Z-axis	X/Y/Z-axis	/
Target axis	An axis of the ROI coordinate system. Compare the angle range with the specified axis of the Picking Pose	Z-axis	X/Y/Z-axis	/
Compare with the negative half-axis of the ROI	If not selected, compare the angle range with the positive direction of the target axis of the ROI coordinate system; if selected, compare the angle range with the negative direction of the target axis of the ROI coordinate system	Unchecked	/	/
Custom coordinate system	The coordinate system of the Picking Pose	Default coordinate system	Default coordinate system; camera coordinate system; ROI coordinate system; robot arm coordinate system	/

3.1.13 [Advanced] Symmetry center Object Pose optimization

• Function

Search for the symmetry center of the Target Object based on the instance Mask, then combine it with the plane of the instance or the pose of the ROI 3D center point to calculate the optimal Picking Pose.

Before using this function, first make sure the instance Mask is symmetrical

• Usage scenario

Applicable when the instance Mask of a symmetrical Target Object is also symmetrical, but the Picking Pose is not near the expected center; at the same time, the Target Object has a plane that can be used as a reference, for example, there is a plane on the top of the object, or ROI 3D can be used as a reference for the projected pose.

Applicable project scenarios include brake discs (general circles), refractory bricks (depalletizing), symmetrical irregular parts, fuel fillers, and so on.

• Parameter description

Parameter	Description	Default	Range	Tuning recommendation
Target Object Symmetry type	Target Object Symmetry type of the instance Mask	Rotational symmetry	Rotational symmetry: after the Target Object rotates by a certain angle around the center point, its shape completely overlaps with the original position; mirror symmetry: the Target Object uses a certain axis / plane as the mirror, and the left-right or upper-lower sides are completely symmetrical.	Circles and rectangles are both rotationally symmetrical and mirror-symmetrical, so rotational symmetry is preferred; for trapezoids and other shapes that are symmetrical only along a certain axis or plane, choose mirror symmetry.
Gaussian blur level	Tolerance for determining whether the actual Point Cloud overlaps after rotation	3	\[1,99\]	The larger the value, the stronger the blur, the more smoothing is applied, and the more lenient the overlap judgment after rotation; The smaller the value, the weaker the blur, the less smoothing is applied, and the stricter the overlap judgment. An odd number must be entered
Rotation angle setting	When the symmetry mode is rotational symmetry, it indicates the rotation angle interval, that is, the angle difference between two adjacent rotations. When the symmetry mode is mirror symmetry, it indicates the rotation range, that is, the angle interval within which the Point Cloud can rotate around the symmetry axis.	180	\[1,360\]	For rotational symmetry, set it based on the symmetry of the Target Object. A circle is symmetrical at any angle, so 60 degrees is recommended; A square has 90-degree symmetry, so it can be set to 90 degrees or 180 degrees; A rectangle has 180-degree symmetry, so set it to 180. For mirror symmetry, adjust it within 5° to ensure fine tuning near the symmetry axis
Image scaling ratio	Adjusts the size of the Point Cloud image. The larger this ratio, the smaller the Point Cloud image size and the lower the GPU memory usage, but image detail loss increases, resulting in reduced calculation accuracy .	2	\[1,10000000\]
Search range	Based on the initially determined center of the Target Object, this defines the range expanded outward to search for Point Cloud features. The actual range is (search range*2*image scaling ratio)	10	\[1,10000000\]	For example, for a square Target Object, the initially determined center position of the Target Object is point O. If the search range is set to 10 and the image scaling ratio is 1, then the actual search range is a square region centered at point O with a side length of 10×2×1=20. Point Cloud features are searched within this region to further determine the symmetry center of the Target Object and the optimal Picking Pose. As another example, for a circular Target Object, if the search range is set to 8 and the image scaling ratio is 2, then the actual search range is a circular region centered at the initially determined center of the Target Object with a diameter of 8×2×2=32. Point Cloud features are searched within this region to further determine the Object Pose of the Target Object and the optimal Picking Pose.
Use ROI3D as the reference projection plane	If selected, ROI3D is used as the reference projection plane	Unchecked	/	Select this when the Point Cloud has no obvious plane and the projection plane is difficult to determine; leave it unchecked when the Point Cloud has a clear plane.
Save symmetry center process data	If selected, the debug data generated during the symmetry center process is saved. You can view it in the `\ProjectName`\data`\PickLight`\HistoricalDataTimestamp`\find`\_symmetry_center folder	Unchecked	/	Select this when you need to inspect the detailed process images
Symmetry axis prior type	Effective in ``{=html}mirror symmetry``{=html} mode. Specifies the known Target Object Symmetry type and fixes the asymmetric orientation	Automatic search	Automatic searchSymmetric along the long axisSymmetric along the short axis	If the symmetry axis of the Target Object is the long axis, choose "Symmetric along the long axis". If the symmetry axis of the Target Object is the short axis, choose "Symmetric along the short axis". If uncertain, choose "Automatic search"
Pose adjustment type	Whether to inherit pose-related information from the input pose	Default pose	Default poseInherit rotationInherit translation	/
Symmetry score Threshold	Symmetry results with a symmetry score lower than this Threshold are abnormal results. When set to 0, no filtering is performed	0.0	\[0.0, 1.0\]	/

• Example

3.2 Pick Point Filtering

3.2.1 Filter by fine matching score

• Function description

Filter Pick Points based on the pose fine matching score.

• Parameter description

Parameter	Description	Default	Range
Score Threshold	Retain Pick Points whose fine matching score is greater than this Threshold	0.5	[0, 1]

• Example

3.2.2 Filter Pick Points of occluded Target Objects

• Function description

Determine whether there are too many occluding object Point Clouds in the target detection area along the specified ROI axis or the Picking Pose axis at the Pick Point of the grasped Target Object. If so, the Target Object is considered occluded and the Pick Point is filtered out.

• Usage scenario

Applicable to depalletizing and ordered scenarios in which Target Objects are picked layer by layer, but the model recognizes lower-layer Target Objects. When picking a lower-layer Target Object, the gripper may collide with the upper-layer Target Object.

• Parameter description

Parameter	Description	Default	Range	Unit
Cuboid length in X direction	Set the cuboid length in the X direction of the Picking Pose	1500	[1, 10000]	mm
Cuboid length in Y direction	Set the cuboid length in the Y direction of the Picking Pose	1500	[1, 10000]	mm
Cuboid length in Z direction	Set the cuboid length in the Z direction of the Picking Pose	800	[1, 10000]	mm
Distance Threshold between detection area and Pick Point origin	Along the ROI axis, the nearby cuboid surface area farther than this distance Threshold from the Pick Point origin is regarded as the target detection area	50	[1, 1000]	mm
Point Cloud count Threshold in detection area	If the number of occluding object Point Clouds in the target detection area exceeds this Threshold, the Pick Point is considered occluded	1000	[0, 100000]	/
Specified axis direction	Based on the pose reference specified axis direction, set the specific location of the target detection area within the cuboid space (for example, near the front/back/left/right/top/bottom surface of the cuboid)	[0,0,-1]	[1,0,0]: positive X-axis[-1,0,0]: negative X-axis[0,1,0]: positive Y-axis[0,-1,0]: negative Y-axis[0,0,1]: positive Z-axis[0,0,-1]: negative Z-axis	/
Use ROI 3D pose reference	If selected, adjust the collision detection area according to the ROI 3D pose reference	Unchecked	/	/
Save visualization data	If selected, the visualization data is stored according to the saved data path to help observe whether the generated cuboid is reasonable; if not selected, it is not saved	Unchecked	/	/

• Example

3.2.3 Filter by Picking Pose angle range

• Function description

Determine whether the angle of the Picking Pose is within the constrained angle range, and filter out all Pick Points that do not meet the condition.

• Usage scenario

Prevent collisions caused by abnormal robot arm Picking Pose angles.

• Parameter description

Parameter	Description	Default	Range	Unit
Angle filtering Threshold	Calculate the maximum angle between the specified axis of the ROI and the specified axis of the Picking Pose. Pick Points whose angle is greater than the current Threshold will be filtered out	30	[-360, 360]	degree°
Invert ROI specified axis direction	If selected, use the negative direction of the specified ROI axis for angle calculation; otherwise, use the positive direction of the specified ROI axis for angle calculation	Selected	/	/
Specified Picking Pose axis	Specify an axis of the Picking Pose for angle calculation	Z-axis	X/Y/Z-axis	/
Specified ROI axis	Specify an axis of the ROI coordinate system for angle calculation	Z-axis	X/Y/Z-axis	/

• Example

3.2.4 Filter Pick Points outside the ROI 3D type region

• Function description

Determine whether the Pick Point is within the ROI 3D range, and remove Pick Points that are outside the ROI 3D area.

• Usage scenario

Prevent picking outside the ROI area, which may cause collisions between the robot arm and the target object.

• Parameter description

Parameter	Description	Default
ROI3D type region	Usually "workspace"; "pick area" is a smaller ROI region than "workspace", which can restrict Pick Points to an ROI region smaller than the "workspace" to avoid some collision cases.	Workspace

• Example

As shown in the figure below, when the ROI3D area and ROI2D are area a, the corresponding Pick Point is in the upper-right corner.

When the ROI3D area and ROI2D are changed to area b, the original Pick Point is outside the ROI area, so that Pick Point is removed and a new Pick Point is generated within area b.

---

3.2.5 [New] Filter Pick Points where the Target Object collides with the gripper (including the original function)

[New] Filter Pick Points where the Target Object collides with the gripper

• Function description

Collision detection between the gripper and the Point Cloud near the Pick Point. If the number of Point Clouds in contact with the gripper exceeds the pick collision Threshold, the Pick Point of the Target Object is considered to have a collision risk.

• Usage scenario

Used when collision detection is required between the gripper and the Point Cloud near the Target Object being picked.

• Parameter description

Parameter	Description	Default	Range
Collision Threshold	Collision distance Threshold. If the distance between the scene and the gripper surface is smaller than this Threshold, it is considered a collision. The larger the Threshold, the stricter it is. Unit: mm	7	1-1000
Collision Point Cloud sampling	Sampling size for collision Point Clouds. The larger the value, the faster the cycle time; the smaller the value, the slower the cycle time. Effective only in "Target Object scene Point Cloud only" and "bin + Target Object scene Point Cloud" modes. Unit: mm	5	1 - 1000
Save visualization data for gripper collision detection	Save visualization data for collision detection between the gripper and the picked Target Object	Unchecked	Selected/Unchecked

Filter Pick Points where the Target Object collides with the gripper

• Function description

Collision detection between the gripper and the Point Cloud near the Pick Point. If the number of Point Clouds in contact with the gripper exceeds the pick collision Threshold, the Pick Point of the Target Object is considered to have a collision risk.

• Usage scenario

Used when collision detection is required between the gripper and the Point Cloud near the Target Object being picked.

• Parameter description

Parameter	Description	Default	Range
Pick collision Threshold	The maximum number of Point Clouds the gripper may contain near the Pick Point. For example, 20 means that if the number of scene Point Clouds contained by the gripper exceeds 20, it is considered a collision	20	0-10000
Collision Point Cloud sampling (m)	Downsampling size of the Point Cloud in the collision area. The larger the value, the faster the detection speed, but the lower the accuracy. Applicable scenario: scenarios requiring high cycle rates	0.002	0.0001 - 0.5000
Save visualization data for gripper collision detection	Save visualization data for collision detection between the gripper and the picked Target Object	Unchecked	Selected/Unchecked
Import gripper model	Select and import the gripper model used for collision detection from a folder	/	/

**The gripper should be simplified to fewer than 500 faces**

3.2.6 [Advanced] Retain the one Pick Point with the largest/smallest pose value among instance Pick Points and filter the remaining Pick Points

• Function description

Convert the pose to the specified coordinate system, sort poses according to the value of the specified sorting axis, and retain the pose with the maximum or minimum value. This is suitable for cylindrical Target Objects when keeping the top or bottom Pick Point.

• Parameter description

Parameter	Description	Default	Range
Specified coordinate system	Select which coordinate system the pose should be converted to for processing	ROI coordinate system	ROI coordinate system/camera coordinate system
Specified sorting axis	Select which axis value of the pose to sort by	Z-axis	X/Y/Z-axis
Take minimum value	If selected, retain the pose with the minimum value on the sorting axis; otherwise, retain the pose with the maximum value on the sorting axis	Unchecked	/

• Example

3.2.7 [Advanced] Filter Pick Points close to the previous N Pick Points

• Function description

If the variation between the current Pick Point and any Pick Point in the cache is within the Threshold range, the Pick Point will be filtered out.

• Parameter description

Parameter	Description	Default	Range	Unit
Upper limit of Pick Point change (+)
X(mm)	Upper limit of X coordinate	2	[0, 10000000]	mm
Y(mm)	Upper limit of Y coordinate	2	[0, 10000000]	mm
Z(mm)	Upper limit of Z coordinate	2	[0, 10000000]	mm
RX(°)	Upper limit of RX rotation amount	1	[0, 180]	degree°
RY(°)	Lower limit of RY rotation amount	1	[0, 180]	degree°
RZ(°)	Lower limit of RZ rotation amount	1	[0, 180]	degree°
Lower limit of Pick Point change (-)
X(mm)	Lower limit of X coordinate	2	[0, 10000000]	mm
Y(mm)	Lower limit of Y coordinate	2	[0, 10000000]	mm
Z(mm)	Lower limit of Z coordinate	2	[0, 10000000]	mm
RX(°)	Lower limit of RX rotation amount	1	[0, 180]	degree°
RY(°)	Lower limit of RY rotation amount	1	[0, 180]	degree°
RZ(°)	Lower limit of RZ rotation amount	1	[0, 180]	degree°
Pick Point cache count	Number of Pick Points cached. After the current Pick Point comparison is completed, it will be added to the cache in real time	5	[1, 100]	/

3.2.8 [Advanced] Filter Object Poses close to the previous N Object Poses

• Function description

If the variation between the current Object Pose and any Object Pose in the cache is within the Threshold range, the Object Pose will be filtered out. When an Object Pose is determined to be similar, all Pick Points on that Target Object will be filtered out.

• Parameter description

Parameter	Description	Default	Range	Unit
Upper limit of Object Pose change (+)
X(mm)	Upper limit of X coordinate	2	[0, 10000000]	mm
Y(mm)	Upper limit of Y coordinate	2	[0, 10000000]	mm
Z(mm)	Upper limit of Z coordinate	2	[0, 10000000]	mm
RX(°)	Upper limit of RX rotation amount	1	[0, 180]	degree°
RY(°)	Lower limit of RY rotation amount	1	[0, 180]	degree°
RZ(°)	Lower limit of RZ rotation amount	1	[0, 180]	degree°
Lower limit of Object Pose change (-)
X(mm)	Lower limit of X coordinate	2	[0, 10000000]	mm
Y(mm)	Lower limit of Y coordinate	2	[0, 10000000]	mm
Z(mm)	Lower limit of Z coordinate	2	[0, 10000000]	mm
RX(°)	Lower limit of RX rotation amount	1	[0, 180]	degree°
RY(°)	Lower limit of RY rotation amount	1	[0, 180]	degree°
RZ(°)	Lower limit of RZ rotation amount	1	[0, 180]	degree°
Object Pose cache count	Number of vision Object Poses cached. After the comparison of the current Object Pose is completed, it will be added to the cache in real time	5	[1, 100]	/

3.2.9 [Advanced] Filter Pick Points outside the upper and lower limits of Pick coordinates

• Function description

Retain other Pick Points within the specified range of a reference Pick Point and filter out abnormal Pick Points.

• Usage scenario

Prevent incorrect robot picking and ensure picking accuracy.

This function is not applicable to depalletizing scenarios

• Parameter description

Parameter	Description	Default	Unit
Reference Pick coordinates
X(mm)	X coordinate of the reference Pick Point	0	mm
Y(mm)	Y coordinate of the reference Pick Point	0	mm
Z(mm)	Z coordinate of the reference Pick Point	0	mm
RX(°)	RX rotation amount of the reference Pick Point	0	degree
RY(°)	RY rotation amount of the reference Pick Point	0	degree
RZ(°)	RZ rotation amount of the reference Pick Point	0	degree
Upper limit of Pick coordinates (+)
X(mm)	Upper limit of the X coordinate. For example, if the X coordinate of the reference Pick Point is 100 and the upper limit is set to 10, the allowed range is: [100-lower limit, 110]	10000000, meaning no limit.	mm
Y(mm)	Upper limit of the Y coordinate. For example, if the Y coordinate of the reference Pick Point is 100 and the upper limit is set to 10, the allowed range is: [100-lower limit, 110]	10000000	mm
Z(mm)	Upper limit of the Z coordinate. For example, if the Z coordinate of the reference Pick Point is 100 and the upper limit is set to 10, the allowed range is: [100-lower limit, 110]	10000000	mm
RX(°)	Upper limit of the RX rotation amount. For example, if the RX rotation amount of the reference Pick Point is 180 and the upper limit is set to 10, the allowed range is (default angle wraparound applies): [[-180, -170], [180-lower limit, 180]]	180, meaning no limit.	degree°
RY(°)	Upper limit of the RY rotation amount. For example, if the RY rotation amount of the reference Pick Point is 180 and the upper limit is set to 10, the allowed range is (default angle wraparound applies): [[-180, -170], [180-lower limit, 180]]	180	degree°
RZ(°)	Upper limit of the RZ rotation amount. For example, if the RZ rotation amount of the reference Pick Point is 180 and the upper limit is set to 10, the allowed range is (default angle wraparound applies): [[-180, -170], [180-lower limit, 180]]	180	degree°
Lower limit of Pick coordinates (-)
X(mm)	Lower limit of the X coordinate. For example, if the X coordinate of the reference Pick Point is 100 and the lower limit is set to 10, the allowed range is: [100-lower limit value, 110]	10000000	mm
Y(mm)	Lower limit of the Y coordinate. For example, if the Y coordinate of the reference Pick Point is 100 and the lower limit is set to 10, the allowed range is: [100-lower limit, 110]	10000000	mm
Z(mm)	Lower limit of the Z coordinate. For example, if the Z coordinate of the reference Pick Point is 100 and the lower limit is set to 10, the allowed range is: [100-lower limit, 110]	10000000	mm
RX(°)	Lower limit of the RX rotation amount. For example, if the RX rotation amount of the reference Pick Point is 180 and the lower limit is set to 10, the allowed range is (default angle wraparound applies): [[-180, -180+upper limit], [170, 180]]	180, meaning no limit.	degree°
RY(°)	Lower limit of the RY rotation amount. For example, if the RY rotation amount of the reference Pick Point is 180 and the lower limit is set to 10, the allowed range is (default angle wraparound applies): [[-180, -180+upper limit], [170, 180]]	180	degree°
RZ(°)	Lower limit of the RZ rotation amount. For example, if the RZ rotation amount of the reference Pick Point is 180 and the lower limit is set to 10, the allowed range is (default angle wraparound applies): [[-180, -180+upper limit], [170, 180]]	180	degree°

---

3.3 Pick Point Sorting

3.3.1 Reference coordinate system

• Function description

Set a unified coordinate system for all instances to group and sort instances.

• Usage scenario

Common to depalletizing scenarios, random picking scenarios, and ordered loading/unloading scenarios

Strategies related to coordinates should first set the reference coordinate system

• Parameter description

Parameter	Description	Illustration
Camera coordinate system	The coordinate system origin is above the object, and the positive Z-axis direction points downward; the XYZ values are the values of the center point of the object in this coordinate system
ROI coordinate system	The coordinate system origin is approximately at the center of the pallet stack, and the positive Z-axis direction points upward; the XYZ values are the values of the center point of the object in this coordinate system
Robot arm coordinate system	The coordinate system origin is on the robot arm itself, and the positive Z-axis direction generally points upward; the XYZ values are the values of the center point of the object in this coordinate system
Pixel coordinate system	The coordinate system origin is at the top-left vertex of the RGB image and is a 2D planar coordinate system; the X and Y values are the x value of the bbox detection box and the y value of the bbox detection box, and Z is 0

3.3.2 General picking strategy

• Parameter description

Parameter	Description
Strategy	Select which value is used for grouping and sorting and how to sort it, including Pick Point center X/Y/Z coordinate values from large to small/from small to large (mm), from the middle to the sides / from the sides to the middle along the Pick Point XY coordinate axis (mm). Multiple items can be superimposed and executed in order.
Grouping step size	According to the selected strategy, divide Pick Points into several groups based on the step size. The grouping step size is the interval between two groups of Pick Points
Number of leading groups to keep	After grouping and sorting, how many groups of instances need to be retained

Strategy name*	Description	Grouping step size		Number of leading groups to keep
		Default	Range	Default
Pick Point center X/Y/Z coordinate values from large to small / from small to large (mm)	Use the X/Y/Z coordinate values of the Pick Point center for grouping and sorting	200.000	[0, 10000000]	10000
From the middle to the sides / from the sides to the middle along the Pick Point XY coordinate axis (mm)	Use the X/Y coordinate values of the Pick Point center and perform grouping and sorting in the direction of "middle to sides" or "sides to middle"	200.000	[0, 10000000]	10000

3.3.3 Carton combination strategy

To solve the problems of low efficiency and limited applicable scenarios in traditional single-pick depalletizing, PickWiz adds a carton combination strategy in depalletizing scenarios to support picking multiple Target Objects in a single operation. It supports the core scenarios of "cartons with consistent dimensions" and "rectangular suction cups", covering more real project scenarios.

3.3.3.1 Multi-pick runtime configuration

（1）In sack single depalletizing or carton single depalletizing scenarios, enable Vision computation configuration - Vision computation acceleration.

（2）Under the Pick Point sorting module, select the carton combination strategy;

（3）Strategy selection: available options are the default combination strategy or combination along a specified carton pose axis. These are two methods for finding the largest number of cartons that can be combined.

• Default combination strategy: Find the largest number of cartons that can be combined along the X-axis and Y-axis directions of a carton Picking Pose.

• Combine along a certain carton pose axis: Find the largest number of cartons that can be combined along the X-axis or Y-axis direction of a carton Picking Pose. This is suitable for scenarios where cartons are arranged in a straight line. When using this strategy, you need to choose the carton combination direction, namely the Picking Pose X-axis or the Picking Pose Y-axis.

Note:

The combination direction is related to the positive and negative axis directions. Cartons can be combined only when they are on the same axis and in the same direction, and after combination, the orientation of the whole stack of cartons remains consistent with the orientation of a single carton before combination. Therefore, before combining cartons, make sure all cartons to be combined are placed in the same orientation, and unify the coordinates of the cartons to be combined to the same axis direction.

（4）Combination conditions: determine which cartons can be combined and how many can be combined at most.

• Maximum cartons per row: the maximum number of cartons that can be combined in one row, default is 2.

• Maximum number of combination rows: the maximum number of carton rows that can be combined, default is 1.

• Maximum spacing (mm): cartons to be combined cannot be too far apart in the "combination direction", In the combination direction (axis direction), when the spacing between two adjacent cartons or cartons in different rows is less than this value, they can be combined into one group. The default is 10.

  • Example: when searching for the maximum number of cartons along the Picking Pose X-axis, if the spacing between two adjacent cartons in the Picking Pose X-axis direction is 8 mm (≤10), they can be combined; if the spacing is 12 mm (>10), they cannot be combined.

• Maximum misalignment distance (mm): cartons to be combined cannot be too far apart in the direction "perpendicular to the combination direction" . In the direction perpendicular to the combination direction (axis direction), when the misalignment distance between two adjacent cartons or cartons in different rows is less than this value, they can be combined into one group. The default is 10.

  • Example: when searching for the maximum number of cartons along the Picking Pose X-axis, if two adjacent cartons are offset in the Picking Pose Y-axis direction by 8 mm (≤10), they can be combined; if they are offset by 15 mm (>10), they are no longer aligned and cannot be picked together.

• Maximum angular deviation (°): cartons to be combined should face almost the same direction. In the combination direction (axis direction), when the rotational deviation angle of the cartons is less than this value, they can be combined into one group. The default is 10.

  • Example: if a carton is rotated by 5° relative to the combination direction, as long as it does not exceed 10°, it can be combined; if it is rotated by 15° (>10), the orientation differs too much, the robot will be skewed when picking, and it cannot be combined.

3.3.3.2 Robot configuration

（1）On the robot configuration page, add new placeholders in Vision computation communication message - Robot to PickWiz commands - Vision detection send command: maximum cartons per row and maximum number of combination rows, as shown below;

（2）In Vision computation communication message - PickWiz to robot commands - Pick-related information - Returned information when picking Target Objects, add Object Dimensions length, Object Dimensions width, and Target Object orientation.

After the robot configuration is completed, click the Run button.

3.3.3.3 View multi-pick runtime results

（1）In the 3D Matching window, hover the mouse over an instance to view the combined picking information of a single instance after carton combination, including 2D recognition results, Picking Pose, and instance combination information.

In the visualization window, click the Settings button in the upper right corner to set how the combined instance information is displayed.

Right-click an instance to view the combined picking information and Target Object information of the single instance.

（2）In the 2D recognition window, you can use the relevant combination buttons in the menu bar to view the combined ID, combined Mask, and combined bounding box.

Previous pageDepalletizing Visual Parameter Tuning Guide

Next pageSurface Target Object Visual Parameter Tuning Guide