This chapter mainly introduces how to adjust vision parameters for real-world scenarios involving ordered loading and unloading, random picking, and positioning/assembly of planar workpieces.

Getting Started:

Background Introduction

Build a Project

(1) Create a new project for planar workpiece ordered loading/unloading / planar workpiece random picking / planar workpiece positioning and assembly / planar workpiece positioning and assembly (matching only) (the project name and project path can be customized, but the project name cannot contain Chinese characters)

Workpiece type: planar workpiece (not circular, cylindrical, or quadrilateral, and with only small differences between the front and back sides)

(2) Configure the Camera and Robot

(3) Add a workpiece

• Workpiece Information

The workpiece name can be customized. The workpiece type defaults to the standard workpiece and cannot be changed. The workpiece ID can also be customized and is used to automatically switch the workpiece during Robot picking.

Point Cloud file: the workpiece Point Cloud. Create the workpiece Point Cloud in Point Cloud Template Creation.

Fine matching Point Cloud template: used for fine matching

Camera parameters: not required

• Model Information

Vision model: the 2D recognition solution used for planar workpieces is CAD-based synthetic data training (one-click integration), and the 3D matching solution is 3D registration. The vision model used by different planar workpiece applications must be obtained through one-click integration training.

Mesh file: usually upload the workpiece CAD. To eliminate some noise, it is necessary to normalize the mesh file. You can also normalize the mesh file in Point Cloud Template Creation.

Workpiece attributes: elongated, symmetrical, highly reflective, Low solidity type

Incoming material pattern: custom incoming material pattern ------ enter the incoming material pattern; closely fitted ------ range of the number of workpieces in each row and column

Operating environment: enter the environment file. The environment generated during data generation in one-click integration will be automatically replaced with the entered environment to improve recognition performance.

Workpiece texture: enter the workpiece texture. When training the model in one-click integration, the entered workpiece texture will be used for data augmentation to improve recognition performance.

Mixed random-scene data: when enabled, synthetic data for both random scenes and ordered scenes will be generated during one-click integration training to improve recognition performance.

Maximum number of model recognitions: default is 20; modify it according to scenario requirements

• Pick Point: set Pick Points according to the workpiece

Absolute coordinate system: takes the initial point as the origin. The initial point comes with the workpiece Point Cloud and CAD.

Pick Point coordinate system (offset): takes the current Pick Point as the origin.

(4) Add the Tool, eye-hand calibration, and ROI

(5) Optional functional options: Instance Optimization, Collision Detection, Collision Detection (new version), Image Classification, front/back recognition (via Point Cloud template)

Instance optimization: optimizes the instances generated by the model and processes the instance masks.

Collision Detection (new version): the Collision Detection function is used to detect collisions between the Tool and the container, and to filter out Picking Poses that may collide. Collision Detection User Guide

Image Classification: used to identify different textures, orientations, and other features of the same workpiece. Image Classification User Guide

front/back recognition (via Point Cloud template): Point Cloud templates of the front and back sides of the workpiece can be imported to determine whether the picked workpiece is front-side or back-side, and Pick Points can be configured separately for the front and back sides of the workpiece. front/back recognition (via Point Cloud template) User Guide

(6) Test data (historical data is provided for subsequent practice. You can use the 2D images and 3D Point Clouds in the historical data foreground\input folder`{=tex} instead of capturing images with the Camera to configure the ROI)

Planar workpiece ordered loading/unloading data:

Point Cloud file:

Mesh file:

Vision model:

Tool:

Historical data:

Vision Parameters

• 2D recognition: recognize and segment instances from actual scenes

Preprocessing: process the 2D image before Instance Segmentation (commonly used: fill holes in the depth map & edge enhancement & extract the topmost texture & remove the image background outside ROI 3D)

Instance Segmentation: segment instances (scaling ratio & lower Confidence threshold & automatic enhancement). Clearing the Return mask checkbox can improve speed.

Point Cloud generation: the way to generate the instance Point Cloud, either by using the segmented instance mask or bounding box to generate the instance Point Cloud / using the filtered instance mask or bounding box to generate the instance Point Cloud

Instance filtering: filter the segmented instances

Instance sorting: sort the instances

• 3D computation: compute the pose of the instance in the Camera coordinate system and generate Pick Points

Preprocessing: preprocess the 3D Point Cloud before computing Pick Points

Pose estimation: compute the pose of the instance in the Camera coordinate system (coarse matching, fine matching) and generate Pick Points

• Pick Point processing: filter, adjust, and sort Pick Points

Pick Point filtering: filter Pick Points

Pick Point adjustment: adjust Pick Points

Pick Point sorting: sort Pick Points

---

1. 2D Recognition

1.1 Preprocessing

Preprocessing for 2D recognition processes the 2D image before Instance Segmentation.

1.1.1 Bilateral Filtering

• Function

Image smoothing based on bilateral filtering

• Parameter Description

Parameter	Description	Default Value	Value Range
Maximum depth difference	Maximum depth difference for bilateral filtering	0.03	[0.01, 1]
Filter kernel size	Convolution kernel size for bilateral filtering	7	[1, 3000]

1.1.2 Convert Depth to Normal Vector Map

• Function

Compute pixel normal vectors from the depth map and convert the image into a normal vector map

1.1.3 Image Enhancement

• Function

Common image enhancement functions, such as color saturation, contrast, brightness, and sharpness

• Parameter Description

Parameter	Description	Default Value	Value Range
Image enhancement type	Enhance a specific element of the image	Contrast	Color saturation, contrast, brightness, sharpness
Image enhancement threshold	How much to enhance a specific element of the image	1.5	[0.1, 100]

1.1.4 Histogram Equalization

• Function

Improve image contrast

• Parameter Description

Parameter	Description	Default Value	Value Range
Local mode	Local or global histogram equalization. When checked, local histogram equalization is used; when unchecked, global histogram equalization is used.	Checked	/
Contrast threshold	Contrast threshold	3	[1,1000]

1.1.5 Filter Depth Map by Color

• Function

Filter the depth map based on color values

• Parameter Description

Parameter	Description	Default Value	Value Range
Fill kernel size	Size of color filling	3	[1,99]
Filter depth by HSV - maximum color range value	Maximum color value	[180,255,255]	[[0,0,0],[255,255,255]]
Filter depth by HSV - minimum color range value	Minimum color value	[0,0,0]	[[0,0,0],[255,255,255]]
Keep the area within the color range	When checked, keeps the area within the color range; when unchecked, keeps the area outside the color range	/	/

1.1.6 Gamma Image Correction

• Function

Gamma correction changes image brightness

• Parameter Description

Parameter	Description	Default Value	Value Range
Gamma compensation coefficient	When this value is less than 1, the image becomes darker; when greater than 1, the image changes accordingly	1	[0.1,100]
Gamma correction coefficient	When this value is less than 1, the image becomes darker and is suitable for overly bright images; when greater than 1, the image becomes brighter and is suitable for overly dark images	2.2	[0.1,100]

1.1.7 Fill Holes in the Depth Map

• Function

Fill hole regions in the depth map and smooth the filled depth map

• Applicable Scenarios

Because of issues such as structural occlusion of the workpiece itself and uneven lighting, parts of the workpiece may be missing in the depth map

• Parameter Description

Parameter	Description	Default Value	Value Range
Fill kernel size	Size of hole filling	3	[1,99]

The fill kernel size can only be an odd number

• Parameter Tuning

Adjust according to the detection result. If filling is excessive, reduce the parameter; if filling is insufficient, increase the parameter.

• Example

1.1.8 Edge Enhancement

• Function

Set the edge portions of the texture in the image to the background color or to a color with high contrast from the background color, so as to highlight the edge information of the workpiece

• Applicable Scenarios

Edges are unclear because workpieces occlude or overlap each other

• Parameter Description

Parameter	Description	Default Value	Parameter Range	Tuning Recommendation
Normal z-direction filtering threshold	The filtering threshold for the angle between the normal vector corresponding to each point in the depth map and the positive Z-axis of the Camera coordinate system. If the angle between the normal vector of a point and the positive Z-axis of the Camera coordinate system is greater than this threshold, the color at the corresponding location of that point in the 2D image will be set to the Background color or to a color with high contrast from the Background color.	30	[0,180]	For flat workpiece surfaces, this threshold can be smaller. For curved workpieces, increase it appropriately according to the degree of surface tilt.
Background color	RGB color threshold of the Background color	128	[0,255]
Automatically adjust contrast Background	Checked: after automatic contrast background adjustment is enabled, the colors of points in the 2D image whose angles are greater than the filtering threshold are set to a color with high contrast from the Background color. Unchecked: the colors of points in the 2D image whose angles are greater than the filtering threshold are set to the color corresponding to the Background color.	Unchecked	/

• Example

1.1.9 Extract the Topmost Texture

• Function

Extract the texture of the topmost or bottommost workpiece, while setting other areas to the background color or to a color with high contrast from the background color.

• Applicable Scenarios

Factors such as poor lighting conditions, similar color textures, tightly stacked piles, interleaved stacking, or occlusion may make it difficult for the model to distinguish texture differences between upper-layer and lower-layer workpieces, which can easily lead to false detections.

• Parameter Description

Parameter	Description	Default Value	Parameter Range	Unit	Tuning Recommendation
Distance threshold (mm)	If the distance between a point and the topmost plane (or bottommost plane) is lower than this threshold, the point is considered to be within the topmost plane (or bottommost plane) and should be retained. Otherwise, it is considered to belong to the lower layer (or upper layer), and the color of the lower-layer (or upper-layer) point is set to the Background color or to a color with high contrast from the Background color.	50	[0.1, 1000]	mm	Generally set to 1/2 of the workpiece height
Number of clustered Point Cloud points	The expected number of points participating in clustering, that is, the number of sampled Point Cloud points within the ROI 3D area	10000	[1,10000000]	/	The more Number of clustered Point Cloud points, the slower the model inference speed but the higher the accuracy; the fewer Number of clustered Point Cloud points, the faster the model inference speed but the lower the accuracy
Minimum number of category points	Minimum number of points used to filter categories	1000	[1, 10000000]	/	/
Automatically calculate contrast Background	Checked: after automatic contrast background calculation is enabled, other areas outside the topmost (or bottommost) layer in the 2D image are set to a color with high contrast from the background color threshold. Unchecked: other areas outside the topmost (or bottommost) layer in the 2D image are set to the color corresponding to the background color threshold.	Checked	/	/	/
Background color threshold	RGB color threshold of the Background color	128	[0,255]	/	/

• Example

1.1.10 Remove Image Background Outside ROI 3D

• Function

Remove the background in the 2D image outside the ROI 3D area

• Applicable Scenarios

Too much image background noise affects detection results

• Parameter Description

Parameter Name	Description	Default Value	Value Range
Fill kernel size	Size of hole filling	5	[1,99]
Number of iterations	Number of image dilation iterations	1	[1,99]
Automatically calculate contrast Background	Checked: after automatic contrast background calculation is enabled, areas outside the ROI in the 2D image are set to a color with high contrast from the background color threshold. Unchecked: areas outside the ROI in the 2D image are set to the color corresponding to the background color threshold.	Checked	/
Background color threshold	RGB color threshold of the Background color	128	[0,255]

The fill kernel size can only be an odd number

• Parameter Tuning

If more background noise needs to be removed from the image, reduce the fill kernel size.

• Example

---

1.2 Instance Segmentation

1.2.1 Scaling Ratio

• Function

Improve the accuracy and recall of 2D recognition by scaling the original image proportionally before inference.

• Applicable Scenarios

If the detection effect is poor (for example, no instance is detected, recognition is missed, one bounding box covers multiple instances, or a bounding box does not fully cover an instance), this function should be adjusted.

• Parameter Description

Default value: 1.0

Value range: [0.01, 3.00]

Step size: 0.01

• Parameter Tuning

  • Run with the default value and view the detection result in the visualization window. If no instance is detected, recognition is missed, one bounding box covers multiple instances, or a bounding box does not fully cover an instance, this function should be adjusted.

  

  

  • In 2D recognition, the percentage shown on the instance is the Confidence score, and the number is the instance ID (the recognition order of the instance).

  • In 2D recognition, the colored shading on an instance is the mask, and the rectangle surrounding the instance is the bounding box.

  • Try different scaling ratios and observe changes in the detection result to gradually determine the scaling ratio range. If the detection effect improves significantly at a certain scaling ratio, use that scaling ratio as the lower bound; if the detection effect degrades significantly at a certain scaling ratio, use that scaling ratio as the upper bound.

  If no satisfactory detection result can be obtained after trying all scaling ratios, the ROI area can be adjusted

  As shown below, when the scaling ratio is 0.33, the detection effect improves significantly, so 0.33 can be determined as the lower bound of the scaling ratio range.

  

  

  

  When the scaling ratio is 3, the detection effect is still good, so 3 can be determined as the upper bound of the scaling ratio range.

  

  

  

  • If the actual scenario does not require high picking accuracy, a scaling ratio with good detection performance can be selected within the [0.33,3] range. If the actual scenario requires higher picking accuracy, the scaling ratio range should be refined further and adjusted with a smaller step size until the scaling ratio with the best detection performance is found.

---

1.2.2 Lower Confidence Threshold

• Function

Retain only recognition results whose scores from the deep learning model are higher than the lower Confidence threshold

• Applicable Scenarios

This function can be adjusted when the instances selected by the detection boxes do not match expectations

• Parameter Description

Default value: 0.5

Value range: [0.01, 1.00]

• Parameter Tuning

  • If the model detects too few instances, reduce this threshold. If the value is too small, the accuracy of image recognition may be affected.

  

  

  • If an excessively small lower Confidence threshold causes incorrect instances to be detected and these incorrect instances need to be removed, increase this threshold. If the value is too large, the number of retained detection results may become zero, resulting in no output.

1.2.3 Enable Automatic Enhancement

• Function

All values in the input scaling ratio and rotation angle are combined for inference, and all results above the configured lower Confidence threshold after combination are returned, which can improve model inference accuracy, but it also increases processing time.

• Applicable Scenarios

A single scaling ratio cannot meet actual scenario requirements, resulting in incomplete detection, or the objects are placed with a large tilt angle.

• Example

If Automatic Enhancement - Scaling Ratio is set to [0.8, 0.9, 1.0] and Automatic Enhancement - Rotation Angle is set to [0, 90.0] , then the values in the scaling ratio and rotation angle will be combined pairwise. The model will automatically generate 6 images internally for inference, and finally the results of these 6 inferences will be unified and the results greater than the lower Confidence threshold will be output.

Automatic Enhancement - Scaling Ratio

• Function

Scale the original image multiple times and run inference multiple times to output a consolidated inference result

• Applicable Scenarios

A single scaling ratio cannot meet actual scenario requirements, resulting in incomplete detection

• Parameter Description

Default value: [1.0]

Value range: the range of each scaling ratio is [0.1, 3.0]

Multiple scaling ratios can be set, separated by English commas

• Parameter Tuning

Enter multiple scaling ratios from 1.2.1 Scaling Ratio that produce good detection results

Automatic Enhancement - Rotation Angle

• Function

Rotate the original image multiple times and run inference multiple times to output a consolidated inference result

• Applicable Scenarios

Used when the object is placed with a significant deviation from the coordinate axes

• Parameter Description

Default value: [0.0]

Value range: the value range of each rotation angle is [0, 360]

Multiple rotation angles can be set, separated by English commas

• Parameter Tuning

Adjust Automatic Enhancement - Rotation Angle according to the object angle in the actual scenario. The tilt angle can be judged based on sack patterns and bag opening shapes, or carton edges and brand logos.

1.3 Point Cloud Generation

Instance Point Cloud generation form	Mask form (after segmentation)	—	Generate the Point Cloud using the segmented instance mask
	Bounding box form (after segmentation)	Bounding box scaling ratio (after segmentation)	Generate the Point Cloud using the segmented instance bounding box
		Whether color is required when generating the Point Cloud (after segmentation)	Whether color needs to be attached to the generated instance Point Cloud
	Mask form (after filtering)	—	Generate the Point Cloud using the filtered instance mask
	Bounding box form (after filtering)	Bounding box scaling ratio (after filtering)	Generate the Point Cloud using the filtered instance bounding box
		Whether color is required when generating the Point Cloud (after filtering)	Whether color needs to be attached to the generated instance Point Cloud

If acceleration is not required, there is no need to use the Instance Filtering function. Use Mask form (after segmentation) or Bounding box form (after segmentation) to generate the instance Point Cloud. You can view the generated instance Point Cloud in the project storage folder \ProjectName\data\PickLight\HistoricalDataTimestamp\Builder\pose\input;

If acceleration is required, the Instance Filtering function can be used to filter instances. Use Mask form (after filtering) or Bounding box form (after filtering) to generate the instance Point Cloud. You can view the generated instance Point Cloud in the project storage folder \ProjectName\data\PickLight\HistoricalDataTimestamp\Builder\pose\input

1.4 Instance Filtering

1.4.1 Filter Based on Bounding Box Area

• Function Overview

Filter according to the pixel area of the bounding box of the detected instance.

• Applicable Scenarios

Applicable to scenarios where the areas of instance bounding boxes differ greatly. By setting the upper and lower limits of the bounding box area, image noise can be filtered out, improving image recognition accuracy and preventing noise from increasing the time required for subsequent processing.

• Parameter Description

Parameter	Description	Default Value	Parameter Range	Unit
Minimum area (pixels)	This parameter sets the minimum filtering area of the bounding box. Instances whose bounding box area is lower than this value will be filtered out.	1	[1, 10000000]	pixels
Maximum area (pixels)	This parameter sets the maximum filtering area of the bounding box. Instances whose bounding box area is higher than this value will be filtered out.	10000000	[2, 10000000]	pixels

• Example

Run with the default values. The bounding box area of each instance can be viewed in the logs, as shown below.

Adjust Minimum area and Maximum area according to the bounding box area of each instance. For example, set Minimum area to 20000 and Maximum area to 30000 to filter out instances whose pixel area is less than 20000 or greater than 30000. The instance filtering process can be viewed in the logs.

1.4.2 Filter Based on Bounding Box Aspect Ratio

• Function Overview

Instances whose bounding box aspect ratios are outside the specified range will be filtered out

• Applicable Scenarios

Applicable to scenarios where instance bounding box aspect ratios differ greatly

• Parameter Description

Parameter	Description	Default Value	Parameter Range
Minimum aspect ratio	Minimum value of the bounding box aspect ratio. Instances whose bounding box aspect ratio is lower than this value will be filtered out.	0	[0, 10000000]
Maximum aspect ratio	Maximum value of the bounding box aspect ratio. Instances whose bounding box aspect ratio is higher than this value will be filtered out.	10000000	[0, 10000000]
Use X/Y axis side lengths as the aspect ratio	By default, this is unchecked, and the ratio of the longer side to the shorter side of the bounding box is used as the aspect ratio, which is suitable when the longer and shorter sides of the bounding box differ greatly in length; when checked, the ratio of the side length of the bounding box on the X-axis to that on the Y-axis in the pixel coordinate system is used as the aspect ratio, which is suitable when the longer-side/shorter-side ratio of most normal instance bounding boxes is similar, but some abnormally recognized instance bounding boxes have large differences in the ratio of length on the X-axis to length on the Y-axis.	Unchecked	/

1.4.3 Filter Instances Based on Category ID

• Function Overview

Filter by instance category

• Applicable Scenarios

Applicable to scenarios where the incoming materials contain multiple types of workpieces

• Parameter Description

Parameter	Description	Default Value
Retained category IDs	Retain instances whose category IDs are in the list; instances whose category IDs are not in the list will be filtered out	[0]

• Example

1.4.4 Filter Based on Side Lengths of the Instance Point Cloud

• Function Overview

Filter according to the long side and short side of the instance Point Cloud

• Applicable Scenarios

Applicable to scenarios where the distances of the instance Point Cloud on the x-axis or y-axis differ greatly. By setting the distance range of the instance Point Cloud, image noise can be filtered out, improving image recognition accuracy and preventing noise from increasing the time required for subsequent processing.

• Parameter Description

Parameter	Description	Default Value	Parameter Range	Unit
Short side length range (mm)	Side length range of the short side of the Point Cloud	[0, 10000]	[0, 10000]	mm
Long side length range (mm)	Side length range of the long side of the Point Cloud	[0, 10000]	[0, 10000]	mm
Lower bound for edge denoising (%)	Extract the lower percentile bound of X/Y values (in the Camera coordinate system) from the instance Point Cloud, and remove Point Clouds outside the upper and lower bounds to avoid noise affecting length calculation	5	[0, 100]	/
Upper bound for edge denoising (%)	Extract the upper percentile bound of X/Y values (in the Camera coordinate system) from the instance Point Cloud, and remove Point Clouds outside the upper and lower bounds to avoid noise affecting length calculation	95	[0, 100]	/
Side length type	Filter according to the long side and/or short side of the instance Point Cloud. Instances whose long-side and/or short-side lengths are outside the range will be filtered out.	Instance Point Cloud short side	Instance Point Cloud short side; Instance Point Cloud long side; Instance Point Cloud long side and short side	/

• Example

1.4.5 Category ID Filtering Based on the Classifier

• Function Overview

Filter instances based on classifier category IDs. Instances not in the reference categories will be filtered out.

• Applicable Scenarios

In multi-category workpiece scenarios, the vision model may detect multiple types of workpieces, but the actual operation may require only one specific category. In such cases, this function can be used to filter out unnecessary workpieces.

• Parameter Description

The default value is [0], which means that instances with category ID 0 are retained by default. Instances whose category IDs are not in the list will be filtered out.

1.4.6 Filter Based on Three-Channel Color

• Function Overview

Instances can be filtered out by three-channel color thresholds (HSV or RGB).

• Applicable Scenarios

Cases where incorrect instances and correct instances can be clearly distinguished by color.

• Parameter Description

Parameter	Description	Default Value	Value Range
Maximum color range value	Maximum color value	[180,255,255]	[[0,0,0],[255,255,255]]
Minimum color range value	Minimum color value	[0,0,0]	[[0,0,0],[255,255,255]]
Filtering percentage threshold	Color pass-rate threshold	0.05	[0,1]
Reverse filtering	When checked, removes instances whose proportion outside the color range is lower than the threshold. When unchecked, removes instances whose proportion within the color range in the instance image is lower than the threshold.	Unchecked	/
Color mode	The color space selected for color filtering	HSV color space	RGB color space; HSV color space

• Example

1.4.7 Filter Based on Confidence

• Function Overview

Filter according to the Confidence score of the instance

• Applicable Scenarios

Applicable to scenarios where instance Confidence values differ greatly

• Parameter Description

Parameter	Description	Default Value	Parameter Range
Reference Confidence value	Retain instances whose Confidence is greater than the threshold, and filter out instances whose Confidence is less than the threshold.	0.5	[0,1]
Invert filtering result	After inversion, retain instances whose Confidence is less than the threshold, and filter out instances whose Confidence is greater than the threshold.	Unchecked	/

• Example

1.4.8 Filter Based on Point Cloud Quantity

• Function Overview

Filter according to the number of points in the downsampled instance Point Cloud

• Applicable Scenarios

The instance Point Cloud contains a large amount of noise

• Parameter Description

Parameter	Description	Default Value	Parameter Range
Minimum Point Cloud quantity	Minimum value of the Point Cloud quantity	3500	[1, 10000000]
Maximum Point Cloud quantity	Maximum value of the Point Cloud quantity	8500	[2, 10000000]
Filter instances whose quantity falls within the interval	When checked, filters out instances whose Point Cloud quantity is within the interval between the minimum and maximum values; when unchecked, filters out instances whose Point Cloud quantity is outside the interval	Unchecked	/

1.4.9 Filter Based on Mask Area

• Function Overview

Filter image masks according to the sum of mask pixels (that is, the pixel area) of the detected instances.

• Applicable Scenarios

Applicable to scenarios where instance mask areas differ greatly. By setting the upper and lower limits of the mask area, noise in image masks can be filtered out, improving image recognition accuracy and preventing noise from increasing the time required for subsequent processing.

• Parameter Setting Description

Parameter Name	Description	Default Value	Parameter Range	Unit
Reference minimum area	This parameter sets the minimum filtering area of the mask. Instances whose mask area is lower than this value will be filtered out.	1	[1, 10000000]	pixels
Reference maximum area	This parameter sets the maximum filtering area of the mask. Instances whose mask area is higher than this value will be filtered out.	10000000	[2, 10000000]	pixels

• Example

1.4.10 Filter Based on Visibility

• Function Overview

Filter according to the visibility score of the instance

• Applicable Scenarios

Applicable to scenarios where instance visibility values differ greatly

• Parameter Description

Parameter	Description	Default Value	Parameter Range
Reference visibility threshold	Retain instances whose visibility is greater than the threshold, and filter out instances whose visibility is less than the threshold. Visibility is used to judge the degree to which an instance is visible in the image: the more the workpiece is occluded, the lower its visibility.	0.5	[0,1]
Invert filtering result	After inversion, retain instances whose visibility is less than the threshold, and filter out instances whose visibility is greater than the threshold.	Unchecked	/

1.4.11 Filter Instances with Overlapping Bounding Boxes

• Function Overview

Filter out instances whose bounding boxes intersect and overlap

• Applicable Scenarios

Applicable to scenarios where instance bounding boxes intersect each other

• Parameter Description

Parameter	Description	Default Value	Parameter Range
Bounding box overlap ratio threshold	Threshold for the ratio of the intersecting area of bounding boxes to the area of the instance bounding box	0.05	[0, 1]
Filter the instance with the larger bounding box area	When checked, filters out the instance with the larger area among the two instances whose bounding boxes intersect; when unchecked, filters out the instance with the smaller area among the two instances whose bounding boxes intersect	Checked	/

• Example

New in this version: filter enclosed instances. Run with the default values and view the bounding box intersection status of instances in the logs. After instance filtering, 2 instances remain.

The logs show that 12 instances were filtered out because of bounding box intersections, leaving 2 instances whose bounding boxes do not intersect.

Set Bounding box overlap ratio threshold to 0.1, and check whether to filter larger instances. View the instance filtering process in the logs. Nine instances are filtered out because the ratio of the intersecting area of the bounding boxes to the area of the instance bounding box is greater than 0.1; three instances are retained because the ratio is less than 0.1; and two instances have no bounding box intersections.

Set Bounding box overlap ratio threshold to 0.1, and uncheck whether to filter larger instances. View the instance filtering process in the logs. For 9 instances, the ratio of the intersecting area of the bounding boxes to the area of the instance bounding box is greater than 0.1, but 2 of these instances are retained because their bounding box areas are smaller than those of the instances intersecting them. Therefore, 7 instances are filtered out; 3 instances are retained because the ratio of the intersecting area of the bounding boxes to the area of the instance bounding box is less than 0.1; and 2 instances have no bounding box intersections.

1.4.12 [Expert] Filter Instances with Concave/Convex Masks Based on the Area Ratio of Mask / Circumscribed Polygon of the Mask

• Function Overview

Calculate the area ratio of the mask to the circumscribed polygon of the mask. If the ratio is less than the configured threshold, the instance will be filtered out.

• Applicable Scenarios

Applicable to cases where the workpiece mask has serrations / concave-convex irregularities.

• Parameter Description

Parameter	Description	Default Value	Value Range
Area ratio threshold	Threshold for the mask / convex hull area ratio. If the ratio is less than the configured threshold, the instance will be filtered out.	0.1	[0,1]

1.4.13 [Expert] Filter Based on the Average Point Cloud Distance

• Function Overview

Filter based on the average value of the distances from points in the Point Cloud to the fitted plane, removing non-flat instance Point Clouds

• Applicable Scenarios

Applicable to scenarios where the Point Cloud of a planar workpiece is bent

• Parameter Description

Parameter	Description	Default Value	Parameter Range	Unit
Plane segmentation distance threshold (mm)	Extract a plane from the bent instance Point Cloud. Points whose distance from the plane is less than this threshold are regarded as points on the plane.	10	[-1000, 1000]	mm
Average distance threshold (mm)	Average value of the distances from points in the instance Point Cloud to the extracted plane	20	[-1000, 1000]	mm
Remove instances whose average distance is less than the threshold	When checked, filters out instances whose average distance from the points to the extracted plane is less than the average distance threshold. When unchecked, filters out instances whose average distance from the points to the extracted plane is greater than the average distance threshold.	Unchecked	/	/

1.4.14 [Expert] Filter Occluded Instances Based on the Area Ratio of Mask / Bounding Box

• Function Overview

Calculate the area ratio of the mask to the bounding box. Instances whose ratios are outside the minimum and maximum ranges will be filtered out.

• Applicable Scenarios

Used to filter instances of occluded workpieces

• Parameter Description

Conversely, a smaller ratio indicates possible occlusion.

Parameter	Description	Default Value	Value Range
Minimum area ratio	Lower bound of the mask / bounding box area ratio range. The smaller the ratio, the more severely the instance is occluded.	0.1	[0,1]
Maximum area ratio	Upper bound of the mask / bounding box area ratio range. The closer the ratio is to 1, the less the instance is occluded.	1.0	[0,1]

1.4.15 [Expert] Determine Whether All Top-Layer Instances Have Been Fully Detected

• Function Overview

As one of the foolproof mechanisms, determine whether all top-layer instances have been fully detected. If any top-layer instance has not been detected, an error will be reported and the Workflow will be terminated.

• Applicable Scenarios

Applicable to scenarios where one image is captured for multiple picks or where picking must be performed in sequence, preventing missed picks from affecting subsequent operations due to incomplete instance detection

• Parameter Description

Parameter	Description	Default Value	Parameter Range	Unit	Parameter Tuning
Distance threshold	Used to determine top-layer workpieces. If the distance between a point and the highest point of the workpiece Point Cloud is less than the distance threshold, the point is considered part of the top-layer Point Cloud; otherwise, it is not considered part of the top-layer Point Cloud.	5	[0.1, 1000]	mm	Should be less than the height of the workpiece

1.5 Instance Sorting

• Function Overview

Group, sort, and extract instances according to the selected strategy

• Applicable Scenarios

Common to depalletizing, random picking, and ordered loading/unloading scenarios

If sorting is not required, you do not need to configure a specific strategy.

1.5.1 Reference Coordinate System

• Function Overview

Set a unified coordinate system for all instances to group and sort instances

• Applicable Scenarios

Common to depalletizing scenarios, random picking scenarios, and ordered loading/unloading scenarios

Coordinate-related strategies should be configured only after setting the reference coordinate system

• Parameter Description

Parameter	Description	Illustration
Camera coordinate system	The coordinate system origin is above the object, and the positive Z-axis points downward; XYZ values are the values of the object center point in this coordinate system
ROI coordinate system	The coordinate system origin is approximately at the center of the stack, and the positive Z-axis points upward; XYZ values are the values of the object center point in this coordinate system
Robot coordinate system	The coordinate system origin is on the Robot itself, and the positive Z-axis generally points upward; XYZ values are the values of the object center point in this coordinate system
Pixel coordinate system	The coordinate system origin is at the top-left vertex of the RGB image and is a two-dimensional planar coordinate system; X and Y values are the x and y coordinates of the bounding box, and Z is 0

1.5.2 General Picking Strategy

• Parameter Description

Parameter	Description	Default Value
Strategy	Select which value is used for grouping and sorting and how to sort, including the XYZ coordinates of the center of the instance Point Cloud, bounding box aspect ratio, distance from the center of the instance Point Cloud to the ROI center, and so on. Multiple items can be superimposed and executed sequentially in order.	Center X coordinate of instance Point Cloud from small to large (mm)
Grouping step size	According to the selected strategy, instances are divided into several groups by the step size. The grouping step size is the interval between groups. For example, if the strategy is "Center Z coordinate of instance Point Cloud from large to small (mm)", then the Z coordinates of all instance Point Cloud centers are sorted from large to small and grouped according to the step size, and the corresponding instances are also divided into several groups.	/
Extract the first several groups	How many groups of instances need to be retained after grouping and sorting	10000

Strategy name*	Description	Grouping step size		Extract the first several groups
		Default Value	Value Range	Default Value
Center XYZ coordinate values of instance Point Cloud from large to small / from small to large (mm)	Use the XYZ coordinate values of the center of each instance Point Cloud for grouping and sorting Before using this strategy for sorting, the reference coordinate system should be set first	200.000	(0, 10000000]	10000
From the middle to both sides / from both sides to the middle along the XY coordinate axes of the center of the instance Point Cloud (mm)	Use the XY coordinate values of the center of each instance Point Cloud and group and sort them in the direction of "from the middle to both sides" or "from both sides to the middle" Before using this strategy for sorting, the reference coordinate system should be set first	200.000	(0, 10000000]	10000
Center XY coordinate values of bounding boxes from large to small / from small to large (mm)	Use the XY coordinate values of the center point of each instance bounding box in the pixel coordinate system for grouping and sorting	200.000	(0, 10000000]	10000
Bounding box aspect ratio from large to small / from small to large	Use the ratio of the long side to the short side of the bounding box for grouping and sorting	1	(0, 10000]	10000
From the middle to both sides / from both sides to the middle along the XY coordinate axes of the bounding box center (mm)	Use the XY coordinate values of the center point of the bounding box and group and sort them in the direction of "from the middle to both sides" or "from both sides to the middle"	200.000	(0, 10000000]	10000
Workpiece type ID from large to small / from small to large	Use the workpiece type ID for grouping and sorting, suitable for multi-category workpiece scenarios	1	[1, 10000]	10000
Local feature ID from large to small / from small to large	Use the ID of the local feature for grouping and sorting	1	[1, 10000]	10000
Confidence from large to small / from small to large	Use the Confidence of each instance for grouping and sorting	1	(0, 1]	10000
Visibility from small to large / from large to small	Use the visibility of each instance for grouping and sorting	1	(0, 0.1]	10000
Mask area from large to small / from small to large	Use the mask area of each instance for grouping and sorting	10000	[1, 10000000]	10000
Distance from the center of the instance Point Cloud to the ROI center from near to far / from far to near (mm)	Use the distance between the center of each instance Point Cloud and the center of the ROI coordinate system for grouping and sorting	200.000	(0, 10000000]	10000
Distance from the center of the instance Point Cloud to the origin of the Robot coordinate system from near to far / from far to near (mm)	Use the distance between the center of each instance Point Cloud and the origin of the Robot coordinate system for grouping and sorting	200.000	(0, 10000000]	10000

• Example

1.5.3 Custom Picking Strategy

(1) Function Description

Switch Picking Strategy to Custom Picking Strategy, then click Add to add one custom picking strategy.

• Customize the picking order for each workpiece. If the general picking strategy is difficult to use for picking, or if suitable parameters are difficult to tune because of Point Cloud noise and other issues, you can consider using a custom picking strategy.

• The custom picking strategy is suitable for depalletizing scenarios and ordered loading/unloading scenarios, but not for random picking scenarios, because the workpieces for a custom picking strategy must be ordered (that is, the workpiece order is fixed).

• A custom picking strategy can only be combined with a single general picking strategy, and the strategy can only select the Z coordinate from small to large

(2) Parameter Description

Parameter	Description	Default Value	Value Range	Parameter Tuning
IOU threshold	Represents the overlap threshold between the annotated bounding box and the detected bounding box. The overlap is used to determine which image's sorting method should be selected when sorting the current workpiece instance.	0.7	[0,1]	The larger the threshold, the stricter the matching, but the worse the anti-interference capability. Small shape or position changes may cause matching to fail, which may lead to matching the wrong custom strategy and sorting in the wrong order.
Pixel distance threshold	Represents the size difference between a matched bounding box and the detected bounding box.	100	[0,1000]	The smaller the threshold, the stricter the matching, and the better the anti-interference capability. However, if the workpiece placement between different layers is similar, the wrong custom strategy may still be matched, resulting in incorrect sorting order.

(3) Select the Reference Coordinate System

When using a custom picking strategy, only the Camera coordinate system or the pixel coordinate system can be selected

If there are multiple layers of workpieces, select the Camera coordinate system; if there is only one layer of workpieces, select the pixel coordinate system

(4) Strategy, Grouping Step Size, and Extract the First Several Groups

Parameter	Description	Default Value
Strategy	Only Center Z coordinate value of instance Point Cloud from large to small / from small to large (mm) can be selected	/
Grouping step size	According to the strategy of ordering Z coordinates from small to large, the Z coordinates of instances are sorted from small to large and divided into groups according to the step size	10000
Extract the first several groups	How many groups of instances need to be retained after grouping and sorting	10000

(5) Capture Image / Add Local Image

Click Capture Image to acquire an image from the currently connected Camera, or click Add Local Image to import an image locally. For however many layers there are, or however many different placement forms of workpieces there are, you need to capture or add the same number of images. If every layer is the same, only one image is required. Right-click an image to delete it.

On the acquired image, click and hold the left mouse button and drag to annotate a bounding box. The DELETE key can be used to delete annotated bounding boxes step by step.

---

2. 3D Computation

2.1 Preprocessing

Preprocessing for 3D computation processes the 3D Point Cloud before pose estimation and Pick Point generation for instances.

2.1.1 Point Cloud Clustering Denoising

• Function

Remove noise through Point Cloud clustering

• Applicable Scenarios

There is a large amount of noise in the instance Point Cloud

• Parameter Description

Parameter Name	Description	Default Value	Value Range	Unit	Tuning Recommendation
Distance threshold for Point Cloud clustering (mm)	Determines whether Point Clouds in space belong to the same category. If the distance between Point Clouds is lower than this threshold, they are considered the same category.	5	[0.1, 1000]	mm	Generally does not need to be changed. It should be greater than the point spacing of the workpiece Point Cloud and smaller than the minimum distance between the workpiece Point Cloud and the noise Point Cloud.
Minimum point count threshold	Point Cloud clusters with fewer than this number of points will be filtered out	100	[1,10000000]	/	Generally does not need to be changed. Increase the minimum point count threshold according to the amount of noise in the instance Point Cloud.
Maximum point count threshold	Point Cloud clusters with more than this number of points will be filtered out	100000	[1,10000000]	/	Generally does not need to be changed. If the number of points in the workpiece Point Cloud exceeds 100000, increase the maximum point count threshold.
Select the top Point Cloud in the ROI	When checked, calculates and sorts the average Z coordinate in the ROI coordinate system for Point Clouds of the same category, and retains the Point Cloud category with the largest average Z coordinate (top Point Cloud). When unchecked, retains all Point Clouds that meet the conditions.	Unchecked	/	/	If the workpiece Point Cloud is above the noise Point Cloud, checking this retains the workpiece Point Cloud; if the workpiece Point Cloud is below the noise Point Cloud, checking this also requires adjusting the Z-axis of the ROI coordinate system downward so that the average Z coordinate of the workpiece Point Cloud is the largest, thereby retaining the workpiece Point Cloud.
Visualize process data	When checked, saves the denoised Point Cloud, which can be found in C:_data	Unchecked	/	/	In debugging mode, this can be checked if visualized data needs to be saved

• Example

(1) Without using Point Cloud clustering denoising, the generated instance Point Clouds are shown below. Instances 0, 1, 2, 3, 4, and 5 all contain noise.

(2) Check Point Cloud clustering denoising and run with the default values. If clustering succeeds, categories in the instance Point Cloud whose point counts are greater than 100 and less than 100000 will all be retained, while categories whose point counts are less than 100 or greater than 100000 will be filtered out.

The Point Cloud after clustering denoising can be viewed in the visualization window

(3) Increase Minimum point count threshold to 400. The 2nd and 3rd Point Cloud categories of instance 0 will be filtered out, the 1st Point Cloud category of instance 1 will be filtered out, the Point Cloud of instance 2 will not be filtered, the 3rd Point Cloud category of instance 3 will be filtered out, the Point Cloud of instance 4 will not be filtered, and the 1st Point Cloud category of instance 5 will be filtered out.

(4) Check whether to select the top Point Cloud in the ROI to retain only the Point Cloud category with the largest average Z coordinate in the ROI coordinate system. For instances 0, 1, 2, 3, 4, and 5, only the top Point Cloud will be retained, and the lower-layer noise will all be filtered out.

If the noise is above and the workpiece Point Cloud is below, the Z-axis of the ROI should be adjusted downward in the visualization interface so that the average Z coordinate of the workpiece Point Cloud is the largest and the workpiece Point Cloud is retained; otherwise, the retained top Point Cloud will be noise

(5) If clustering fails, the log error is shown below. In that case, increase the distance threshold for Point Cloud clustering. The distance threshold for Point Cloud clustering should be greater than the point spacing of the workpiece Point Cloud and smaller than the minimum distance between the workpiece Point Cloud and the noise Point Cloud.

Use MeshLab to measure the point spacing of the workpiece Point Cloud in the generated instance Point Cloud and the minimum distance between the workpiece Point Cloud and the noise Point Cloud, as shown below

If the number of points in the workpiece Point Cloud is greater than the maximum point count threshold, the workpiece Point Cloud may be completely filtered out because its point count exceeds 100000, causing clustering to fail. In that case, increase the maximum point count threshold.

---

2.1.2 Point Cloud Downsampling

• Function

Sample the Point Cloud according to the specified point spacing to reduce the number of computed points and improve model inference speed, although accuracy may decrease

• Applicable Scenarios

When the number of Point Cloud points in the actual scenario is too large, check Point Cloud Downsampling.

• Parameter Description

Parameter	Description	Default Value	Parameter Range	Unit
Point spacing for downsampling (mm)	Sample the Point Cloud according to the specified point spacing	5.0	[0.1, 1000]	mm

• Parameter Tuning

  • The larger the value of Point spacing for downsampling, the fewer points remain after downsampling, so Pick Point computation becomes faster, but accuracy may decrease

  • The smaller the value of Point spacing for downsampling, the more points remain after downsampling, so Pick Point computation becomes slower, but accuracy improves

2.1.3 Compute Normal Vectors

• Function

Compute Point Cloud normal vectors for subsequent Point Cloud processing

• Parameter Description

Parameter Name	Description	Default Value	Value Range
Fix normal vector orientation	Whether to fix the orientation when computing normal vectors. When enabled, the orientation of the normal vector is determined by the orientation reference vector.	Checked	/
Neighbor point count for normal vector computation	The larger the value, the more neighboring points are referenced, but local variations may be ignored; the smaller the value, the opposite applies.	30	[1,200]
Orientation reference vector	Orientation reference vector for normal vector computation	[0,0,1]	/

• Parameter Tuning

Cannot be changed

2.1.4 Point Cloud Contour Extraction

• Function

Extract the contour of the workpiece from the instance Point Cloud

• Applicable Scenarios

When using 2.2.4 Enable contour mode, Point Cloud contour extraction should also be checked

• Parameter Description

Parameter Name	Description	Default Value	Value Range	Unit	Tuning Recommendation
Reference radius (mm)	Search radius for extracting contours in the instance Point Cloud	10	[0.1,10000000000]	mm	The reference radius is recommended to be set to 1/2 of the point spacing used for downsampling in 2.1.2Point Cloud Downsampling, and it must be greater than the Point Cloud spacing
Point Cloud contour search mode	Mode for searching Point Cloud contours	Normal mode	Normal mode; plane mode	/	Normally select normal mode; for planar workpieces, select plane mode

• Example

2.1.5 Filter Point Clouds by HSV Color Range (Hue, Saturation, Value)

• Function

Filter Point Clouds according to hue, saturation, and value in the Point Cloud image, and screen out Point Cloud regions that match the target range

• Parameter Description

Parameter Name	Description	Default Value	Value Range
Filter depth by HSV - maximum color range value	Maximum color value for filtering Point Clouds	[0.9,0.9,0.9]	[[0,0,0],[1,1,1]]
Filter depth by HSV - minimum color range value	Minimum color value for filtering Point Clouds	[0.0,0.0,0.0]	[[0,0,0],[1,1,1]]

• Example

2.1.6 Filter Point Clouds by Three-Channel Color

• Function

Filter Point Clouds by three-channel color to screen out Point Cloud regions that match the target range

• Parameter Description

Parameter Name	Description	Default Value	Value Range
Filter Point Clouds by three-channel color - maximum color value	Maximum color value for filtering Point Clouds	[0.9,0.9,0.9]	[[0,0,0],[1,1,1]]
Filter depth by three-channel color - minimum color value	Minimum color value for filtering Point Clouds	[0.0,0.0,0.0]	[[0,0,0],[1,1,1]]

• Example

2.1.7 Select Point Clouds Within the ROI Area

• Function

Select Point Clouds within the ROI 3D area from the instance Point Cloud. This default function cannot be deleted.

• Example

2.1.8 Remove Points Whose Normals Exceed the Angle Threshold

• Function

Remove Point Cloud points whose angle between the normal vector and the axis direction of the standard normal vector is greater than the normal vector angle threshold

• Applicable Scenarios

Planar workpiece loading/unloading (materials are mutually isolated)

• Parameter Description

Parameter Name	Description	Default Value	Value Range	Unit
Angle threshold	Point-cloud points whose normal-vector angle exceeds this threshold are removed.	15	[-360, 360]
Standard normal vector axis direction	The angle formed between the Point Cloud normal vector and the standard normal vector axis direction	Z-axis	X/Y/Z-axis	/
Whether to use the ROI coordinate system	When checked, calculates the angle between the normal vector and the axes of the ROI coordinate system; when unchecked, calculates the angle between the normal vector and the axes of the Camera coordinate system	Unchecked	/	/

• Parameter Tuning

2.1.9 Point Cloud Plane Segmentation

• Function

Retain or remove the plane with the largest number of points in the instance Point Cloud

• Applicable Scenarios

The instance Point Cloud contains a noisy plane

• Parameter Description

Parameter	Description	Default Value	Value Range	Unit	Tuning Recommendation
Reference distance for plane fitting (mm)	If the distance between a point and the plane is lower than the reference distance, the point is considered to be on the plane; otherwise, it is considered to be outside the plane	3	[0.001,10000]	mm	Generally does not need to be changed
Remove plane	When checked, removes the plane with the largest number of points; when unchecked, retains the plane with the largest number of points	Unchecked	/	/	If the plane with the largest number of points is the workpiece, retain the plane and leave this unchecked; if the plane with the largest number of points is noise, remove the plane and check this option

• Example

2.1.10 Point Cloud Outlier Removal

• Function

Identify and remove outlier noise in the Point Cloud to improve Point Cloud quality

• Applicable Scenarios

The instance Point Cloud contains many outlier noise points

• Parameter Description

Parameter Name	Description	Default Value	Value Range
Reference neighbor point count	The number of neighboring points around each point in the Point Cloud, that is, the neighborhood size. For dense Point Clouds, even a small neighborhood is sufficient to reflect the features of the workpiece, so a smaller value can be used; for sparse Point Clouds, a larger neighborhood is needed to reflect the features of the workpiece, so a larger value should be used.	30	[1, 10000000]
Standard deviation multiplier	Used to identify outlier noise. If the deviation of a point's coordinates from the mean coordinates of the instance Point Cloud exceeds the standard deviation multiplier, the point is considered an outlier. The smaller the value, the more points are considered outliers and removed, but this may cause misjudgment and remove important workpiece features; the larger the value, the fewer points are considered outliers and removed, but some outliers may be retained and affect recognition accuracy.	0.005	[0.0001, 2]

• Parameter Tuning

Generally does not need to be changed. If the Point Cloud becomes too sparse after Point Cloud Outlier Removal, the standard deviation multiplier should be increased

• Example

2.1.11 Filter Out Point Clouds Whose Object Distance Exceeds the Limit

• Function

Filter out Point Clouds in the specified direction to remove noise and improve image recognition accuracy

• Parameter Description

Parameter	Description	Default Value	Parameter Range	Unit	Tuning Recommendation
Specified axis	Specified axis of the Point Cloud, used to filter out Point Clouds in the specified direction	Z-axis	X/Y/Z-axis	/	Specified axis generally does not need to be changed
Threshold (mm)	In the direction of the specified axis, if the distance between the lower-layer Point Cloud and the workpiece Point Cloud is greater than this threshold, the lower-layer Point Cloud will be filtered out; if the distance is less than this threshold, the lower-layer Point Cloud will be retained	750	[0, 1000]	mm	Adjust the threshold according to the actual scenario. The larger the threshold, the fewer Point Clouds are filtered out; the smaller the threshold, the more Point Clouds are filtered out.
Select coordinate system	Filter out Point Clouds under the selected coordinate system	ROI coordinate system	Camera coordinate system; ROI coordinate system; object coordinate system	/

• Example

2.1.12 Point Cloud Downsampling (workpiece Point Cloud)

• Function Overview

The scene Point Cloud used for fine matching can be downsampled (different from coarse matching). When using it, it is recommended to add it and move it to the top of the preprocessing function list.

• Applicable Scenarios

Scenarios that use fine matching but have high processing time and cannot meet the overall takt time requirement.

• Parameter Description

Parameter	Description	Default Value	Value Range	Unit
Point spacing for downsampling (mm)	Sample the Point Cloud according to the specified point spacing to reduce the number of points and improve vision computation speed	1	[0.1,1000]	mm

The larger the value, the larger the point spacing for downsampling, the fewer points remain after downsampling, and the faster the vision computation speed, but accuracy may decrease;

The smaller the value, the smaller the point spacing for downsampling, the more points remain after downsampling, and the slower the vision computation speed, but accuracy may improve.

After running, the Point Cloud template in the workpiece configuration can be updated to the downsampled workpiece Point Cloud in the historical data, so as to appropriately reduce coarse matching time.

• Example

2.1.13 [Expert] Read Instance Point Cloud

• Function Overview

Read the instance Point Cloud

• Applicable Scenarios

Planar workpiece scenarios

• Parameter Description

Parameter	Description	Default Value	Value Range	Unit
Point Cloud path	Workpiece Point Cloud path. If not filled in, the Point Cloud uploaded in the workpiece interface is used.	/	/	/

2.1.14 Optimize the Mask According to the Point Cloud

• Function

Based on the Point Cloud within ROI 3D, remove Point Cloud data in the mask that is not within ROI 3D, thereby improving mask accuracy

2.2 Point Cloud Matching Pose Estimation

2.2.1 Model Point Cloud Downsampling Size (mm)

• Function

Before coarse matching, the size of the sampling box used when downsampling the template Point Cloud

• Applicable Scenarios

Planar workpiece ordered loading/unloading, planar workpiece random picking, planar workpiece positioning and assembly, planar workpiece positioning and assembly (matching only)

• Parameter Description

Default value: 5

Value range: [0.001, 500]

Unit: mm

• Parameter Tuning

  • The larger the size, the fewer points are retained during downsampling, and the fewer points remain in the template Point Cloud after downsampling

  It is recommended that the number of points in the template Point Cloud after downsampling be less than 300

  

  

  • If the log reports the error "The number of model points is greater than 1000. Please appropriately increase the model Point Cloud downsampling size parameter in 3D matching", then model Point Cloud downsampling size should be increased to reduce the number of points in the template Point Cloud after downsampling;

2.2.2 Repeated Sampling Ratio Coefficient

• Function

The degree of repeated sampling for point pairs in the processed template Point Cloud

• Applicable Scenarios

Planar workpiece ordered loading/unloading, planar workpiece random picking, planar workpiece positioning and assembly, planar workpiece positioning and assembly (matching only)

• Parameter Description

Default value: 1

Value range: [0.1,1]

• Parameter Tuning

  • The larger the value, the higher the success rate of coarse matching, but the longer the coarse matching time

  

  

  

  • If the log reports the error "Please adjust parameters such as the point spacing for downsampling in 3D matching, model Point Cloud downsampling size, repeated sampling ratio coefficient, etc. For details, refer to the CPFV tuning guide", then Repeated sampling ratio coefficient should be increased. It is generally recommended to start tuning from 1. A value of 1 is the slowest but gives the highest fine matching success rate, while 0.1 is the fastest but gives the poorest fine matching success rate.

  

  • If the log reports the error "Timeout warning triggered, please refer to the CPFV tuning guide to modify the parameters", then Repeated sampling ratio coefficient should be reduced

---

2.2.3 Pose Angle Prior

• Function

Before coarse matching, provide the approximate orientation of the workpiece in the ROI coordinate system in advance to improve the accuracy of coarse matching

• Applicable Scenarios

Planar workpiece ordered loading/unloading, planar workpiece positioning and assembly, and planar workpiece positioning and assembly (matching only), where the rotational symmetry of the planar workpiece itself leads to poor coarse matching and fine matching results

Not applicable to planar workpiece random picking scenarios

• Parameter Description

The value contains 4 digits. The first 3 digits represent the direction of the workpiece, and the 4th digit uses 0/1/2 to represent the X/Y/Z axis of the ROI coordinate system respectively.

Default value: []

Value range: [1,0,0,0] indicates the positive X-axis direction, and [-1,0,0,0] indicates the negative X-axis direction;

[0,1,0,1] indicates the positive Y-axis direction, and [0,-1,0,1] indicates the negative Y-axis direction;

[0,0,1,2] indicates the positive Z-axis direction, and [0,0,-1,2] indicates the negative Z-axis direction.

Format: [[1,0,0,0]], with two square brackets; directions of the X/Y/Z axes can be combined, for example, [[1,0,0,0],[0,1,0,1],[0,0,1,2]] indicates the positive X-axis, positive Y-axis, and positive Z-axis directions

• Example

Determine according to the orientation of the planar workpiece. As shown below, the planar workpiece faces the positive X-axis direction, so Pose Angle Prior can be set to [[1,0,0,0]]

---

2.2.4 Enable Contour Mode

• Function

Extract the contour of the instance Point Cloud and use the downsampled contour of the template Point Cloud and the downsampled contour of the instance Point Cloud for coarse matching

• Applicable Scenarios

Planar workpiece ordered loading/unloading, planar workpiece random picking, planar workpiece positioning and assembly, and planar workpiece positioning and assembly (matching only). Since the contour features of the workpiece are relatively obvious, using contour mode can avoid poor coarse matching caused by the planar workpiece easily slipping.

When using contour mode, 2.1.4 Point Cloud Contour Extraction should also be checked to extract contours from the instance Point Cloud

• Example

  • When contour mode is used, the log will display the prompt "Using contour mode, found number of contour points"

  

Contour Mode

• Function

Extract the contour of the template Point Cloud and use the downsampled contour of the template Point Cloud and the downsampled contour of the instance Point Cloud for coarse matching

• Applicable Scenarios

Planar workpiece ordered loading/unloading, planar workpiece random picking, planar workpiece positioning and assembly, and planar workpiece positioning and assembly (matching only). If the coarse matching result is poor, check this function to use contour Point Clouds for coarse matching again.

• Parameter Tuning

Normal mode: default

Plane mode: for planar workpieces, select plane mode

• Example

2.2.5 Search Radius for Coarse Registration Contours

• Function

Use the downsampled contour of the template Point Cloud and the downsampled contour of the instance Point Cloud for coarse matching, and set the search radius for extracting contour Point Clouds from the template Point Cloud and instance Point Cloud

• Applicable Scenarios

Planar workpiece ordered loading/unloading, planar workpiece random picking, planar workpiece positioning and assembly, planar workpiece positioning and assembly (matching only)

• Parameter Description

Default value: 0.005

Value range: [0.0001,0.5]

Unit: m

2.2.6 Object Pose Correction

Fine Matching Search Radius (mm)

• Function

During fine matching, the template Point Cloud is matched with the instance Point Cloud, and every point in the template Point Cloud needs to search for its nearest point in the instance Point Cloud. The fine matching search radius represents both the search radius in the instance Point Cloud and the distance threshold between each point in the template Point Cloud and the nearest point in the instance Point Cloud. If the distance between a point and its nearest point is less than the fine matching search radius, the two points are considered matchable; otherwise, they are considered not matchable.

• Applicable Scenarios

Planar workpiece ordered loading/unloading, planar workpiece random picking, and planar workpiece positioning and assembly scenarios

• Parameter Description

Default value: 10

Value range: [1, 500]

Unit: mm

• Parameter Tuning

Usually does not need to be changed

Fine Matching Search Mode

• Function

The way the template Point Cloud searches for nearest points in the instance Point Cloud during fine matching

• Applicable Scenarios

If the fine matching result between the template Point Cloud and the instance Point Cloud is poor, this function should be adjusted

• Parameter Description

Parameter	Description
Point-to-point	Each point in the template Point Cloud searches for its nearest point in the instance Point Cloud (the point with the shortest straight-line distance within the search radius), which is suitable for all workpieces
Point-to-plane	Each point in the template Point Cloud searches for its nearest point in the instance Point Cloud along its normal vector, which is suitable for workpieces with obvious geometric features
Combination of point-to-point and point-to-plane	First use point-to-point mode to optimize the workpiece pose in the instance Point Cloud, and then use point-to-plane mode to optimize the workpiece pose in the instance Point Cloud. This is suitable for workpieces with obvious geometric features. Using this method will increase takt time

---

Use Contour Mode

• Function

Extract contour Point Clouds from the template Point Cloud and the instance Point Cloud for coarse matching

• Applicable Scenarios

In planar workpiece ordered loading/unloading, planar workpiece random picking, and planar workpiece positioning and assembly scenarios, if the result of coarse matching using keypoints is poor, this function should be checked to use contour Point Clouds for coarse matching again

• Parameter Tuning

The coarse matching result affects the fine matching result. If the fine matching result is poor, you can check Use Contour Mode

Contour Search Range (mm)

• Function

Search radius for extracting contour Point Clouds from the template Point Cloud and the instance Point Cloud

• Applicable Scenarios

General workpiece ordered loading/unloading, general workpiece random picking, and general workpiece positioning and assembly scenarios

• Parameter Description

Default value: 5

Value range: [0.1, 500]

Unit: mm

• Parameter Tuning

If the value is smaller, the radius for searching contour Point Clouds is smaller, which is suitable for extracting fine workpiece contours, but the extracted contours may contain outlier noise;

If the value is larger, the radius for searching contour Point Clouds is larger, which is suitable for extracting wider workpiece contours, but the extracted contours may ignore some detailed features.

Save Pose Estimation [Fine Matching] Data

• Function

When checked, saves fine matching data

• Applicable Scenarios

Planar workpiece ordered loading/unloading, planar workpiece random picking, planar workpiece positioning and assembly, planar workpiece positioning and assembly (matching only)

• Example

Fine matching data is saved in the project save path \ProjectFolder\data\PickLight\HistoricalDataTimestamp\Builder\pose\output.

2.2.7 Pose Adjustment Based on Axis Rotation

• Function

Rotate and translate the instance Point Cloud around the given axis based on the first Pick Point of the workpiece, calculate the matching score between the template Point Cloud and the instance Point Cloud after each rotation and translation, and select the instance Point Cloud with the highest matching score as the final pose of the rotationally symmetric workpiece.

• Applicable Scenarios

Deviation occurs when matching the instance Point Cloud of a rotationally symmetric workpiece with the template Point Cloud, and the Point Cloud can be fully matched only after rotating by a certain angle

Cannot be used together with Recognition Type, front/back recognition (via Point Cloud template), or local feature recognition in the function options

• Parameter Description

Parameter	Description	Default Value	Parameter Range	Tuning Recommendation
Rotation angle interval	Rotate the instance Point Cloud by equal angular intervals. The difference between two adjacent rotations is the rotation angle interval. For example, if the first rotation is 30°, the second is 60°, and the third is 90°, then the rotation angle interval is 30°.	5	\[1, 180\]	If the workpiece has many features and is difficult to match and high-precision matching is required, a smaller angle interval can be set to perform more matching operations, but the computation load will increase. If the workpiece shape is simple and has fewer features, a larger angle interval can be set to improve computation efficiency.
Rotation angle range	How large an angle range the instance Point Cloud can rotate from its initial state.	90	\[1, 180\]	The Point Cloud rotation range is the positive and negative interval of the configured value. For example, if the angle range is set to 120°, the Point Cloud rotation range is \[-120°,120°\], and the instance Point Cloud will rotate from -120° to 120° at the specified angle interval. Number of rotations = rotation angle range / rotation angle interval. The rotation angle range is generally set to 180, 120, or 90, according to the characteristic angle of the rotationally symmetric workpiece. For example, if an equilateral triangular workpiece can overlap itself after rotating by 120° , then the rotation angle range should be ≥120°. If it is set to 180°, more rotations will be performed and more rotated instance Point Clouds will be matched with the template Point Cloud. For example, if a regular quadrilateral workpiece can overlap itself after rotating by 90°, then the rotation angle range should be ≥90°. If Pose Angle Prior has been filled in and the approximate orientation of the workpiece is already known, the rotation angle range can be reduced according to the possible deviation range to reduce computation.
Template Point Cloud file path	Upload the template Point Cloud file of the workpiece. If not uploaded, the Point Cloud template uploaded in the workpiece interface is used.	/	/
Evaluation mode	Evaluate the quality of the matching result from different perspectives	Iso-target	Iso-target; Iso-source; Average; Strict; Loose; Fast	Iso-target: evaluate the quality of the matching result from the perspective of the template Point Cloud, checking whether the features of the template Point Cloud are well matched. Iso-source: evaluate the quality of the matching result from the perspective of the instance Point Cloud, checking whether the changes in the instance Point Cloud during matching and the features of the matched instance Point Cloud are reasonable. Average: comprehensively considers factors such as the error and fitting degree of the template Point Cloud and the instance Point Cloud during matching to provide an overall evaluation result. Strict: uses a relatively strict method to evaluate the matching result. Only when the Point Clouds have a very high matching degree will the result be considered good. Suitable for scenarios with many workpiece features and high matching precision requirements, such as precision parts. Loose: uses a relatively loose evaluation method and allows a certain range of matching errors. Suitable for scenarios with lower matching precision requirements where faster matching results are preferred. Fast: emphasizes quick evaluation and is suitable for scenarios with high requirements for matching speed and low requirements for matching accuracy.
ICP threshold	The criterion for determining whether registration is successful. If the error of the matching result is less than this threshold, the matching is successful; if the error is greater than this threshold, the matching is unsuccessful.	0.005	\[0.000001, 1\]	Generally does not need to be changed. If the actual scenario requires higher matching precision, reduce this threshold; if the actual scenario requires higher matching speed and lower precision, increase this threshold. If the workpiece Point Cloud quality is good, the threshold can be reduced; if the workpiece Point Cloud quality is poor, the threshold can be increased.
Rotation axis selection	The axis around which the instance Point Cloud rotates	Z-axis	X/Y/Z-axis	Generally does not need to be changed. If one axis of the rotationally symmetric workpiece is critical for recognizing the workpiece pose, that feature axis can be used as the rotation axis.
Save visualization data	Whether to save visualization data	Unchecked	/
Use edges for optimization	Use the edge contour of the workpiece to optimize matching, reduce matching result error, and make the matching between the instance Point Cloud and the template Point Cloud finer	Unchecked	/	If the edge contour of the workpiece has unique geometric features, check this function to improve matching accuracy. For example, for workpieces with complex shapes and large differences in edge contours, using the edge contour Point Cloud can identify the workpiece pose more accurately.
Edge Point Cloud file path	Upload the edge Point Cloud file of the workpiece. If not uploaded, the edge Point Cloud is extracted from the Point Cloud template uploaded in the workpiece interface.	/	/
Optimize rotation result	After finding the best pose during matching, optimize it again to reduce matching result error and make the matching between the instance Point Cloud and the template Point Cloud finer	Checked	/	Checked by default to improve matching accuracy and generally does not need to be changed
Optimization mode	Mode for optimizing the matching result	Point	Point; Plane; Full	Point (point mode): optimization based on correspondences between points. It calculates point-to-point distance errors and iteratively adjusts them. Suitable for workpieces with simple Point Clouds and few features. Plane (plane mode): considers the planar features of the Point Cloud and incorporates point-to-plane relationships into registration computation. Suitable for workpieces with many planes, such as plate-like workpieces. Full (full mode): comprehensively considers various geometric features such as points, lines, and planes for registration optimization, making fuller use of Point Cloud information, but with relatively higher computational cost. Suitable for workpieces with complex Point Clouds, rich geometric features, and high precision requirements.
Optimization threshold	The criterion for determining whether registration has reached the expected accuracy during optimization. If the registration error is less than this threshold, optimization succeeds; if the registration error is greater than this threshold, optimization fails and iteration must continue.	0.002	\[0.0001, 1\]	Generally does not need to be changed. Reduce this threshold for scenarios with high registration precision requirements; increase this threshold for scenarios with lower registration precision requirements.
Enable center movement mode	When enabled, translation mode is also added, so translation is superimposed while rotating the instance Point Cloud	Unchecked		If there is axial offset in the registration result, center movement mode can be enabled to optimize the registration result Note that enabling center movement mode requires the evaluation mode to be set to Fast
x-axis movement range	Movement range along the Pick Point x-axis in center movement mode (mm)	0	\[0,100\]	The default value is 0, which means the x-axis does not move during evaluation; a non-zero value means x-axis movement is enabled Note that the specified axis direction refers to the direction before Pick Point direction adjustment. Functions in Pick Point processing may change the axis direction. You can disable Pick Point processing functions to view the correct axis direction.
x-axis movement step size	Movement step size along the Pick Point x-axis in center movement mode (mm)	2	\[0.01,10\]	The step size must be set reasonably according to offset conditions and accuracy requirements. A step size that is too small will greatly increase takt time.
y-axis movement range	Movement range along the Pick Point y-axis in center movement mode (mm)	0	\[0,100\]	The default value is 0, which means the y-axis does not move during evaluation; a non-zero value means y-axis movement is enabled Note that the specified axis direction refers to the direction before Pick Point direction adjustment. Functions in Pick Point processing may change the axis direction. You can disable Pick Point processing functions to view the correct axis direction.
y-axis movement step size	Movement step size along the Pick Point y-axis in center movement mode (mm)	2	\[0.01,10\]	The step size must be set reasonably according to offset conditions and accuracy requirements. A step size that is too small will greatly increase takt time.
z-axis movement range	Movement range along the Pick Point z-axis in center movement mode (mm)	0	\[0,100\]	The default value is 0, which means the z-axis does not move during evaluation; a non-zero value means z-axis movement is enabled Note that the specified axis direction refers to the direction before Pick Point direction adjustment. Functions in Pick Point processing may change the axis direction. You can disable Pick Point processing functions to view the correct axis direction.
z-axis movement step size	Movement step size along the Pick Point z-axis in center movement mode (mm)	2	\[0.01,10\]	The step size must be set reasonably according to offset conditions and accuracy requirements. A step size that is too small will greatly increase takt time.

• Example

Axial matching offset

After enabling center movement mode

• Description of the number of poses evaluated in pose adjustment based on axis rotation

Total number of pose computations = number of x-axis movement points * number of y-axis movement points * number of z-axis movement points * number of angle points

An excessively large movement range or an excessively small step size will increase the total number of poses that need to be evaluated and lengthen takt time. The range and step size parameters must be set reasonably.

2.3 Empty ROI Determination

• Function

Determine whether any workpieces (Point Clouds) remain within ROI 3D. If the number of 3D points in ROI 3D is less than this value, it indicates that no workpiece Point Cloud remains, and in this case no Point Cloud is returned

• Parameter Description

Default value: 1000

Value range: [0, 100000]

• Usage Workflow

Set the minimum point-count threshold for ROI 3D. If it is lower than this threshold, the workpiece Point Cloud in ROI 3D is insufficient, and it is therefore determined that there is no workpiece in ROI 3D;

In the Robot configuration, add a new visual status code to facilitate subsequent Robot signal processing.

3. Pick Point Processing

Previous pageGeneral Target Object Visual Parameter Tuning Guide

Next pageSurface Target Object Loading/Unloading Visual Parameter Tuning Guide (Mutually Isolated Materials)