General Target Object Visual Parameter Tuning Guide

About 26584 wordsAbout 89 min

This chapter explains how to adjust visual parameters for General Target Object ordered loading/unloading, random picking, and pose-based locating/assembly according to real-world scenarios.

(1) Create a General Target Object ordered loading/unloading, General Target Object random picking, or General Target Object locating/assembly project (project name and path may be customized; project name must not contain Chinese characters)

Target Object type: General Target Object (not circle, cylinder, or quadrilateral, and with significant front/back differences)

(2) Configure camera and robot

(3) Add Target Object

Target Object information

Target Object name may be customized. Target Object type defaults to Standard Target Object and cannot be changed. Target Object ID can be customized to enable automatic Target Object switching during robot picking.

Point Cloud file: Target Object Point Cloud, created in Point Cloud Template Creation

Fine-matching point cloud template: used for fine matching

Custom keypoints: General Target Object requires uploading a keypoint file for the Target Object

Camera parameters: not required

Model information

Visual Model: the 2D recognition solution for General Target Object uses CAD-based synthetic data training (One-Click Linking), and the 3D matching solution uses Rigid Transform. Visual Models for different General Target Object applications must be obtained via One-Click Linking training.

Mesh file: usually upload the Target Object CAD. To remove noise, a standardized mesh file is required — this can also be produced in Point Cloud Template Creation.

Target Object attributes: elongated, symmetric, highly reflective, low solidity

Work environment: provide an environment file; the environment generated in One-Click Linking will be automatically replaced by the provided environment to improve recognition

Target Object texture: provide Target Object texture; One-Click Linking will use this texture for data augmentation during model training to improve recognition

Model maximum recognition count: default 20, change according to scenario

Pick Points: set Pick Points based on the Target Object

Absolute coordinate system: origin is the initial point; the initial point comes from the Target Object point cloud and CAD.

Pick Point coordinate system (offset): origin is the current Pick Point.

(4) Add Tools, perform eye-hand calibration, and configure ROI

(5) Optional functional options: Instance Optimization, Collision Detection, Collision Detection (New), Visual Classification, Recognition Types, Face Recognition (via point cloud templates), Local Feature Recognition

Instance Optimization: post-process model-generated instances and their masks.

Collision Detection (New): used to detect collisions between the Tool and the container, filtering Picking Poses that may collide. Collision Detection Usage Guide

Visual Classification: used to distinguish different textures or orientations of the same Target Object. Visual Classification Usage Guide

Recognition Types: used to recognize multiple Target Object types. Recognition Types Usage Guide

Local Feature Recognition: locate Target Objects using local features. Local Feature Recognition Usage Guide

Face Recognition (via point cloud templates): import point cloud templates for the Target Object front and back to match front or back faces of picked Target Objects, and set Pick Points separately for front/back. Face Recognition (via point cloud templates) Usage Guide

(6) Test data (Historical Data is provided for subsequent practice; you may use the {=tex}\input folder's 2D images and 3D point clouds from Historical Data's foreground to substitute camera captures when configuring ROI)

General Target Object ordered loading/unloading data:

Point Cloud file:

Keypoint file:

Mesh file:

Visual Model:

Tool:

Historical Data:

Visual Parameters

2D recognition: identify and segment instances from real scenes

Preprocessing: operate on 2D images before Instance Segmentation (common ops: fill depth holes & edge enhancement & extract top-layer texture & remove image background outside ROI3D)

Instance Segmentation: segment instances (scale & Confidence lower bound threshold & auto-augment). Acceleration can be disabled. Returns masks

Point Cloud generation: ways to generate instance Point Clouds — use segmented instance masks or bounding boxes / use filtered instance masks or bounding boxes

Instance filtering: filter segmented instances

Instance sorting: sort instances

3D computation: compute instance Pose in the camera coordinate system and generate Pick Points

Preprocessing: preprocess 3D point clouds before computing Pick Points

Pose estimation: compute instance Pose in the camera coordinate system (coarse match, fine match) and generate Pick Points

Pick Point processing: filter, adjust, and sort Pick Points

Pick Point filtering: filter Pick Points

Pick Point adjustment: adjust Pick Points

Pick Point sorting: sort Pick Points

1. 2D recognition

This section explains and provides tuning suggestions for functions that affect 2D image recognition results: preprocessing, Instance Segmentation, instance filtering, and instance sorting.

1.1 Preprocessing

Preprocessing for 2D recognition operates on 2D images before Instance Segmentation.

1.1.1 Bilateral Filter

Function

Image smoothing based on bilateral filtering

Parameter Description

Parameter	Description	Default Value	Range
Maximum depth difference	Maximum depth difference for bilateral filtering	0.03	[0.01, 1]
Filter kernel size	Convolution kernel size for bilateral filtering	7	[1, 3000]

1.1.2 Depth-to-Normal Map

Function

Compute per-pixel normals from the depth map and convert the image into a normal map

1.1.3 Image Enhancement

Function

Common image enhancements such as saturation, contrast, brightness, and sharpness

Parameter Description

Parameter	Description	Default Value	Range
Enhancement type	Which image element to enhance	Contrast	Saturation, Contrast, Brightness, Sharpness
Enhancement threshold	How much to enhance the chosen image element	1.5	[0.1, 100]

1.1.4 Histogram Equalization

Function

Improve image contrast

Parameter Description

Parameter	Description	Default Value	Range
Local mode	Local or global histogram equalization. Checked = local; unchecked = global	Checked	/
Contrast threshold	Contrast threshold	3	[1,1000]

1.1.5 Filter Depth Map by Color

Function

Filter the depth map based on color values

Parameter Description

Parameter	Description	Default Value	Value Range
Fill kernel size	Size of the color fill	3	[1,99]
HSV-based depth filter - maximum color value	Maximum color value	[180,255,255]	[[0,0,0],[255,255,255]]
HSV-based depth filter - minimum color value	Minimum color value	[0,0,0]	[[0,0,0],[255,255,255]]
Preserve regions within the color range	Checked: preserve regions within the color range; unchecked: preserve regions outside the color range	/	/

1.1.6 Gamma Image Correction

Function

Gamma correction changes image brightness

Parameter Description

Parameter	Description	Default Value	Value Range
Gamma compensation factor	If this value is less than 1, the image becomes darker; if it is greater than 1, the image becomes brighter	1	[0.1,100]
Gamma correction factor	If this value is less than 1, the image becomes darker, which is suitable for images that are too bright; if it is greater than 1, the image becomes brighter, which is suitable for images that are too dark	2.2	[0.1,100]

1.1.7 Fill Holes in the Depth Map

Function

Fill hole regions in the depth map and smooth the filled depth map

Use Case

Part of the target object may be missing in the depth map due to structural occlusion of the target object itself, uneven lighting, and similar issues

Parameter Description

Parameter	Description	Default Value	Value Range
Fill kernel size	Size of hole filling	3	[1,99]

The fill kernel size can only be an odd number

Parameter Tuning

Adjust according to the detection results. If overfilling occurs, reduce the Parameter; if filling is insufficient, increase the Parameter

Example

1.1.8 Edge Enhancement

Function

Set edge regions of textures in the image to the background color or to a color with strong contrast from the background color, so the edge information of the target object stands out

Use Case

The edges are unclear because target objects occlude or overlap each other

Parameter Description

Parameter	Description	Default Value	Value Range	Tuning Suggestion
Normal Z-direction filtering threshold	Angle filtering threshold between the normal vector corresponding to each point in the depth map and the positive Z-axis direction of the camera coordinate system. If the angle between a point's normal vector and the positive Z-axis direction of the camera coordinate system is greater than this threshold, the color at the corresponding position in the 2D image will be set to the Background Color or to a color with strong contrast from the Background Color	30	[0,180]	For flat target object surfaces, this threshold can be smaller. For curved target objects, increase it appropriately according to the surface tilt
Background Color	RGB color threshold of the background color	128	[0,255]
Auto-adjust contrast background	Checked After enabling Auto-adjust contrast background, the colors of points in the 2D image whose angles are greater than the filtering threshold are set to a color with strong contrast from the Background ColorUnchecked After disabling Auto-adjust contrast background, the colors of points in the 2D image whose angles are greater than the filtering threshold are set to the color corresponding to the Background Color	Unchecked	/

Example

1.1.9 Extract the Topmost Texture

Function

Extract the texture of the topmost or bottommost target objects, and set other regions to the background color or to a color with strong contrast from the background color.

Use Case

Poor lighting conditions, similar color textures, tight stacking, staggered stacking, or occlusion may make it difficult for the model to distinguish the texture differences between upper and lower target objects, which can easily lead to incorrect detections.

Parameter Description

Parameter	Description	Default Value	Value Range	Unit	Tuning Suggestion
Distance threshold (mm)	If the distance from a point to the topmost plane (or bottommost plane) is lower than this threshold, it is considered a point within the topmost plane (or bottommost plane) and should be preserved. Otherwise, it is considered a lower-layer (or upper-layer) point, and its color is set to the background color or to a color with strong contrast from the background color	50	[0.1, 1000]	mm	Generally set to 1/2 of the target object height
Cluster Point Cloud count	Expected number of points participating in clustering, that is, the number of sampled Point Cloud points within the ROI 3D area	10000	[1,10000000]	/	The larger the Cluster Point Cloud count, the slower the model inference but the higher the accuracy; the smaller the Cluster Point Cloud count, the faster the model inference but the lower the accuracy
Minimum category point count	Minimum number of points used to filter categories	1000	[1, 10000000]	/	/
Automatically compute contrast background	Checked After enabling Automatically compute contrast background, regions outside the topmost (or bottommost) layer in the 2D image are set to a color with strong contrast from the background color thresholdUnchecked After disabling Automatically compute contrast background, regions outside the topmost (or bottommost) layer in the 2D image are set to the color corresponding to the background color threshold	Checked	/	/	/
Background color threshold	RGB color threshold of the background color	128	[0, 255]	/	/

Example

1.1.10 Remove Image Background Outside ROI 3D

Function

Remove the background outside the ROI 3D area in the 2D image

Use Case

Too much background noise in the image affects the detection results

Parameter Description

Parameter Name	Description	Default Value	Value Range
Fill kernel size	Size of hole filling	5	[1,99]
Iteration count	Number of image dilation iterations	1	[1,99]
Automatically compute contrast background	Checked After enabling Automatically compute contrast background, regions outside the ROI in the 2D image are set to a color with strong contrast from the background color thresholdUnchecked After disabling Automatically compute contrast background, regions outside the ROI in the 2D image are set to the color corresponding to the background color threshold	Checked	/
Background color threshold	RGB color threshold of the background color	128	[0,255]

The fill kernel size can only be an odd number

Tuning

If you need to remove more background noise from the image, reduce the Fill kernel size

Example

1.2 Instance Segmentation

1.2.1 Scale Ratio

Function

Improve the accuracy and recall of 2D recognition by proportionally scaling the original image before inference.

Use Case

Adjust this function when the detection results are poor (for example, no instance is detected, there are missed detections, one bounding box covers multiple instances, or a bounding box does not fully cover an instance).

Parameter Description

Default Value: 1.0

Value Range: [0.01, 3.00]

Step Size: 0.01

Tuning
- Run with the default value and inspect the detection results in the visualization window. If no instance is detected, there are missed detections, one bounding box covers multiple instances, or a bounding box does not fully cover an instance, adjust this function.

In 2D recognition, the percentage displayed on an instance is the Confidence score, and the number is the instance ID (the recognition order of the instance).
In 2D recognition, the colored shading on an instance is the Mask, the rectangle around the instance is the bounding box, and the colored points are keypoints.

As shown below, when the Scale Ratio is 0.83, the detection results improve significantly, so 0.83 can be used as the lower limit of the Scale Ratio range

When the Scale Ratio is 1.5, the detection results degrade significantly, so 1.5 can be used as the upper limit of the Scale Ratio range

1.2.2 Confidence Lower Threshold

Function

Keep only recognition results whose Deep Learning model scores are higher than the Confidence lower threshold

Use Case

Adjust this function when the instances selected in the detection results do not match expectations

Parameter Description

Default Value: 0.5

Value Range: [0.01, 1.00]

Tuning
- If the model detects too few instances, reduce this threshold. If the value is too small, image recognition accuracy may be affected.

If a Confidence lower threshold that is too small causes incorrect instances to be detected and these incorrect instances need to be removed, increase this threshold. If the value is too large, the number of retained detection results may become zero and no results will be output.

1.2.3 Enable Auto Enhancement

Function

Combine all input Scale Ratios and rotation angles for inference, and return all results above the configured Confidence lower threshold after combination. This can improve model inference accuracy, but it also increases processing time.

Use Case

A single Scale Ratio cannot meet the requirements of the actual scenario, causing incomplete detection, or the objects are placed with a large tilt.

Example

If Auto Enhancement - Scale Ratio is set to [0.8, 0.9, 1.0] and Auto Enhancement - Rotation Angle is set to [0, 90.0] then the values in the Scale Ratio and rotation angle lists are paired in all combinations. The model automatically generates 6 images for inference, merges the 6 inference results, and outputs the results greater than the Confidence lower threshold.

Auto Enhancement - Scale Ratio

Function

Scale the original image multiple times and run inference multiple times, then output the aggregated inference results

Use Case

A single Scale Ratio cannot meet actual scenario requirements, resulting in incomplete detection

Parameter Description

Default Value: [1.0]

Value Range: Each Scale Ratio must be within [0.1, 3.0]

You can set multiple Scale Ratios, separated by English commas

Tuning

Enter multiple Scale Ratios from 1.2.1 Scale Ratio that produced good detection results

Auto Enhancement - Rotation Angle

Function

Rotate the original image multiple times and run inference multiple times, then output the aggregated inference results

Use Case

Use this when the object placement deviates significantly from the coordinate axes

Parameter Description

Default Value: [0.0]

Value Range: Each rotation angle must be within [0, 360]

You can set multiple rotation angles, separated by English commas

Tuning

Adjust Auto Enhancement - Rotation Angle according to the object angles in the actual scenario. The tilt angle can be judged from sack patterns and bag opening shapes, or from carton edges and brand logos

1.3 Point Cloud Generation

Instance Point Cloud generation method	Mask form (after segmentation)	—	Use the segmented instance Mask to generate the Point Cloud
	Bounding box form (after segmentation)	Bounding box scale ratio (after segmentation)	Use the segmented instance bounding box to generate the Point Cloud
	Bounding box form (after segmentation)	Whether color is required when generating the Point Cloud (after segmentation)	Whether the generated instance Point Cloud needs color attached
	Mask form (after filtering)	—	Use the filtered instance Mask to generate the Point Cloud
	Bounding box form (after filtering)	Bounding box scale ratio (after filtering)	Use the filtered instance bounding box to generate the Point Cloud
	Bounding box form (after filtering)	Whether color is required when generating the Point Cloud (after filtering)	Whether the generated instance Point Cloud needs color attached

If acceleration is not required, you do not need to use the Instance Filtering function. Use Mask form (after segmentation) or Bounding box form (after segmentation) to generate the instance Point Cloud. You can view the generated instance Point Cloud in the project storage folder \ProjectName\data\PickLight\HistoricalDataTimestamp\Builder\pose\input folder.

If acceleration is required, you can use the Instance Filtering function to filter instances. Use Mask form (after filtering) or Bounding box form (after filtering) to generate the instance Point Cloud. You can view the generated instance Point Cloud in the project storage folder \ProjectName\data\PickLight\HistoricalDataTimestamp\Builder\pose\input folder.

1.4 Instance Filtering

1.4.1 Filter Based on Bounding Box Area

Function Introduction

Filter based on the pixel area of the bounding boxes of detected instances.

Usage Scenario

Applicable to scenarios where the bounding box areas of instances differ greatly. By setting the upper and lower limits of the bounding box area, noise in the image can be filtered out, improving image recognition accuracy and avoiding additional time consumption in subsequent processing caused by noise.

Parameter Description

Parameter	Description	Default Value	Parameter Range	Unit
Minimum Area (Pixels)	This Parameter is used to set the minimum filtering area of the bounding box. Instances whose bounding box area is lower than this value will be filtered out	1	[1, 10000000]	Pixels
Maximum Area (Pixels)	This Parameter is used to set the maximum filtering area of the bounding box. Instances whose bounding box area is higher than this value will be filtered out	10000000	[2, 10000000]	Pixels

Example

Run with the default values to view the bounding box area of each instance in the log, as shown below.

Adjust Minimum Area and Maximum Area according to the bounding box area of each instance. For example, set Minimum Area to 20000 and Maximum Area to 30000 to filter out instances whose pixel area is smaller than 20000 or larger than 30000. The instance filtering process can be viewed in the log.

1.4.2 Filter Based on Bounding Box Aspect Ratio

Function Introduction

Instances whose bounding box aspect ratio is outside the specified range will be filtered out

Usage Scenario

Applicable to scenarios where instance bounding box aspect ratios differ greatly

Parameter Description

Parameter	Description	Default Value	Parameter Range
Minimum Aspect Ratio	The minimum value of the bounding box aspect ratio. Instances whose bounding box aspect ratio is lower than this value will be filtered out	0	[0, 10000000]
Maximum Aspect Ratio	The maximum value of the bounding box aspect ratio. Instances whose bounding box aspect ratio is higher than this value will be filtered out	10000000	[0, 10000000]
Use X/Y Axis Side Length as Aspect Ratio	By default, this is unchecked, and the ratio of the longer side to the shorter side of the bounding box is used as the aspect ratio, which is suitable when the lengths of the long and short sides of the bounding box differ greatly; after checking it, the ratio of the side length of the bounding box on the X-axis to that on the Y-axis in the pixel coordinate system is used as the aspect ratio, which is suitable for cases where the long-side/short-side ratio of most normal instance bounding boxes is similar, but the ratio of the length on the X-axis to the length on the Y-axis of some abnormally recognized instance bounding boxes differs greatly.	Unchecked	/

1.4.3 Filter Instances Based on Class ID

Function Introduction

Filter instances by class

Usage Scenario

Applicable to scenarios where incoming materials contain multiple types of target objects

Parameter Description

Parameter	Description	Default Value
Class IDs to Keep	Retain instances whose class IDs are in the list; instances whose class IDs are not in the list will be filtered out	[0]

Example

1.4.4 Filter Based on the Side Length of the Instance Point Cloud

Function Introduction

Filter based on the long side and short side of the instance Point Cloud

Usage Scenario

Applicable to scenarios where the distances of the instance Point Cloud on the x-axis or y-axis differ greatly. By setting the distance range of the instance Point Cloud, noise in the image can be filtered out, improving image recognition accuracy and avoiding additional time consumption in subsequent processing caused by noise.

Parameter Description

Parameter	Description	Default Value	Parameter Range	Unit
Short Side Length Range (mm)	The side length range of the short side of the Point Cloud	[0, 10000]	[0, 10000]	mm
Long Side Length Range (mm)	The side length range of the long side of the Point Cloud	[0, 10000]	[0, 10000]	mm
Lower Edge Denoising Limit (%)	Extract the lower percentage limit of the X/Y values (Camera coordinate system) in the instance Point Cloud, and remove the Point Clouds outside the upper and lower limits to avoid noise affecting length calculation	5	[0, 100]	/
Upper Edge Denoising Limit (%)	Extract the upper percentage limit of the X/Y values (Camera coordinate system) in the instance Point Cloud, and remove the Point Clouds outside the upper and lower limits to avoid noise affecting length calculation	95	[0, 100]	/
Side Length Type	Filter according to the long side and short side of the instance Point Cloud. Instances whose long-side or short-side lengths are not within the range will be filtered out	Short Side of Instance Point Cloud	Short side of instance Point Cloud; long side of instance Point Cloud; long and short sides of instance Point Cloud	/

Example

1.4.5 Filter Based on Class ID from the Classifier

Function Introduction

Filter instances based on classifier class ID. Instances not in the reference classes will be filtered out.

Usage Scenario

In multi-class target object scenarios, the vision model may detect multiple types of target objects, but the actual task may only require one specific class. In this case, this function can be used to filter out unnecessary target objects

Parameter Description

The default value is [0], which means instances with class ID 0 are retained by default. Instances whose class IDs are not in the list will be filtered out.

1.4.6 Filter Based on Three-Channel Colors

Function Introduction

Instances can be filtered out by three-channel color thresholds (HSV or RGB).

Usage Scenario

Situations where incorrect instances and correct instances are clearly distinguishable by color.

Parameter Description

Parameter	Description	Default Value	Value Range
Maximum Color Range Value	Maximum color value	[180,255,255]	[[0,0,0],[255,255,255]]
Minimum Color Range Value	Minimum color value	[0,0,0]	[[0,0,0],[255,255,255]]
Filtering Percentage Threshold	Color pass-rate threshold	0.05	[0,1]
Reverse Filtering	Checked means remove instances whose proportion outside the color range is lower than the threshold; unchecked means remove instances whose proportion within the color range in the instance image is lower than the threshold	Unchecked	/
Color Mode	The color space selected in color filtering	HSV Color Space	RGB Color SpaceHSV Color Space

Example

1.4.7 Filter Based on Confidence

Function Introduction

Filter according to the instance Confidence score

Usage Scenario

Applicable to scenarios where the Confidence values of instances differ greatly

Parameter Description

Parameter	Description	Default Value	Value Range
Reference Confidence	Retain instances whose Confidence is greater than the threshold, and filter out instances whose Confidence is lower than the threshold.	0.5	[0,1]
Invert filtering result	After inversion, retain instances whose Confidence is lower than the threshold, and filter out instances whose Confidence is greater than the threshold.	Unchecked	/

Example

1.4.8 Filter by Point Cloud Count

Function Introduction

Filter based on the downsampled instance Point Cloud count

Usage Scenario

The instance Point Cloud contains a large amount of noise

Parameter Description

Parameter	Description	Default Value	Value Range
Minimum Point Cloud Count	Minimum value of the Point Cloud count	3500	[1, 10000000]
Maximum Point Cloud Count	Maximum value of the Point Cloud count	8500	[2, 10000000]
Filter instances whose count is within the interval	Checked: filter instances whose Point Cloud count is between the minimum and maximum values. Unchecked: filter instances whose Point Cloud count is outside the interval	Unchecked	/

1.4.9 Filter by Mask Area

Function Introduction

Filter image Masks based on the sum of Mask pixels of detected instances (that is, the pixel area).

Usage Scenario

Applicable to scenarios where instance Mask areas differ greatly. By setting upper and lower limits for Mask area, noise in image Masks can be filtered out, image recognition accuracy can be improved, and extra processing time caused by noise can be avoided.

Parameter Description

Parameter Name	Description	Default Value	Value Range	Unit
Reference minimum area	This Parameter is used to set the minimum filtering area of the Mask. Instances whose Mask area is lower than this value will be filtered out	1	[1, 10000000]	pixels
Reference maximum area	This Parameter is used to set the maximum filtering area of the Mask. Instances whose Mask area is higher than this value will be filtered out	10000000	[2, 10000000]	pixels

Example

1.4.10 Filter by Visibility

Function Introduction

Filter based on the visibility score of the instance

Usage Scenario

Applicable to scenarios where the visibility scores of instances differ greatly

Parameter Description

Parameter	Description	Default Value	Value Range
Reference visibility threshold	Retain instances whose visibility is greater than the threshold, and filter out instances whose visibility is lower than the threshold. Visibility is used to determine how visible an instance is in the image. The more the target object is occluded, the lower the visibility.	0.5	[0,1]
Invert filtering result	After inversion, retain instances whose visibility is lower than the threshold, and filter out instances whose visibility is greater than the threshold.	Unchecked	/

1.4.11 Filter Instances with Overlapping Bounding Boxes

Function Introduction

Filter instances whose bounding boxes intersect and overlap

Usage Scenario

Applicable to scenarios where instance bounding boxes intersect each other

Parameter Description

Parameter	Description	Default Value	Value Range
Bounding box overlap ratio threshold	Threshold for the ratio of the bounding box intersection area to the instance bounding box area	0.05	[0, 1]
Filter the instance with the larger bounding box area	Checked: among two instances with intersecting bounding boxes, filter out the instance with the larger area. Unchecked: filter out the instance with the smaller area	Checked	/

Example

Newly added Filter enclosed instances. Run with the default values and view the intersection status of instance bounding boxes in the logs. After instance filtering, 2 instances remain

The logs show that 12 instances were filtered out because their bounding boxes intersected, leaving 2 instances whose bounding boxes do not intersect

Set Bounding box overlap ratio threshold to 0.1 and check Whether to filter the larger instance. View the instance filtering process in the logs. 9 instances are filtered out because the ratio of the bounding box intersection area to the instance bounding box area is greater than 0.1, 3 instances are retained because the ratio is less than 0.1, and 2 instances have bounding boxes that do not intersect.

Set Bounding box overlap ratio threshold to 0.1 and uncheck Whether to filter the larger instance. View the instance filtering process in the logs. For 9 instances, the ratio of the bounding box intersection area to the instance bounding box area is greater than 0.1, but 2 of them are retained because their bounding box areas are smaller than those of the instances intersecting with them, so 7 instances are filtered out. 3 instances are retained because the ratio of the bounding box intersection area to the instance bounding box area is less than 0.1, and 2 instances have bounding boxes that do not intersect.

1.4.12 [Master] Filter Instances with Uneven Masks Based on the Mask/Enclosing Polygon Area Ratio

Function Introduction

Calculate the area ratio of the Mask to the polygon enclosing the Mask. If the ratio is lower than the configured threshold, the instance will be filtered out

Usage Scenario

Applicable when the target object Mask has jagged or uneven edges.

Parameter Description

Parameter	Description	Default Value	Value Range
Area ratio threshold	Threshold for the Mask/convex hull area ratio. If the ratio is lower than the configured threshold, the instance will be filtered out.	0.1	[0,1]

1.4.13 [Master] Filter by Average Point Cloud Distance

Function Introduction

Filter based on the average distance from points in the Point Cloud to the fitted plane, and remove uneven instance Point Clouds

Usage Scenario

Applicable to scenarios where the Point Cloud of a planar target object is bent

Parameter Description

Parameter	Description	Default Value	Value Range	Unit
Plane segmentation distance threshold (mm)	Extract a plane from the bent instance Point Cloud. Points whose distance to the plane is less than this threshold are regarded as points on that plane	10	[-1000, 1000]	mm
Average distance threshold (mm)	Average distance from points in the instance Point Cloud to the extracted plane	20	[-1000, 1000]	mm
Remove instances whose average distance is lower than the threshold	Checked: filter out instances whose average distance from points to the extracted plane is lower than the Average distance threshold. Unchecked: filter out instances whose average distance from points to the extracted plane is greater than the Average distance threshold	Unchecked	/	/

1.4.14 [Master] Filter Occluded Instances Based on the Mask/Bounding Box Area Ratio

Function Introduction

Calculate the Mask/bounding box area ratio. Instances whose ratios are outside the minimum and maximum range will be filtered out

Usage Scenario

Used to filter instances of occluded target objects

Parameter Description

Conversely, it indicates possible occlusion.

Parameter	Description	Default Value	Value Range
Minimum area ratio	Lower limit of the Mask/bounding box area ratio range. The smaller the ratio, the more severely the instance is occluded	0.1	[0,1]
Maximum area ratio	Upper limit of the Mask/bounding box area ratio range. The closer the ratio is to 1, the less the instance is occluded	1.0	[0,1]

1.4.15 [Master] Determine Whether All Top-Layer Instances Are Fully Detected

Function Introduction

One of the fail-safe mechanisms. It determines whether all top-layer instances have been detected. If any top-layer instance is not detected, an error is reported and the workflow ends

Usage Scenario

Applicable to scenarios where one image is used for multiple picks or picking must be performed in sequence, to prevent missed picks and subsequent operational impact caused by incomplete instance detection

Parameter Description

Parameter	Description	Default Value	Value Range	Unit	Tuning
Distance threshold	Used to determine top-layer target objects. If the distance between a point and the highest point of the target object's Point Cloud is smaller than the distance threshold, the point is considered part of the top-layer Point Cloud; otherwise, it is not considered part of the top-layer Point Cloud.	5	[0.1, 1000]	mm	It should be smaller than the height of the target object

1.5 Instance Sorting

Function Introduction

Group, sort, and extract instances according to the selected strategy

Usage Scenario

Applicable to depalletizing, unordered picking, and ordered loading/unloading scenarios

If sorting is not required, no specific strategy needs to be configured.

1.5.1 Reference Coordinate System

Function Introduction

Set a unified coordinate system for all instances for grouping and sorting

Usage Scenario

Applicable to depalletizing scenarios, unordered picking scenarios, and ordered loading/unloading scenarios

Before using coordinate-related strategies, set the reference coordinate system first

Parameter Description

Parameter	Description	Illustration
Camera coordinate system	The origin of the coordinate system is above the object, and the positive Z-axis points downward; the XYZ values are the values of the object center point in this coordinate system
ROI coordinate system	The origin of the coordinate system is approximately at the center of the stack, and the positive Z-axis points upward; the XYZ values are the values of the object center point in this coordinate system
Robot coordinate system	The origin of the coordinate system is on the robot itself, and the positive Z-axis generally points upward; the XYZ values are the values of the object center point in this coordinate system
Pixel coordinate system	The origin of the coordinate system is at the top-left corner of the RGB image and it is a 2D planar coordinate system; the X and Y values are the x and y values of the bbox detection box, and Z is 0

1.5.2 General Grasping Strategy

Parameter Description

Parameter	Description	Default Value
Strategy	Select which value to use for grouping and sorting, and how to sort, including instance Point Cloud center XYZ coordinates, bounding box aspect ratio, distance from the instance Point Cloud center to the ROI center, and more. Multiple strategies can be stacked and executed in sequence	Instance Point Cloud center X coordinate from small to large (mm)
Grouping step size	According to the selected strategy, instances are divided into groups based on the step size. The grouping step size is the interval between two groups. For example, if the strategy selected is "Instance Point Cloud center Z coordinate from large to small (mm)", then the Z coordinates of all instance Point Cloud centers are sorted from large to small and grouped by the step size, and the corresponding instances are also divided into several groups	/
How many leading groups to extract	After grouping and sorting, how many groups of instances need to be retained	10000

Strategy Name*	Description	Grouping Step Size		How Many Leading Groups to Extract
Strategy Name*	Description	Default Value	Value Range	Default Value
Instance Point Cloud center XYZ coordinate from large to small / from small to large (mm)	Use the XYZ coordinate values of each instance Point Cloud center for grouping and sorting Before using this strategy for sorting, set the reference coordinate system first	200.000	(0, 10000000]	10000
From the middle to both sides / from both sides to the middle along the instance Point Cloud center XY axis (mm)	Use the XY coordinate values of each instance Point Cloud center for grouping and sorting in the direction of "middle to both sides" or "both sides to middle" Before using this strategy for sorting, set the reference coordinate system first	200.000	(0, 10000000]	10000
Bounding box center XY coordinate from large to small / from small to large (mm)	Use the XY coordinate values of the center point of each instance bounding box in the pixel coordinate system for grouping and sorting	200.000	(0, 10000000]	10000
Bounding box aspect ratio from large to small / from small to large	Use the ratio of the longer side to the shorter side of the bounding box for grouping and sorting	1	(0, 10000]	10000
From the middle to both sides / from both sides to the middle along the bounding box center XY axis (mm)	Use the XY coordinate values of the bounding box center point for grouping and sorting in the direction of "middle to both sides" or "both sides to middle"	200.000	(0, 10000000]	10000
Target Object type ID from large to small / from small to large	Use the target object type ID for grouping and sorting, which is suitable for multi-class target object scenarios	1	[1, 10000]	10000
Local feature ID from large to small / from small to large	Use the local feature ID for grouping and sorting	1	[1, 10000]	10000
Confidence from large to small / from small to large	Use the Confidence of each instance for grouping and sorting	1	(0, 1]	10000
Visibility from small to large / from large to small	Use the visibility of each instance for grouping and sorting	1	(0, 0.1]	10000
Mask area from large to small / from small to large	Use the Mask area of each instance for grouping and sorting	10000	[1, 10000000]	10000
Instance Point Cloud center distance to ROI center from near to far / from far to near (mm)	Use the distance between each instance Point Cloud center and the center of the ROI coordinate system for grouping and sorting	200.000	(0, 10000000]	10000
Instance Point Cloud center distance to the robot coordinate origin from near to far / from far to near (mm)	Use the distance between each instance Point Cloud center and the origin of the robot coordinate system for grouping and sorting	200.000	(0, 10000000]	10000

Example

1.5.3 Custom Grasping Strategy

（1）Function Description

Switch Grasping Strategy to Custom Grasping Strategy, and click Add to add a custom grasping strategy.

Customize the picking order for each target object. If it is difficult to achieve picking with the general grasping strategy, or it is difficult to tune appropriate parameters because of Point Cloud noise and other issues, you can consider using a custom grasping strategy
The custom grasping strategy is applicable to depalletizing scenarios and ordered loading/unloading scenarios, but not to unordered picking scenarios, because the target objects in a custom grasping strategy must be ordered (that is, the target object order is fixed)
A custom grasping strategy can only be used in combination with a single general grasping strategy, and the strategy can only select the Z coordinate from small to large

（2）Parameter Description

Parameter	Description	Default Value	Value Range	Parameter Tuning
IOU Threshold	Represents the overlap threshold between the annotated bbox and the detected bbox. The overlap is used to determine which image's sorting method should be selected for the current target object instance sorting.	0.7	[0,1]	The larger the threshold, the stricter the matching and the worse the anti-interference capability. Small shape or position changes may cause matching failure, which may match the wrong custom strategy and sort in the wrong order
Pixel Distance Threshold	Represents the dimensional difference between the matchable bbox and the detected bbox.	100	[0,1000]	The smaller the threshold, the stricter the matching and the better the anti-interference capability. If target object placement between different layers is relatively similar, the custom strategy may also be mismatched, leading to an incorrect sorting order.

（3）Select the Reference Coordinate System

When using a custom grasping strategy, only the Camera coordinate system or the pixel coordinate system can be selected

If there are multiple layers of target objects, select the Camera coordinate system; if there is only one layer of target objects, select the pixel coordinate system

（4）Strategy, Grouping Step Size, How Many Leading Groups to Extract

Parameter	Description	Default Value
Strategy	Only Instance Point Cloud Center Z Coordinate Value from Large to Small / from Small to Large (mm) can be selected	/
Grouping Step Size	According to the strategy of sorting Z coordinates from small to large, the Z coordinates of instances are sorted from small to large, and instances are divided into several groups according to the step size	10000
How Many Leading Groups to Extract	After grouping and sorting, how many groups of instances need to be retained	10000

（5）Capture an Image / Add a Local Image

Click Capture an Image to obtain an image from the currently connected Camera, or click Add a Local Image to import an image locally. For however many layers there are or however many different placement patterns of target objects there are, you need to capture or add that many images. If each layer is the same, only one image is required. Right-click the image to delete it.

Press and hold the left mouse button and drag on the acquired image to annotate the bbox. The DELETE key can be used to delete the annotated bbox step by step.

2. 3D Calculation

This section mainly explains the functions related to Pick Point generation that affect the result, along with Parameter tuning suggestions

2.1 Preprocessing

The preprocessing for 3D calculation processes the 3D Point Cloud before pose estimation and Pick Point generation for instances

2.1.1 Point Cloud Clustering Denoising

Function

Remove noise by Point Cloud clustering

Usage Scenario

There is a lot of noise in the instance Point Cloud

Point Cloud clustering denoising may filter out part of the target object's Point Cloud because the target object's Point Cloud is connected to the noise Point Cloud

Parameter Description

Parameter Name	Description	Default Value	Value Range	Unit	Parameter Tuning Suggestion
Distance Threshold for Point Cloud Clustering (mm)	Determines whether Point Clouds in space belong to the same class. If the distance between Point Clouds is lower than this threshold, they are considered the same class	5	[0.1, 1000]	mm	Generally no need to change. It should be greater than the point spacing of the target object's Point Cloud and smaller than the minimum distance between the target object's Point Cloud and the noise Point Cloud
Minimum Point Count Threshold	Point Cloud clusters with fewer points than this threshold will be filtered out	100	[1,10000000]	/	Generally no need to change. Increase the Minimum Point Count Threshold according to the amount of noise in the instance Point Cloud
Maximum Point Count Threshold	Point Cloud clusters with more points than this threshold will be filtered out	100000	[1,10000000]	/	Generally no need to change. If the number of points in the target object's Point Cloud is greater than 100000, increase the Maximum Point Count Threshold
Select the Top ROI Point Cloud	Checked means calculate and sort the average Z coordinate values of Point Clouds of the same class in the ROI coordinate system, and retain the Point Cloud class with the largest average Z coordinate value (the top Point Cloud); unchecked means retain all Point Clouds that meet the conditions	Unchecked	/	/	If the target object's Point Cloud is above the noise Point Cloud, checking this option retains the target object's Point Cloud; if the target object's Point Cloud is below the noise Point Cloud, you should also adjust the Z-axis of the ROI coordinate system downward while checking this option so that the target object's Point Cloud can be retained
Visualize Process Data	Checked means save the denoised Point Cloud, which can be viewed in C:_data	Unchecked	/	/	In debug mode, check this if you need to save visualization data

Example

（1）Without using Point Cloud clustering denoising, the generated instance Point Clouds are shown below, and instances 0, 1, and 2 all contain a large amount of Background noise

（2）Check Point Cloud Clustering Denoising and run with the default values. If clustering succeeds, Point Cloud classes in the instance Point Cloud with more than 100 and fewer than 100000 points will be retained, while classes with fewer than 100 or more than 100000 points will be filtered out. The log shows that after clustering, the Point Cloud counts of instances 4, 5, and 9 are all greater than 100 and less than 100000, so their Point Clouds are not filtered out.

（3）Increase the Minimum Point Count Threshold to 400, and only Class 0 Point Cloud is retained after denoising for instances 4, 5, and 9

（4）Check Select the Top ROI Point Cloud to retain only the Point Cloud class with the largest average Z coordinate value in the ROI coordinate system.

As shown below, instance 7 has 5 Point Cloud classes after clustering. If the Minimum Point Count Threshold is set to 400, Classes 3 and 4 of instance 7 are filtered out, while Classes 0, 1, and 2 are retained.

Check Select the Top ROI Point Cloud, and only Class 0 Point Cloud is retained for instance 7

If the noise is above and the target object's Point Cloud is below, the ROI Z-axis should be adjusted downward in the visualization window so that the average Z coordinate value of the target object's Point Cloud is the largest and the target object's Point Cloud is retained; otherwise, the retained top Point Cloud will be noise

（5）If clustering fails and the log reports an error as shown below, the Distance Threshold for Point Cloud Clustering should be increased. The Distance Threshold for Point Cloud Clustering should be greater than the point spacing of the target object's Point Cloud and smaller than the minimum distance between the target object's Point Cloud and the noise Point Cloud

Use meshlab to measure the point spacing in the target object's Point Cloud in the generated instance Point Cloud and the minimum distance between the target object's Point Cloud and the noise Point Cloud, as shown below

If the number of points in the target object's Point Cloud is greater than the Maximum Point Count Threshold, all points may be filtered out because the Point Cloud count is greater than 100000, causing clustering failure, so the Maximum Point Count Threshold should be increased

2.1.2 Point Cloud Downsampling

Function

Sample the Point Cloud according to the specified point spacing to reduce the number of calculation points and improve model Inference speed, but accuracy may decrease

Usage Scenario

If the number of Point Cloud points in the actual scenario is too large, check Point Cloud Downsampling.

Parameter Description

Parameter	Description	Default Value	Parameter Range	Unit
Downsampling Point Spacing (mm)	Sample the Point Cloud according to the specified point spacing	5.0	[0.1, 1000]	mm

Parameter Tuning
- The larger the value of Downsampling Point Spacing, the fewer points remain in the downsampled Point Cloud, the faster Pick Point calculation becomes, but accuracy may decrease
- The smaller the value of Downsampling Point Spacing, the more points remain in the downsampled Point Cloud, the slower Pick Point calculation becomes, but accuracy improves

2.1.3 Calculate Normals

Function

Calculate Point Cloud Normals for subsequent Point Cloud processing

Parameter Description

Parameter Name	Description	Default Value	Value Range
Fix Normal Orientation	Whether to fix the orientation when calculating Point Cloud Normals. After enabling, the Normal orientation is determined by the reference vector	Checked	/
Number of Neighboring Points for Normal Calculation	The larger the value, the more neighboring points are referenced, but local changes may be ignored; the smaller the value, the opposite applies	30	[1,200]
Orientation Reference Vector	The orientation reference vector for Normal calculation	[0,0,1]	/

Parameter Tuning

Cannot be changed

2.1.4 Point Cloud Contour Extraction

Function

Extract the target object's contour from the instance Point Cloud

Usage Scenario

When using Use Contour Mode in **2.2.2 Object Pose Correction **, you should also check Point Cloud Contour Extraction

Parameter Description

Parameter Name	Description	Default Value	Value Range	Unit	Parameter Tuning Suggestion
Reference Radius (mm)	The search radius for extracting contours from the instance Point Cloud	10	[0.1,10000000000]	mm	The reference radius is recommended to be set to 1/2 of the Downsampling Point Spacing in 2.1.2 Point Cloud Downsampling, and it must be greater than the Point Cloud spacing
Point Cloud Contour Search Mode	The mode for searching Point Cloud contours	Normal Mode	Normal mode; planar mode	/	Generally select Normal Mode; choose planar mode for planar target objects

Example

2.1.5 Filter Point Cloud by HSV Color Range (Hue, Saturation, Value)

Function

Filter the Point Cloud according to hue, saturation, and brightness in the Point Cloud image, and screen out Point Cloud regions that match the target range

Parameter Description

Parameter Name	Description	Default Value	Value Range
HSV-Based Depth Filtering - Maximum Color Range Value	Maximum color value for filtering the Point Cloud	[0.9,0.9,0.9]	[[0,0,0],[1,1,1]]
HSV-Based Depth Filtering - Minimum Color Range Value	Minimum color value for filtering the Point Cloud	[0.0,0.0,0.0]	[[0,0,0],[1,1,1]]

Example

2.1.6 Filter Point Cloud by Three-Channel Color

Function

Filter the Point Cloud by three-channel color and screen out Point Cloud regions that match the target range

Parameter Description

Parameter Name	Description	Default Value	Value Range
Three-Channel Color-Based Point Cloud Filtering - Maximum Color Value	Maximum color value for filtering the Point Cloud	[0.9,0.9,0.9]	[[0,0,0],[1,1,1]]
Three-Channel Color-Based Depth Filtering - Minimum Color Value	Minimum color value for filtering the Point Cloud	[0.0,0.0,0.0]	[[0,0,0],[1,1,1]]

Example

2.1.7 Select Point Clouds Within the ROI Area

Function

Select Point Clouds within the ROI 3D region from the instance Point Cloud. This default function cannot be deleted

Example

2.1.8 Remove Points Whose Normals Exceed the Angle Threshold

Function

Remove Point Clouds whose angle between the Normal vector and the axis direction of the standard Normal exceeds the Normal angle threshold

Usage Scenario

Loading and unloading of planar target objects (materials are isolated from each other)

Parameter Description

Parameter Name	Description	Default Value	Value Range	Unit
Angle Threshold	Point Clouds with an angle greater than this threshold are considered different instances	15	[-360, 360]
Standard Normal Axis Direction	The angle formed between the Point Cloud Normal and the axis direction of the standard Normal	Z-axis	X/Y/Z-axis	/
Whether to Use the ROI Coordinate System	Checked means calculate the angle between the Normal and the axis of the ROI coordinate system; unchecked means calculate the angle between the Normal and the axis of the Camera coordinate system	Unchecked	/	/

2.1.9 Point Cloud Plane Segmentation

Function

Retain or remove the plane with the largest number of Point Cloud points in the instance Point Cloud

Usage Scenario

There is a noisy plane in the instance Point Cloud

Parameter Description

Parameter	Description	Default Value	Value Range	Unit	Parameter Tuning Suggestion
Reference Distance for Plane Fitting (mm)	If the distance between a point and the plane is lower than the reference distance, it is considered a point on the plane; otherwise, it is considered a point outside the plane	3	[0.001，10000]	mm	Generally no need to change
Remove Plane	Checked means remove the plane with the largest number of Point Cloud points; unchecked means retain the plane with the largest number of Point Cloud points	Unchecked	/	/	If the plane with the largest number of Point Cloud points is the target object, retain the plane and leave it unchecked; if the plane with the largest number of Point Cloud points is noise, remove the plane and check this option

2.1.10 Remove Point Cloud Outliers

Function

Identify and remove outlier noise in the Point Cloud to improve Point Cloud quality

Usage Scenario

The instance Point Cloud contains a lot of outlier noise

Parameter Description

Parameter Name	Description	Default Value	Value Range
Reference Neighborhood Point Count	The number of neighboring points for each point in the Point Cloud, that is, the neighborhood size. For dense Point Clouds, even a small neighborhood is sufficient to reflect the target object's features, so a smaller value can be used; for sparse Point Clouds, a larger neighborhood is needed to reflect the target object's features, so a larger value should be used.	30	[1, 10000000]
Standard Deviation Multiplier	Used to identify outlier noise. If the deviation between a point's coordinates and the average coordinates of the instance Point Cloud exceeds the standard deviation multiplier, the point is considered an outlier. The smaller the value, the more points are considered outliers and removed, but this may lead to misjudgment and removal of important target object features; the larger the value, the fewer points are considered outliers and removed, but some outliers may be retained and affect target object recognition accuracy.	0.005	[0.0001, 2]

Parameter Tuning

Generally no need to change. If the Point Cloud becomes too sparse after Point Cloud Outlier Removal, you should increase the standard deviation multiplier

Example

2.1.11 Filter Out Point Clouds Whose Object Distance Exceeds the Limit

Function

Filter out Point Clouds in the specified direction, remove noise points, and improve the accuracy of image recognition

Parameter Description

Parameter	Description	Default Value	Parameter Range	Unit	Parameter Tuning Suggestion
Specified Axis	The specified axis of the Point Cloud, used to filter out Point Clouds in the specified direction	Z-axis	X/Y/Z-axis	/	Specified Axis generally does not need to be changed
Threshold (mm)	In the specified axis direction, if the distance between the lower-layer Point Cloud and the target object's Point Cloud is greater than this threshold, the lower-layer Point Cloud will be filtered out; if the distance between the lower-layer Point Cloud and the target object's Point Cloud is smaller than this threshold, the lower-layer Point Cloud will be retained	750	[0, 1000]	mm	Adjust the Threshold according to the actual scenario. The larger the Threshold, the fewer Point Clouds are filtered out; the smaller the Threshold, the more Point Clouds are filtered out
Select Coordinate System	Filter out Point Clouds under the selected coordinate system	ROI Coordinate System	Camera coordinate system; ROI coordinate system; object's own coordinate system	/

Example

2.1.12 Optimize the Mask According to the Point Cloud

Function

Based on the Point Cloud within ROI 3D, remove Mask points that are outside ROI 3D to improve Mask accuracy

2.2 Pose Estimation

2.2.1 Coarse Matching Evaluation Threshold (mm)

Function

During coarse matching, the keypoints of the template Point Cloud are matched with the keypoints predicted by the model for the instance Point Cloud. If the average distance between the two groups of keypoints after matching is lower than the evaluation threshold, the two groups of keypoints are considered matchable; if the average distance after matching is higher than the evaluation threshold, the two groups of keypoints are considered not matchable.

Usage Scenario

General target object ordered loading/unloading, general target object unordered picking, and general target object positioning assembly scenarios

Parameter Description

Default Value: 10

Value Range: [1, 1000]

Unit: mm

Parameter Tuning

Usually unchanged

Example

You can view the coarse matching result using keypoints in the visualization window, as shown below.

2.2.2 Object Pose Correction

Fine Matching Search Radius (mm)

Function

During Fine Matching, the template Point Cloud is matched with the instance Point Cloud, and each point in the template Point Cloud needs to search for its nearest point in the instance Point Cloud. The Fine Matching search radius represents both the search radius in the instance Point Cloud and the distance threshold between each point in the template Point Cloud and its nearest point in the instance Point Cloud. If the distance between a point and its nearest point is less than the Fine Matching search radius, the two points are considered matchable; otherwise, they are considered not matchable.

Usage Scenario

General target object ordered loading/unloading, general target object unordered picking, and general target object positioning assembly scenarios

Parameter Description

Default Value: 10

Value Range: [1, 500]

Unit: mm

Parameter Tuning

Usually unchanged

Example

You can view the Fine Matching result in the visualization window, as shown below.

Fine Matching Search Mode

Function

The way the template Point Cloud searches for nearest points in the instance Point Cloud during Fine Matching

Usage Scenario

If the Fine Matching result between the template Point Cloud and the instance Point Cloud is poor, this function should be adjusted

Parameter Description

Parameter	Description
Point-to-Point	Each point in the template Point Cloud searches for its nearest point in the instance Point Cloud (the point with the shortest straight-line distance within the search radius), which is suitable for all target objects
Point-to-Plane	Each point in the template Point Cloud searches for its nearest point in the instance Point Cloud along its Normal, which is suitable for target objects with obvious geometric features
Combination of Point-to-Point and Point-to-Plane	First use point-to-point mode to optimize the target object's pose in the instance Point Cloud, and then use point-to-plane mode to optimize the target object's pose in the instance Point Cloud. This is suitable for target objects with obvious geometric features Using this method will increase Takt Time

Use Contour Mode

Function

Extract contour Point Clouds from the template Point Cloud and the instance Point Cloud for coarse matching

Usage Scenario

General target object ordered loading/unloading, general target object unordered picking, and general target object positioning assembly scenarios. If the result of using keypoints for coarse matching is poor, this function should be checked to use contour Point Clouds for another round of coarse matching

Parameter Tuning

The result of coarse matching will affect the Fine Matching result. If the Fine Matching result is poor, you can check Use Contour Mode

You can view the Fine Matching result in the visualization window, as shown below.

Contour Search Range (mm)

Function

The search radius for extracting contour Point Clouds from the template Point Cloud and the instance Point Cloud

Usage Scenario

General target object ordered loading/unloading, general target object unordered picking, and general target object positioning assembly scenarios

Parameter Description

Default Value: 5

Value Range: [0.1, 500]

Unit: m

Parameter Tuning

If the value is smaller, the search radius for contour Point Clouds is smaller, which is suitable for extracting fine target object contours, but the extracted contours may contain outlier noise;

If the value is larger, the search radius for contour Point Clouds is larger, which is suitable for extracting broader target object contours, but the extracted contours may ignore some detailed features.

Save Pose Estimation [Fine Matching] Data

Function

Checked means save Fine Matching data.

Use Case

Applicable to general target object ordered loading/unloading, general target object unordered picking, and general target object positioning assembly scenarios

Example

Fine Matching data is saved in the Project storage path \ProjectFolder\data\PickLight\HistoricalDataTimestamp\Builder\pose\output folder.

2.2.3 Pose Adjustment Based on Axis Rotation

Function

Rotate and translate the instance Point Cloud around the target object's first Pick Point based on the given axis, calculate the matching score between the transformed instance Point Cloud and the template Point Cloud after each rotation and translation, and select the instance Point Cloud with the highest matching score as the final Pose of the rotationally symmetric target object.

Use Case

Use this when deviation occurs while matching the instance Point Cloud of a rotationally symmetric target object with the template Point Cloud, and a certain rotation angle is required for a complete match

Cannot be used together with the Function options Recognition Type, Front/Back Recognition (via Point Cloud Template), or Local Feature Recognition

Parameter Description

Parameter	Description	Default Value	Parameter Range	Parameter Tuning Suggestion
Rotation Angle Interval	Rotate the instance Point Cloud at equal angle intervals. This is the angle difference between two adjacent rotations. For example, if the first rotation is 30°, the second is 60°, and the third is 90°, then the rotation angle interval is 30°.	5	\[1, 180\]	If the target object has many features and matching is difficult and high-precision matching is required, set a smaller angle interval to perform more matching attempts, though computation will increase. If the target object has a simple shape and fewer features, set a larger angle interval to improve computational efficiency.
Rotation Angle Range	The maximum angle range through which the instance Point Cloud can rotate from its initial state.	90	\[1, 180\]	The Point Cloud rotation range is the positive and negative interval of the configured value. For example, if the angle range is set to 120°, the Point Cloud rotation range is \[-120°,120°\]. The instance Point Cloud rotates from -120° to 120° according to the angle interval, and the number of rotations = rotation angle range / rotation angle interval. The Rotation Angle Range is generally set to 180, 120, or 90 according to the characteristic angle of the rotationally symmetric target object. For example, if an equilateral triangular target object coincides with itself after rotating 120°, then the Rotation Angle Range should be ≥120°. If it is set to 180°, more rotations will be performed, and more rotated instance Point Clouds will be matched with the template Point Cloud. For example, if a square target object coincides with itself after rotating 90°, then the Rotation Angle Range should be ≥90°. If Pose Angle Prior is provided and the approximate orientation of the target object is known, the Rotation Angle Range can be reduced according to the possible deviation range to decrease computation.
Template Point Cloud File Path	Upload the template Point Cloud file of the target object. If no file is uploaded, the Point Cloud Template uploaded on the target object page is used.	/	/
Evaluation Mode	Evaluate the matching result from different perspectives	Iso-target	Iso-target；Iso-source；Average；Strict；Loose；Fast	Iso - target: Evaluate the matching result from the perspective of the template Point Cloud to determine whether the template Point Cloud features are well matched. Iso - source: Evaluate the matching result from the perspective of the instance Point Cloud to determine whether the changes in the instance Point Cloud during matching and the features of the matched instance Point Cloud are reasonable. Average: Comprehensively consider factors such as the error during matching and the fitting quality between the template Point Cloud and the instance Point Cloud to provide an overall evaluation result. Strict: Evaluate the matching result using relatively strict criteria. The result is considered good only when the Point Cloud matching quality is very high. This is suitable for scenarios with many target object features and high matching accuracy requirements, such as precision parts. Loose: Use a relatively lenient evaluation method that allows a certain range of matching error. This is suitable for scenarios with lower matching accuracy requirements where faster results are preferred. Fast: Emphasizes rapid evaluation and is suitable for scenarios with high matching speed requirements and lower matching accuracy requirements.
ICP Threshold	The criterion for determining whether registration is successful. If the matching result error is lower than this threshold, the match is considered successful; if the error is greater than this threshold, the match is considered unsuccessful.	0.005	\[0.000001, 1\]	Generally does not need to be changed. If the actual scenario requires higher matching accuracy, decrease this threshold; if the scenario requires higher matching speed and lower accuracy, increase this threshold. If the target object's Point Cloud quality is good, the threshold can be decreased; if the Point Cloud quality is poor, the threshold can be increased.
Rotation Axis Selection	The instance Point Cloud rotates around this axis	Z-axis	X/Y/Z-axis	Generally does not need to be changed. If a certain axis of the rotationally symmetric target object is important for recognizing the target object's Pose, set the rotation axis to that feature axis.
Save Visualization Data	Whether to save visualization data	Unchecked	/
Use Edges for Optimization	Use the target object's edge contour to optimize matching, reduce matching result error, and achieve finer matching between the instance Point Cloud and the template Point Cloud	Unchecked	/	If the target object's edge contour has unique geometric features, check this Function to improve matching accuracy. For example, for complex-shaped target objects with large differences in edge contours, using the Point Cloud of edge contours can identify the target object's Pose more accurately.
Edge Point Cloud File Path	Upload the edge Point Cloud file of the target object. If no file is uploaded, the edge Point Cloud is extracted from the Point Cloud Template uploaded on the target object page.	/	/
Optimize Rotation Result	After the optimal Pose is found during matching, optimize it again to reduce matching result error and achieve finer matching between the instance Point Cloud and the template Point Cloud	Checked	/	Checked by default to improve matching accuracy and generally does not need to be changed
Optimization Mode	The mode used to optimize the matching result	Point	Point；Plane；Full	Point (point mode): Optimize based on point-to-point correspondences, calculate point distance errors, and iteratively adjust. Suitable for target objects with simple Point Clouds and few features. Plane (plane mode): Consider the planar features of the Point Cloud and include the relationship between points and planes in registration calculations. Suitable for target objects with many planar surfaces, such as plate-like target objects. Full (comprehensive mode): Comprehensively considers multiple geometric features such as points, lines, and planes for registration optimization, making more complete use of Point Cloud information but with relatively higher computational cost. Suitable for target objects with complex Point Clouds, rich geometric features, and high accuracy requirements.
Optimization Threshold	The criterion for determining whether registration reaches the expected accuracy during optimization. If the registration error is lower than this threshold, optimization is successful; if the registration error is greater than this threshold, optimization fails and iteration must continue.	0.002	\[0.0001, 1\]	Generally does not need to be changed. Decrease this threshold for scenarios with higher registration accuracy requirements, and increase it for scenarios with lower registration accuracy requirements.
Enable Center Shift Mode	When enabled, translation mode is also added so that translation is superimposed while the instance Point Cloud is rotated	Unchecked		If the registration result has an offset along an axis direction, enable Center Shift Mode to optimize the registration result Note: Enabling Center Shift Mode requires the Evaluation Mode to be set to Fast
x-Axis Shift Range	The shift range along the Pick Point x-axis in Center Shift Mode (mm)	0	\[0,100\]	A default value of 0 means the x-axis does not move during evaluation; a non-zero value means x-axis movement is enabled Note: The specified axis in the axis settings refers to the direction before Pick Point direction adjustment. Functions in Pick Point processing may change the axis direction, so you can disable Pick Point processing functions to inspect the correct axis direction
x-Axis Shift Step Size	The shift step size along the Pick Point x-axis in Center Shift Mode (mm)	2	\[0.01,10\]	The step size must be set appropriately according to the offset condition and required accuracy. Setting the step size too small will significantly increase Takt Time
y-Axis Shift Range	The shift range along the Pick Point y-axis in Center Shift Mode (mm)	0	\[0,100\]	A default value of 0 means the y-axis does not move during evaluation; a non-zero value means y-axis movement is enabled Note: The specified axis in the axis settings refers to the direction before Pick Point direction adjustment. Functions in Pick Point processing may change the axis direction, so you can disable Pick Point processing functions to inspect the correct axis direction
y-Axis Shift Step Size	The shift step size along the Pick Point y-axis in Center Shift Mode (mm)	2	\[0.01,10\]	The step size must be set appropriately according to the offset condition and required accuracy. Setting the step size too small will significantly increase Takt Time
z-Axis Shift Range	The shift range along the Pick Point z-axis in Center Shift Mode (mm)	0	\[0,100\]	A default value of 0 means the z-axis does not move during evaluation; a non-zero value means z-axis movement is enabled Note: The specified axis in the axis settings refers to the direction before Pick Point direction adjustment. Functions in Pick Point processing may change the axis direction, so you can disable Pick Point processing functions to inspect the correct axis direction
z-Axis Shift Step Size	The shift step size along the Pick Point z-axis in Center Shift Mode (mm)	2	\[0.01,10\]	The step size must be set appropriately according to the offset condition and required accuracy. Setting the step size too small will significantly increase Takt Time

Example

Axial matching offset

After enabling Center Shift Mode

Notes on the number of Poses evaluated by Pose Adjustment Based on Axis Rotation

Total number of Pose calculations = number of x-axis shift positions * number of y-axis shift positions * number of z axis shift positions * number of angle positions

If the movement range is too large or the step size is too small, the total number of Poses to evaluate will increase and Takt Time will become longer. Set the range and step size Parameters appropriately

2.3 Empty ROI Detection

Function

Determine whether there are any target objects (Point Clouds) remaining in ROI 3D. If the number of 3D points in ROI 3D is lower than this value, it indicates that no target object Point Cloud remains, and no Point Cloud is returned.

Parameter Description

Default Value: 1000

Value Range: [0, 100000]

Workflow

Set the minimum point count threshold for ROI 3D. If the count is lower than this threshold, the target object Point Cloud in ROI 3D is considered insufficient, and it is therefore determined that no target object is present in ROI 3D;

In robot configuration, add a new Vision status code to facilitate subsequent signal processing by the robot.

3. Pick Point Processing

This section mainly explains functions related to Pick Point filtering and adjustment and provides Parameter Tuning recommendations

3.1 Pick Point Adjustment

3.1.1 Rotate the Pose When It Is Outside the Angle Range

Function Description

When the Pose is outside the set angle range, rotate the Pose counterclockwise around the fixed axis by a certain angle. If it is still outside the set angle range after rotation, a warning will be issued.

Use Case

Only applicable to depalletizing scenarios. This Function keeps the robot's direction stable during picking, avoids repeated 180° rotation of the end Tool during picking, and helps prevent issues such as cable winding.

Parameter Description

Parameter	Description	Default Value	Parameter Range	Unit
Fixed Axis	A certain axis of the Picking Pose. Rotate the Pose counterclockwise around this fixed axis	Z-axis	X/Y/Z-axis	/
Rotation Angle	The angle by which the Pose rotates counterclockwise around the fixed axis. Adjust this angle so the Picking Pose falls within the angle range	0	[-360,360]	Angle
Angle Range	The angle range of the Picking Pose. Set the angle range according to factors such as material placement, end Tool type, and Takt Time	[0,180]	[-180,180]	Angle
Use Current Robot Euler Angles	The Euler Angles "XYZ" are used for Pose calculation by default. When checked, the Euler Angles configured for the current robot are used so the Pose remains consistent with the robot teach pendant.	Unchecked	/	/
Custom Coordinate System	The coordinate system of the Picking Pose	Robot Arm Coordinate System	Default coordinate system; Camera coordinate system; ROI coordinate system; Robot arm coordinate system	/

Example

3.1.2 Rotate the Pose so the Rotation Axis Direction Aligns with the Target Axis Direction

Function Description

Rotate the Pose once around the fixed axis so that the direction of the rotation axis (determined by the right-hand rule) aligns with the positive or negative direction of the target axis in the target coordinate system.

Usage Scenario

Use Case

Avoid collisions between the robot end Tool and the material bin

Parameter Description

Parameter	Description	Default Value	Parameter Range
Rotation Axis	A certain axis of the Picking Pose, determined according to the right-hand rule. Rotate the Picking Pose counterclockwise once around the fixed axis so that the direction of the rotation axis aligns with the positive or negative direction of the target axis in the target coordinate system	X-axis	X/Y/Z-axis
Fixed Axis	Rotate the Picking Pose counterclockwise once around the fixed axis so that the direction of the rotation axis aligns with the positive or negative direction of the target axis in the target coordinate system	Z-axis	X/Y/Z-axis
Target Axis	A certain axis in the target coordinate system. Rotate the Picking Pose counterclockwise once around the fixed axis so that the direction of the rotation axis aligns with the positive or negative direction of this target axis	X-axis	X/Y/Z-axis
Negative Direction of Target Axis	When checked, the direction of the rotation axis aligns with the negative direction of the target axis in the target coordinate system. When unchecked, it aligns with the positive direction of the target axis	Unchecked	/
Custom Coordinate System	The coordinate system of the Picking Pose	Default coordinate system	Default coordinate system; Camera coordinate system; ROI coordinate system; Robot arm coordinate system

Example

3.1.3 Rotate the Pose so the Angle Between the Rotation Axis and the Target Axis Is Minimized

Function Description

Rotate the Pose around the fixed axis by 0, 90, 180, and 270 degrees respectively, calculate the angle between the rotated rotation axis and the positive or negative direction of the target axis in the Camera coordinate system after each rotation, and finally output the Pose with the smallest angle after rotation.

Use Case

Avoid collisions between the robot end Tool and the material bin

Parameter Description

Parameter	Description	Default Value	Parameter Range
Fixed Axis	A certain axis of the Picking Pose. Rotate the Pose counterclockwise around this fixed axis	Z-axis	X/Y/Z-axis
Rotation Axis	A certain axis of the Picking Pose. When rotating the Pose, calculate the angle between this rotation axis and the positive or negative direction of the target axis	X-axis	X/Y/Z-axis
Target Axis	A certain axis in the Camera coordinate system. When rotating the Pose, calculate the angle between the rotation axis and the positive or negative direction of this target axis	X-axis	X/Y/Z-axis
Negative Direction of Target Axis	When checked, calculate the angle between the rotation axis and the negative direction of the target axis. When unchecked, calculate the angle between the rotation axis and the positive direction of the target axis	Checked	/
Custom Coordinate System	The coordinate system of the Picking Pose	Default coordinate system	Default coordinate system; Camera coordinate system; ROI coordinate system; Robot arm coordinate system

Example

3.1.4 Flip the Pose so the Angle Between the Rotation Axis and the Target Axis Is Minimized

Function Description

Rotate the Picking Pose once around the fixed axis so that the angle formed between the rotation axis and the positive or negative direction of the target axis in the ROI coordinate system is acute.

Use Case

Avoid collisions between the robot end Tool and the material bin

Parameter Description

Parameter	Description	Default Value	Parameter Range
Fixed Axis	A certain axis of the Picking Pose. Rotate the Pose counterclockwise around this fixed axis	Z-axis	X/Y/Z-axis
Rotation Axis	A certain axis of the Picking Pose. Rotate the Pose so that the direction of this rotation axis aligns with the positive or negative direction of the target axis	X-axis	X/Y/Z-axis
Target Axis	A certain axis in the ROI coordinate system. Rotate the Pose so that the direction of the rotation axis aligns with the positive or negative direction of this target axis	X-axis	X/Y/Z-axis
Negative Direction of Target Axis	When checked, rotate the Pose so that the direction of the rotation axis aligns with the negative direction of the target axis. When unchecked, rotate the Pose so that the direction of the rotation axis aligns with the positive direction of the target axis	Checked	/
Custom Coordinate System	The coordinate system of the Picking Pose	Default coordinate system	Default coordinate system; Camera coordinate system; ROI coordinate system; Robot arm coordinate system

Example

3.1.5 Point an Axis of the Pose Toward the ROI Center

Function

Rotate the Picking Pose around the fixed axis so that the pointing axis of the Picking Pose points toward the ROI center.

Use Case

Avoid collisions between the robot end Tool and the material bin

Parameter Description

Parameter	Description	Default Value	Parameter Range
Pointing Axis	The axis in the Picking Pose that needs to be adjusted	X-axis	X/Y/Z-axis
Fixed Axis	The axis that remains unchanged during rotation	Z-axis	X/Y/Z-axis
Reverse Alignment	When checked, the pointing axis is aligned toward the ROI center in the reverse direction. When unchecked, the pointing axis is aligned toward the ROI center	Checked	/
Strict Pointing	When checked, forcibly rotate the Picking Pose so that the pointing axis points toward the ROI center	Unchecked	/
Custom Coordinate System	The coordinate system of the Picking Pose	Default coordinate system	Default coordinate system; Camera coordinate system; ROI coordinate system; Robot arm coordinate system

Example

3.1.6 Rotate the Pose so Its Z-axis Aligns with the Z-axis of the Target Coordinate System

Function Description

Rotate the Pose so that the direction of the Pose Z-axis aligns with the Z-axis of the target coordinate system

Use Case

Usually used by default only in depalletizing scenarios and cannot be deleted. It is required to make the Z-axis of the Picking Pose perpendicular to the Z-axis of the ROI coordinate system (four-axis robots) or aligned with the target object's surface direction (six-axis robots)

Parameter Description

Parameter	Description	Default Value	Parameter Range
Robot Configuration	Set according to the onsite robot configuration. You can choose four-axis or six-axis. If a six-axis robot is actually used as a four-axis robot, it should be set to four-axis	Four-axis	Four-axis/Six-axis
Use ROI Z-axis as the Target Direction	When the Robot Configuration is set to four-axis, checking this option rotates the Pose around the X-axis so that the Z-axis direction of the rotated Pose aligns with the positive direction of the ROI Z-axis . When unchecked, the Pose is rotated around the X-axis so that the Z-axis direction of the rotated Pose aligns with the positive direction of the Z-axis in the Camera coordinate system . When the Robot Configuration is set to six-axis, regardless of whether this option is checked, the Pose is rotated around the X-axis so that the Z-axis direction of the rotated Pose aligns with the positive direction of the Z-axis in the object's own coordinate system	Unchecked	/
Custom Coordinate System	The coordinate system of the Picking Pose	Camera Coordinate System	Default Coordinate System; Camera Coordinate System; ROI Coordinate System; Robot Arm Coordinate System

Example

3.1.7 Rotate the Pose Around a Fixed Axis

Function Description

Rotate the Pose around the fixed axis by a certain angle.

Use Case

Avoid collisions between the robot end Tool and the material bin

Parameter Description

Parameter	Description	Default Value	Parameter Range	Unit
Rotation Angle	The angle by which the Pose rotates counterclockwise around the fixed axis	90	[-360, 360]	Angle°
Fixed Axis	A certain axis of the Picking Pose. Rotate the Pose counterclockwise around this fixed axis	Z-axis	X/Y/Z-axis	/
Custom Coordinate System	The coordinate system of the Picking Pose	Default coordinate system	Default coordinate system; Camera coordinate system; ROI coordinate system; Robot arm coordinate system	/

Example

3.1.8 Translate the Pose

Function Description

Move the Pose by a certain distance along the translation axis.

Use Case

Avoid collisions between the robot end Tool and the material bin

Parameter Description

Parameter	Description	Default Value	Parameter Range	Unit
Translation Amount	The distance the Pose moves along the translation axis. A positive Translation Amount means translation in the positive direction of the translation axis, and a negative Translation Amount means translation in the negative direction of the translation axis	0	[-1.0,1.0]	m
Translation Axis	The direction in which the Pose moves	X-axis	X/Y/Z-axis	/
Custom Coordinate System	The coordinate system of the Picking Pose	Robot Arm Coordinate System	Default coordinate system; Camera coordinate system; ROI coordinate system; Robot arm coordinate system	/

Example

3.1.9 Pick Point Teaching

Function Description

Record the Pick Point coordinates generated by the software and the Pick Point coordinates taught under the current working condition, then use the offset between the two to output the transformed Picking Pose.

Use Case

When the system-generated vision Pick Points have an obvious regular offset and the robot TCP coordinate accuracy is limited or difficult to calibrate, this method can directly map the same regular offset to subsequent Pick Points, thereby avoiding robot TCP calibration.

Parameter Description

Parameter	Description	Default Value	Parameter Range
Vision Pose	The Pick coordinates from the detection result
X(mm)	X coordinate of the Vision Pose	0.00	±10000000, indicating no limit.
Y(mm)	Y coordinate of the Vision Pose	0.00	±10000000, indicating no limit.
Z(mm)	Z coordinate of the Vision Pose	0.00	±10000000, indicating no limit.
RX(°)	X-axis rotation of the Vision Pose	0.00	±180
RY(°)	Y-axis rotation of the Vision Pose	0.00	±180
RZ(°)	Z-axis rotation of the Vision Pose	0.00	±180
Picking Pose	The manually taught Pick Point
X(mm)	X coordinate of the Picking Pose	0.00	±10000000, indicating no limit.
Y(mm)	Y coordinate of the Picking Pose	0.00	±10000000, indicating no limit.
Z(mm)	Z coordinate of the Picking Pose	0.00	±10000000, indicating no limit.
RX(°)	X-axis rotation of the Picking Pose	0.00	±180
RY(°)	Y-axis rotation of the Picking Pose	0.00	±180
RZ(°)	Z-axis rotation of the Picking Pose	0.00	±180

3.1.10 Refine the Pose Based on the Plane Normal

Function Description

Correct the target object's Pose by fitting the plane normal so that the Z-axis direction of the target object's Pose remains aligned with the direction of the target object's plane normal

Use Case

Use this when the target object contains a plane and there is a deviation in the inclination angle of the plane when the template Point Cloud is matched with the instance Point Cloud. This Function fine-tunes the target object's plane to improve picking accuracy.

Not applicable to depalletizing scenarios

Parameter Description

Parameter	Description	Default Value	Parameter Range	Unit
Distance Threshold	The distance threshold for fitting a plane to the Point Cloud	0.01	[-1, 1]	m
Save Visualization Data	When checked, visualization data is saved under the historical data timestamp	Checked	/	/
Custom Coordinate System	The coordinate system of the Picking Pose	Camera coordinate system	Default coordinate system; Camera coordinate system; ROI coordinate system; Robot arm coordinate system	/

Example

3.1.11 Sort Pick Points by Inter-Axis Angle

Function

Sort Pick Points according to the angle between a certain axis of the Picking Pose and the target axis of the ROI

Parameter Description

Parameter	Description	Default Value	Parameter Range
Axis Selection	A certain axis of the Picking Pose	Z-axis	X/Y/Z-axis
Target Axis Selection	A certain axis of the ROI coordinate system	Z-axis	X/Y/Z-axis
Reverse Selection	When checked, calculate the angle with the negative direction of the target axis. When unchecked, calculate the angle with the positive direction of the target axis	Unchecked	/
Descending Order Selection	When checked, Pick Points are sorted from small to large by angle. When unchecked, Pick Points are sorted from large to small by angle	Unchecked	/

3.1.12 [Master] Rotate the Pose to Automatically Compensate for Excessive Angle Between It and the Specified Axis

Function Description

Determine whether the angle between the specified axis of the Picking Pose and the target axis falls within the specified range. If not, adjust the Picking Pose so it falls within the specified range

Use Case

Avoid collisions between the robot end Tool and the material bin

Parameter Description

Parameter	Description	Default Value	Parameter Range	Unit
Angle Range	Adjust the Picking Pose so that it falls within the angle range	30	[0, 180]	°
Specified Axis	A certain axis of the Picking Pose. Adjust this axis so that it is within the angle range relative to the target axis of the ROI coordinate system	Z-axis	X/Y/Z-axis	/
Target Axis	A certain axis of the ROI coordinate system, used to compare the angle range against the specified axis of the Picking Pose	Z-axis	X/Y/Z-axis	/
Compare with the Negative Half-axis of ROI	When unchecked, compare the angle range with the positive direction of the target axis in the ROI coordinate system. When checked, compare it with the negative direction of the target axis in the ROI coordinate system	Unchecked	/	/
Custom Coordinate System	The coordinate system of the Picking Pose	Default coordinate system	Default coordinate system; Camera coordinate system; ROI coordinate system; Robot arm coordinate system	/

3.1.13 [Master] Symmetry Center Pose Optimization

Function

Search for the symmetry center of the target object based on the instance Mask, then combine it with the Pose of the instance plane or the ROI 3D center point to calculate the optimal Picking Pose

Before using this Function, first ensure that the instance Mask is symmetrical

Use Case

Applicable when the instance Mask of a symmetrical target object is also symmetrical, but the Pick Point Pose is not near the expected center. At the same time, the target object has a usable reference plane, such as a plane on the top surface of the object, or there is ROI 3D available as a reference for the projected Pose.

Reference project scenarios include brake discs (general circles), refractory bricks (depalletizing), symmetrical irregular parts, fuel filler ports, and so on.

Parameter Description

Parameter	Description	Default Value	Parameter Range	Parameter Tuning Suggestion
Symmetry Center Type	The symmetry type of the instance Mask	Rotational Symmetry	Rotational symmetry: after the target object rotates around the center point by a certain angle, its shape completely overlaps with the original position; mirror symmetry: the target object uses a certain axis / plane as the mirror, and the left and right or upper and lower sides are completely symmetrical.	Circles and rectangles have both rotational symmetry and mirror symmetry, so rotational symmetry is preferred. For trapezoids and other shapes that are only symmetrical along a certain axis or plane, choose mirror symmetry.
Gaussian Blur Level	The tolerance level used to determine whether the rotated instance Point Cloud overlaps	3	\[1,99\]	The larger the value, the stronger the blur, the more smoothing is applied, and the more lenient the overlap judgment after rotation; The smaller the value, the lower the blur, the less smoothing is applied, and the stricter the overlap judgment. An odd number must be entered
Rotation Angle Setting	When the symmetry mode is rotational symmetry, this indicates the rotation angle interval, that is, the angle difference between two adjacent rotations. When the symmetry mode is mirror symmetry, this indicates the rotation range, that is, the angle interval within which the Point Cloud can rotate around the symmetry axis.	180	\[1,360\]	For rotational symmetry, set this based on the symmetry characteristics of the target object. Circles are symmetric at any angle, so 60° is recommended; Squares are symmetric every 90°, so 90° or 180° can be used; Rectangles are symmetric every 180°, so set this to 180. For mirror symmetry, keep the adjustment within 5° to ensure fine adjustment near the symmetry axis
Image Scaling Ratio	Adjust the size of the Point Cloud image. The larger this ratio is, the smaller the Point Cloud image becomes and the less graphics memory it consumes, but more image detail is lost, which reduces computation accuracy.	2	\[1,10000000\]
Search Range	The range for expanding outward from the initially determined target object center to search for Point Cloud features. The actual range is (Search Range * 2 * Image Scaling Ratio)	10	\[1,10000000\]	For example, for a square target object, the initially determined center position is point O. If the Search Range is set to 10 and the Image Scaling Ratio is 1, the actual search range is a square region centered at point O with a side length of 10×2×1=20. Point Cloud features are searched within this region to further determine the symmetry center and optimal Picking Pose of the target object. For another example, for a circular target object, if the Search Range is set to 8 and the Image Scaling Ratio is 2, the actual search range is a circular region centered on the initially determined target object center with a diameter of 8×2×2=32. Point Cloud features are searched within this region to further determine the target object's Pose and optimal Picking Pose.
Use ROI3D as the Reference Projection Plane	When checked, ROI3D is used as the reference projection plane	Unchecked	/	Check this when the Point Cloud has no obvious plane and the projection plane is difficult to determine. Leave it unchecked when the Point Cloud has a clear plane.
Save Symmetry Center Process Data	When checked, debug data for the symmetry center process is saved and can be viewed in the project folder `\ProjectName`{=tex}`\data`{=tex}`\PickLight`{=tex}`\HistoricalDataTimestamp`{=tex}`\find`{=tex}\_symmetry_center folder	Unchecked	/	Check this when you need to inspect the specific process images
Symmetry Axis Prior Type	Takes effect in ``{=html}Mirror Symmetry``{=html} mode. Specify the known symmetry type of the target object to fix the asymmetric orientation	Automatic Search	Automatic Search; Symmetric Along the Long Axis; Symmetric Along the Short Axis	If the symmetry axis of the target object is the long axis, choose "Symmetric Along the Long Axis". If the symmetry axis of the target object is the short axis, choose "Symmetric Along the Short Axis". If you are unsure, choose "Automatic Search"
Pose Adjustment Type	Whether to inherit Pose-related information from the input	Default Pose	Default PoseInherit RotationInherit Translation	/
Symmetry Score Threshold	Symmetry results whose score is lower than this threshold are treated as abnormal results. If set to 0, no filtering is applied	0.0	\[0.0, 1.0\]	/

3.2 Pick Point Filtering

3.2.1 Filter Based on Fine Matching Score

Function Description

Filter Pick Points based on the Pose Fine Matching score

Parameter Description

Parameter	Description	Default Value	Parameter Range
Score Threshold	Keep Pick Points whose Fine Matching score is greater than this threshold	0.5	[0, 1]

Example

3.2.2 Filter Pick Points of Occluded Target Objects

Function Description

Determine whether there are too many occluding Point Clouds in the target detection area of the Pick Point for the picked target object along the ROI specified axis or the Picking Pose axis. If so, the Pick Point is determined to be occluded and is filtered out.

Use Case

Applicable to depalletizing and ordered scenarios in which target objects are picked layer by layer, but the model recognizes a lower-layer target object and the gripper collides with the upper-layer target object when picking the lower-layer one

Parameter Description

Parameter	Description	Default Value	Parameter Range	Unit
Cuboid Length in the X Direction	Set the cuboid length in the X direction of the Picking Pose	1500	[1, 10000]	mm
Cuboid Length in the Y Direction	Set the cuboid length in the Y direction of the Picking Pose	1500	[1, 10000]	mm
Cuboid Length in the Z Direction	Set the cuboid length in the Z direction of the Picking Pose	800	[1, 10000]	mm
Distance Threshold Between the Detection Area and the Pick Point Origin	Along the ROI axis direction, the near-surface region of the cuboid whose distance from the Pick Point origin exceeds this distance threshold is regarded as the target detection area	50	[1, 1000]	mm
Point Cloud Count Threshold in the Detection Area	If the number of occluding Point Clouds in the target detection area exceeds this threshold, the Pick Point is considered occluded	1000	[0, 100000]	/
Specified Axis Direction	Based on the Pose reference specified axis direction, set the specific position of the target detection area within the cuboid space (for example, the near-surface region of the front/back/left/right/top/bottom face of the cuboid)	[0,0,-1]	[1,0,0]: positive X-axis direction[-1,0,0]: negative X-axis direction[0,1,0]: positive Y-axis direction[0,-1,0]: negative Y-axis direction[0,0,1]: positive Z-axis direction[0,0,-1]: negative Z-axis direction	/
Use ROI 3D Pose Reference	When checked, adjust the collision detection area according to the ROI 3D Pose reference	Unchecked	/	/
Save Visualization Data	When checked, save visualization data according to the save data path so you can observe whether the generated cuboid is reasonable. When unchecked, the data is not saved	Unchecked	/	/

Example

3.2.3 Filter Based on the Picking Pose Angle Range

Function Description

Determine whether the angle of the Picking Pose is within the constrained angle range, and filter out all Pick Points that do not meet the condition.

Use Case

Prevent collisions caused by abnormal robot Picking Pose angles.

Parameter Description

Parameter	Description	Default Value	Parameter Range	Unit
Angle Filter Threshold	Calculate the maximum angle between the specified axis of the ROI and the specified axis of the Picking Pose. Pick Points whose angle is greater than the current threshold are filtered out	30	[-360, 360]	°
Invert the ROI Specified Axis Direction	When checked, use the negative direction of the ROI specified axis for angle calculation. When unchecked, use the positive direction of the ROI specified axis for angle calculation	Checked	/	/
Specified Axis of the Picking Pose	Specify an axis of the Picking Pose for angle calculation	Z-axis	X/Y/Z-axis	/
Specified ROI Axis	Specify an axis of the ROI coordinate system for angle calculation	Z-axis	X/Y/Z-axis	/

Example

3.2.4 Filter Pick Points Outside the ROI 3D Type Area

Function Description

Determine whether the Pick Point is within the ROI 3D range, and remove Pick Points that are outside the ROI 3D area.

Use Case

Prevent picking outside the ROI area, which could cause collisions between the robot arm and the target object.

Parameter Description

Parameter	Description	Default Value
ROI3D Type Area	This is usually the "Work Area". The "Pick Area" is an ROI area smaller than the "Work Area", which can restrict Pick Points to an ROI area smaller than the "Work Area" to avoid some collision cases.	Work Area

3.2.5 Filter Pick Points with Collisions Between the Target Object and the Fixture

Function Description

Perform collision detection between the fixture and the Point Cloud near the Pick Point. If the number of Point Cloud points in contact with the fixture exceeds the pick collision threshold, the Pick Point of the target object is considered to have a collision risk

Use Case

Use this when collision detection is required between the fixture and the Point Cloud near the picked target object

Parameter Description

Parameter	Description	Default Value	Parameter Range
Pick Collision Threshold	The maximum number of Point Cloud points the fixture can contain near the Pick Point. For example, if this is 20, a collision is considered to occur when the number of scene Point Cloud points contained by the fixture exceeds 20	20	0-10000
Collision Point Cloud Sampling (m)	The downsampling size of the Point Cloud in the collision area. A larger value increases detection speed but reduces accuracy. Applicable to scenarios with high Takt Time requirements	0.002	0.0001 - 0.5000
Save Visualization Data for Fixture Collision Detection	Save visualization data for collision detection between the fixture and the picked target object	Unchecked	Checked/Unchecked
Import Fixture Model	Select and import the fixture model used for collision detection from the folder	/	/

**The fixture should be simplified so that the number of faces is less than 500**

Example

3.2.6 [Master] Keep the Pick Point with the Maximum/Minimum Pose Value Among Instance Pick Points and Filter the Rest

Function Description

Convert the Pose into the specified coordinate system, sort the Poses according to the values on the specified sorting axis, and keep the Pose with the maximum or minimum value. This is suitable for retaining the top or bottom Pick Point of cylindrical target objects

Parameter Description

Parameter	Description	Default Value	Parameter Range
Specified Coordinate System	Select which coordinate system to convert the Pose into for processing	ROI Coordinate System	ROI Coordinate System/Camera Coordinate System
Specified Sorting Axis	Select which axis value of the Pose to use for sorting	Z-axis	X/Y/Z-axis
Keep the Minimum Value	When checked, keep the Pose with the minimum value on the sorting axis. When unchecked, keep the Pose with the maximum value on the sorting axis	Unchecked	/

Example

3.2.7 [Master] Filter Pick Points Similar to the Previous N Pick Points

Function Description

If the change amount between the current Pick Point and any cached Pick Point is within the threshold range, the current Pick Point will be filtered out

Parameter Description

Parameter	Description	Default Value	Parameter Range	Unit
Upper Limit of Pick Point Change (+)
X(mm)	Upper limit of the X coordinate	2	[0, 10000000]	mm
Y(mm)	Upper limit of the Y coordinate	2	[0, 10000000]	mm
Z(mm)	Upper limit of the Z coordinate	2	[0, 10000000]	mm
RX(°)	Upper limit of RX rotation	1	[0, 180]	°
RY(°)	Lower limit of RY rotation	1	[0, 180]	°
RZ(°)	Lower limit of RZ rotation	1	[0, 180]	°
Lower Limit of Pick Point Change (-)
X(mm)	Lower limit of the X coordinate	2	[0, 10000000]	mm
Y(mm)	Lower limit of the Y coordinate	2	[0, 10000000]	mm
Z(mm)	Lower limit of the Z coordinate	2	[0, 10000000]	mm
RX(°)	Lower limit of RX rotation	1	[0, 180]	°
RY(°)	Lower limit of RY rotation	1	[0, 180]	°
RZ(°)	Lower limit of RZ rotation	1	[0, 180]	°
Pick Point Cache Count	The number of cached Pick Points. After comparison of the current Pick Point is completed, it is added to the cache in real time	5	[1, 100]	/

3.2.8 [Master] Filter Target Object Poses Similar to the Previous N Target Object Poses

Function Description

If the change amount between the current target object Pose and any cached target object Pose is within the threshold range, the current target object Pose will be filtered out. When the target object Pose is judged to be similar, all Pick Points on that target object will be filtered out

Parameter Description

Parameter	Description	Default Value	Parameter Range	Unit
Upper Limit of Target Object Pose Change (+)
X(mm)	Upper limit of the X coordinate	2	[0, 10000000]	mm
Y(mm)	Upper limit of the Y coordinate	2	[0, 10000000]	mm
Z(mm)	Upper limit of the Z coordinate	2	[0, 10000000]	mm
RX(°)	Upper limit of RX rotation	1	[0, 180]	°
RY(°)	Lower limit of RY rotation	1	[0, 180]	°
RZ(°)	Lower limit of RZ rotation	1	[0, 180]	°
Lower Limit of Target Object Pose Change (-)
X(mm)	Lower limit of the X coordinate	2	[0, 10000000]	mm
Y(mm)	Lower limit of the Y coordinate	2	[0, 10000000]	mm
Z(mm)	Lower limit of the Z coordinate	2	[0, 10000000]	mm
RX(°)	Lower limit of RX rotation	1	[0, 180]	°
RY(°)	Lower limit of RY rotation	1	[0, 180]	°
RZ(°)	Lower limit of RZ rotation	1	[0, 180]	°
Target Object Pose Cache Count	The number of cached Vision target object Poses. After comparison of the current target object Pose is completed, it is added to the cache in real time	5	[1, 100]	/

3.2.9 [Master] Filter Pick Points Outside the Upper and Lower Limits of the Pick Coordinates

Function Description

Keep other Pick Points within the specified range of a reference Pick Point, and filter out abnormal Pick Points.

Use Case

Prevent incorrect picking by the robot arm and ensure picking accuracy.

This Function is not applicable to depalletizing scenarios

Parameter Description

Parameter	Description	Default Value	Unit
Reference Pick Coordinates
X(mm)	X coordinate of the reference Pick Point	0	mm
Y(mm)	Y coordinate of the reference Pick Point	0	mm
Z(mm)	Z coordinate of the reference Pick Point	0	mm
RX(°)	RX rotation of the reference Pick Point	0	degrees
RY(°)	RY rotation of the reference Pick Point	0	degrees
RZ(°)	RZ rotation of the reference Pick Point	0	degrees
Upper Limit of Pick Coordinates (+)
X(mm)	Upper limit of the X coordinate. For example, if the X coordinate of the reference Pick Point is 100 and the upper limit is set to 10, the allowed range is: [100-lower limit, 110]	10000000, indicating no limit.	mm
Y(mm)	Upper limit of the Y coordinate. For example, if the Y coordinate of the reference Pick Point is 100 and the upper limit is set to 10, the allowed range is: [100-lower limit, 110]	10000000	mm
Z(mm)	Upper limit of the Z coordinate. For example, if the Z coordinate of the reference Pick Point is 100 and the upper limit is set to 10, the allowed range is: [100-lower limit, 110]	10000000	mm
RX(°)	Upper limit of RX rotation. For example, if the RX rotation of the reference Pick Point is 180 and the upper limit is set to 10, the allowed range is (the default angle wraps): [[-180, -170], [180-lower limit, 180]]	180, indicating no limit.	°
RY(°)	Upper limit of RY rotation. For example, if the RY rotation of the reference Pick Point is 180 and the upper limit is set to 10, the allowed range is (the default angle wraps): [[-180, -170], [180-lower limit, 180]]	180	°
RZ(°)	Upper limit of RZ rotation. For example, if the RZ rotation of the reference Pick Point is 180 and the upper limit is set to 10, the allowed range is (the default angle wraps): [[-180, -170], [180-lower limit, 180]]	180	°
Lower Limit of Pick Coordinates (-)
X(mm)	Lower limit of the X coordinate. For example, if the X coordinate of the reference Pick Point is 100 and the lower limit is set to 10, the allowed range is: [100-lower limit value, 110]	10000000	mm
Y(mm)	Lower limit of the Y coordinate. For example, if the Y coordinate of the reference Pick Point is 100 and the lower limit is set to 10, the allowed range is: [100-lower limit, 110]	10000000	mm
Z(mm)	Lower limit of the Z coordinate. For example, if the Z coordinate of the reference Pick Point is 100 and the lower limit is set to 10, the allowed range is: [100-lower limit, 110]	10000000	mm
RX(°)	Lower limit of RX rotation. For example, if the RX rotation of the reference Pick Point is 180 and the lower limit is set to 10, the allowed range is (the default angle wraps): [[-180, -180+upper limit], [170, 180]]	180, indicating no limit.	°
RY(°)	Lower limit of RY rotation. For example, if the RY rotation of the reference Pick Point is 180 and the lower limit is set to 10, the allowed range is (the default angle wraps): [[-180, -180+upper limit], [170, 180]]	180	°
RZ(°)	Lower limit of RZ rotation. For example, if the RZ rotation of the reference Pick Point is 180 and the lower limit is set to 10, the allowed range is (the default angle wraps): [[-180, -180+upper limit], [170, 180]]	180	°

Example

3.3 Pick Point Sorting

Function Description

Group, sort, and extract Pick Points according to the selected strategy

Use Case

Applicable to unordered picking scenarios and ordered loading/unloading scenarios

If sorting is not required, no specific strategy needs to be configured.

3.3.1 Reference Coordinate System

Function Description

Set a unified coordinate system for all Pick Points to perform Pick Point grouping and sorting

Use Case

Applicable to depalletizing scenarios, unordered picking scenarios, and ordered loading/unloading scenarios

A Reference Coordinate System should be set first before using coordinate-related strategies

Parameter Description

Parameter	Description	Illustration
Camera Coordinate System	The origin of the coordinate system is above the object, and the positive Z-axis direction points downward. The XYZ values are the values of the object center point in this coordinate system
ROI Coordinate System	The origin of the coordinate system is approximately at the center of the stack, and the positive Z-axis direction points upward. The XYZ values are the values of the object center point in this coordinate system
Robot Arm Coordinate System	The origin of the coordinate system is on the robot arm itself, and the positive Z-axis direction generally points upward. The XYZ values are the values of the object center point in this coordinate system
Pixel Coordinate System	The origin of the coordinate system is at the top-left vertex of the RGB image. It is a 2D coordinate system. The X and Y values are the x and y values of the bbox detection box, and Z is 0

3.3.2 General Picking Strategy

Parameter Description

Strategy Name*	Description	Grouping Step Size		Number of Leading Groups to Extract
Strategy Name*	Description	Default Value	Value Range	Default Value
Fine Matching score from high to low / from low to high	Use the Fine Matching score of the instance after 3D matching for grouping and sorting	1	(0, 1]	10000
Target object Pose XYZ values from high to low / from low to high (mm)	Use the XYZ coordinate values of the center point of the instance Point Cloud after 3D matching for grouping and sorting The Reference Coordinate System should be set before using this strategy for sorting	200.000	[0, 10000000]	10000
From the middle to both sides / from both sides to the middle along the target object Pose XY coordinate axis	Use the XY coordinate values of the center point of the instance Point Cloud after 3D matching to perform grouping and sorting in the direction of "middle to both sides" or "both sides to middle"	200.000	[0, 10000000]	10000
Front template first then back template / back template first then front template	Use the dual-template ID of the target object's Point Cloud for grouping and sorting ID of the front template: 0; ID of the back template: 1	2	[1, 2]	2
Pick Point center X/Y/Z coordinate values from high to low / from low to high (mm)	Use the X/Y/Z coordinate values of the Pick Point center for grouping and sorting The Reference Coordinate System should be set before using this strategy for sorting	200.000	[0, 10000000]	10000
From the middle to both sides / from both sides to the middle along the Pick Point XY coordinate axis	Use the X/Y/Z coordinate values of the Pick Point center to perform grouping and sorting in the direction of "middle to both sides" or "both sides to middle"	200.000	[0, 10000000]	10000

How to Use Shadow Mode

General Target Object Visual Parameter Tuning Guide