Visual Parameter Adjustment Guide for Surface-type Target Object Loading and Unloading (Materials Isolated from Each Other)

About 17499 wordsAbout 58 min

This article mainly describes how to adjust vision parameters according to the actual scenario in surface-type target object loading and unloading (materials isolated from each other)

1. 2D Recognition

1.1 Preprocessing

The preprocessing for 2D recognition processes 2D images before Instance Segmentation

1.1.1 Point Cloud Downsampling

Function

Sample the Point Cloud according to the specified point spacing to reduce the number of computed points and improve model Inference speed, although accuracy may decrease

Use Cases

Default function for surface-type target object loading and unloading (materials isolated from each other); cannot be deleted

Parameter Description

Parameter	Description	Default Value	Parameter Range	Unit
Downsampling point spacing (m)	Sample the Point Cloud according to the specified point spacing	0.005	[0.0001, 1]	m

Parameter Tuning
- The larger the value of Downsampling point spacing, the fewer points remain after downsampling, the faster Instance Segmentation runs, but accuracy may decrease
- The smaller the value of Downsampling point spacing, the more points remain after downsampling, the slower Instance Segmentation runs, but accuracy improves

1.1.3 Fill Holes in the Depth Map

Function

Fill hole regions in the depth map and smooth the filled depth map

Use Cases

Due to issues such as structural occlusion of the Target Object itself and uneven lighting, parts of the Target Object may be missing from the depth map

Parameter Description

Parameter	Description	Default Value	Value Range
Fill kernel size	Size used for hole filling	3	[1,99]

Only odd numbers can be entered for the fill kernel size

Parameter Tuning

Adjust according to the detection result. If filling is excessive, decrease the parameter; if filling is insufficient, increase the parameter

Example

1.1.4 Filter Point Cloud by HSV (Hue, Saturation, Value)

Function

Filter the Point Cloud based on hue, saturation, and brightness in the Point Cloud image, and select Point Cloud regions matching the target range

Parameter Description

Parameter Name	Description	Default Value	Value Range
Filter depth by HSV - maximum color range value	Maximum color value for filtering the Point Cloud	[0.9,0.9,0.9]	[[0,0,0],[1,1,1]]
Filter depth by HSV - minimum color range value	Minimum color value for filtering the Point Cloud	[0.0,0.0,0.0]	[[0,0,0],[1,1,1]]

Example

1.1.5 Filter Point Cloud by Three-channel Color

Function

Filter the Point Cloud by three-channel color and select Point Cloud regions matching the target range

Parameter Description

Parameter Name	Description	Default Value	Value Range
Filter Point Cloud by three-channel color - maximum color value	Maximum color value for filtering the Point Cloud	[0.9,0.9,0.9]	[[0,0,0],[1,1,1]]
Filter depth by three-channel color - minimum color value	Minimum color value for filtering the Point Cloud	[0.0,0.0,0.0]	[[0,0,0],[1,1,1]]

Example

1.1.6 Remove Points Whose Normal Exceeds the Angle Threshold

Function

Remove Point Cloud points whose angle between the Normal vector and the axis direction of the reference Normal vector is greater than the Normal angle threshold

Use Cases

Surface-type target object loading and unloading (materials isolated from each other)

Parameter Description

Parameter Name	Description	Default Value	Value Range	Unit
Angle threshold	Point Clouds with an angle greater than this threshold are considered different instances	15	[-360, 360]	/
Reference Normal axis	The angle formed between the Point Cloud Normal and the axis direction of the reference Normal vector	Z axis	X/Y/Z axis	/
Use ROI coordinate system	If checked, the angle between the Normal and the axes of the ROI coordinate system is calculated; otherwise, the angle between the Normal and the axes of the Camera coordinate system is calculated	Unchecked	/	/

Parameter Tuning

1.1.7 Point Cloud Plane Segmentation

Function

Retain or remove the plane with the largest number of points in the instance Point Cloud

Use Cases

The instance Point Cloud contains a noisy plane

Parameter Description

Parameter	Description	Default Value	Value Range	Unit	Tuning Recommendation
Reference distance for plane fitting	If the distance from a point to the plane is less than the reference distance, it is considered an in-plane point; otherwise, it is considered an out-of-plane point	0.003	[0.000001，10]	m	Generally unchanged
Remove plane	If checked, the plane with the largest number of Point Cloud points is removed; otherwise, the plane with the largest number of Point Cloud points is retained	Unchecked	/	/	If the plane with the largest number of Point Cloud points is the Target Object, retain the plane and leave it unchecked; if it is noise, remove the plane and check this option

Example

1.1.8 Filter Out Point Clouds with Out-of-limit Object Distance

Function

Filter Point Clouds in the specified direction to remove noise and improve image recognition accuracy

Parameter Description

Parameter	Description	Default Value	Parameter Range	Unit	Tuning Recommendation
Specified axis	The specified axis of the Point Cloud, used to filter Point Clouds in the specified direction	Z axis	X/Y/Z axis	/	Specified axis generally does not need to be changed
Threshold	Threshold for filtering Point Clouds on the specified axis. Point Clouds whose distance from the Target Object Point Cloud along the specified axis exceeds this threshold will be filtered out	0.75	[0, 1]	m	Adjust Threshold according to the actual scenario. The larger the Threshold, the fewer Point Clouds are filtered out; the smaller the Threshold, the more Point Clouds are filtered out
Select coordinate system	Filter Point Clouds in the selected coordinate system	ROI coordinate system	Camera coordinate system; ROI coordinate system; object coordinate system	/	/

Example

1.1.9 Extract the Top-layer Point Cloud

Function

Extract the top-layer Point Cloud based on the depth map and filter out redundant Background Point Clouds

Use Cases

Applicable to surface-type target object loading and unloading (materials isolated from each other). Under poor lighting conditions, with similar color textures, tightly stacked or interleaved stacks, or occlusion, the model may have difficulty distinguishing texture differences between the upper and lower layers of the Target Object, which can easily cause false detections

Parameter Description

Parameter Name	Description	Default Value	Value Range	Unit	Tuning Recommendation
Distance threshold	If the distance from a point to the topmost plane (or bottommost plane) is less than this threshold, it is considered a point on the topmost plane (or bottommost plane) and should be retained; otherwise, it is considered a point on the lower layer (or upper layer) and is set to black	0.05	[0.0001, 1.0000]	m	Generally adjust to 1/2 of the Target Object height
Clustered Point Cloud count	The expected number of points participating in clustering, that is, the number of sampled Point Cloud points within the ROI 3D region	10000	[1，10000000]	/	The greater the Clustered Point Cloud count, the lower the model Inference speed but the higher the accuracy; the smaller the Clustered Point Cloud count, the higher the model Inference speed but the lower the accuracy

1.2 3D Instance Segmentation

1.2.1 Point Cloud Search Radius

Function

Points within the search radius are considered valid points

Use Cases

Surface-type target object loading and unloading (materials isolated from each other) scenarios

Parameter Description

Default value: 0.01

Value range: [0.0001, 1]

Unit: m

Parameter Tuning

Greater than the downsampling point spacing and less than the minimum actual distance between the Target Object and noise in the scene

1.2.2 Normal Angle Threshold

Function

During Instance Segmentation, points whose angle exceeds this threshold are considered different instances

Use Cases

Surface-type target object loading and unloading (materials isolated from each other) scenarios

Parameter Description

Default value: 15

Value range: [1, 360]

Parameter Tuning

Increasing it means segmentation relies mainly on the distance between Target Objects; decreasing it may lead to incomplete Instance Segmentation

1.2.3 Minimum Point Count Threshold

Function

Retain Point Cloud categories above this value

Use Cases

Surface-type target object loading and unloading (materials isolated from each other) scenarios

Parameter Description

Default value: 100

Value range: [2, 500000]

Parameter Tuning

Adjust according to the Point Cloud categories and counts of instances in the log

1.2.4 Maximum Point Count Threshold

Function

Retain Point Cloud categories below this value

Use Cases

Surface-type target object loading and unloading (materials isolated from each other) scenarios

Parameter Description

Default value: 5000000

Value range: [2, 500000]

Parameter Tuning

Adjust according to the Point Cloud categories and counts of instances in the log

1.2.5 Instance Segmentation Mode

Function

Specify which instances are returned after Instance Segmentation

Use Cases

Surface-type target object loading and unloading (materials isolated from each other) scenarios

Parameter Description

Parameter	Description
Return all instances	By default, return all instances that meet the conditions
Return the highest-position instance (ROI coordinate system)	Return the instance with the largest Z coordinate in the ROI coordinate system; by default, the Z axis of the ROI coordinate system points vertically upward
Return the instance with the largest area	Return the instance with the largest Mask area

Example

1.2.6 Use Original Point Cloud

Function

If checked, the foreground Point Cloud is downsampled and then segmented; if unchecked, the foreground Point Cloud is segmented after 2D preprocessing, downsampling, and clustering

Use Cases

Surface-type target object loading and unloading (materials isolated from each other) scenarios

Parameter Tuning

Checked by default. After checking this option, the foreground Point Cloud is segmented directly, which preserves more features of the Target Object; however, the foreground Point Cloud still needs to be downsampled, so the takt time can be optimized by increasing the downsampling point spacing.

If merged segmentation results are found, this option should be unchecked to separate the merged instances. After unchecking, the downsampling point spacing can be reduced according to the segmentation result; if the merging is not serious, consider reducing the downsampling point spacing first.

1.2.7 Point Cloud Generation

Instance Point Cloud generation mode	Mask mode (after segmentation)	[Deprecated] Point Cloud Generation - bounding box scaling ratio (after segmentation)	Generate the Point Cloud using the segmented instance Mask
	Bounding box mode (after segmentation)	Bounding box scaling ratio (after segmentation)	Generate the Point Cloud using the segmented instance bounding box
	Bounding box mode (after segmentation)	Whether color is needed when generating the Point Cloud (after segmentation)	Whether the generated instance Point Cloud needs attached color
	Mask mode (after filtering)	[Deprecated] Point Cloud Generation - bounding box scaling ratio (after filtering)	Generate the Point Cloud using the filtered instance Mask
	Bounding box mode (after filtering)	Bounding box scaling ratio (after filtering)	Generate the Point Cloud using the filtered instance bounding box
	Bounding box mode (after filtering)	Whether color is needed when generating the Point Cloud (after filtering)	Whether the generated instance Point Cloud needs attached color

Use Cases

(1) If the original Point Cloud is deleted during 2D preprocessing to filter noise, but the 3D Instance Segmentation node needs to use the complete Point Cloud, you can use "Point Cloud Generation";

(2) When instances in the scene are discontinuous (for example, split into two clusters of Point Clouds), you can use "Point Cloud Generation" to enlarge the bounding box ratio and extract complete but separated instances

Parameter Tuning

When Use Original Point Cloud is checked, there is no need to adjust the instance Point Cloud generation mode; just use the default parameters.

In the surface-type target object loading and unloading (materials isolated from each other) scenario, after unchecking **Use Original Point **, Point Cloud clustering is required before Instance Segmentation. The input for Point Cloud clustering must be the Point Cloud after 2D preprocessing, but the subsequent 3D pose estimation node requires the original Point Cloud, so an optional instance Point Cloud generation function is added.

If the instance Point Cloud generation mode is set to Mask mode (after segmentation) or Mask mode (after filtering), then the instance Point Cloud is generated from the segmented Point Cloud. Because the Point Cloud has been downsampled, the instance Point Cloud generated in Mask mode is sparser;

If the instance Point Cloud generation mode is set to Bounding box mode (after segmentation) or Bounding box mode (after filtering), then the instance Point Cloud is generated from the original Point Cloud. Because it uses the original Point Cloud, the instance Point Cloud generated in bounding box mode is denser.

1.3 Instance Filtering

1.3.1 Filter Based on Bounding Box Area

Feature Description

Filter based on the pixel area of the bounding box of the detected instance.

Use Cases

Applicable to scenarios where instance bounding box areas differ greatly. By setting upper and lower limits for the bounding box area, noise in the image can be filtered out, improving image recognition accuracy and preventing noise from increasing subsequent processing time.

Parameter Description

Parameter	Description	Default Value	Parameter Range	Unit
Minimum area (pixels)	This parameter sets the minimum filtering area for the bounding box. Instances whose bounding box area is below this value will be filtered out	1	[1, 10000000]	pixels
Maximum area (pixels)	This parameter sets the maximum filtering area for the bounding box. Instances whose bounding box area is above this value will be filtered out	10000000	[2, 10000000]	pixels

Example

Run with the default values, and you can view the bounding box area of each instance in the log, as shown below.

Adjust **Minimum area **and Maximum area according to the bounding box area of each instance. For example, if Minimum area is set to 20000 and Maximum area is set to 30000, instances with a pixel area smaller than 20000 or larger than 30000 can be filtered out, and the instance filtering process can be viewed in the log.

1.3.2 Filter Based on Bounding Box Aspect Ratio

Feature Description

Instances whose bounding box aspect ratio is outside the specified range will be filtered out

Use Cases

Applicable to scenarios where the aspect ratios of instance bounding boxes differ greatly

Parameter Description

Parameter	Description	Default Value	Parameter Range
Minimum aspect ratio	Minimum bounding box aspect ratio. Instances whose bounding box aspect ratio is below this value will be filtered out	0	[0, 10000000]
Maximum aspect ratio	Maximum bounding box aspect ratio. Instances whose bounding box aspect ratio is above this value will be filtered out	10000000	[0, 10000000]
Use X/Y edge lengths as the aspect ratio	Unchecked by default. The ratio of the longer side to the shorter side of the bounding box is used as the aspect ratio, which is suitable when the long and short sides of the bounding box differ greatly； If checked, the ratio of the bounding box edge length on the X axis to that on the Y axis in the pixel coordinate system is used as the aspect ratio, which is suitable when the long-side/short-side ratios of most normal instance bounding boxes are similar, but some abnormally detected instance bounding boxes differ greatly in their X-axis length/Y-axis length ratio.	Unchecked	/

1.3.3 Filter Instances by Category ID

Feature Description

Filter by instance category

Use Cases

Applicable to scenarios where incoming materials contain multiple types of Target Objects

Parameter Description

Parameter	Description	Default Value
Retained category IDs	Retain instances whose category IDs are in the list; instances whose category IDs are not in the list will be filtered out	[0]

Example

1.3.4 Filter by Edge Length of the Instance Point Cloud

Feature Description

Filter according to the long and short edges of the instance Point Cloud

Use Cases

Applicable to scenarios where the distances of the instance Point Cloud on the x-axis or y-axis differ greatly. By setting the distance range of the instance Point Cloud, noise in the image can be filtered out, improving image recognition accuracy and preventing noise from increasing subsequent processing time.

Parameter Description

Parameter	Description	Default Value	Parameter Range	Unit
Short edge length range (mm)	Edge length range of the short side of the Point Cloud	[0, 10000]	[0, 10000]	mm
Long edge length range (mm)	Edge length range of the long side of the Point Cloud	[0, 10000]	[0, 10000]	mm
Lower limit for edge denoising (%)	Extract the lower percentage limit of X/Y values (Camera coordinate system) in the instance Point Cloud, and remove Point Clouds outside the upper and lower limits to avoid noise affecting length calculation	5	[0, 100]	/
Upper limit for edge denoising (%)	Extract the upper percentage limit of X/Y values (Camera coordinate system) in the instance Point Cloud, and remove Point Clouds outside the upper and lower limits to avoid noise affecting length calculation	95	[0, 100]	/
Edge length type	Filter by the long edge and short edge of the instance Point Cloud. Instances whose long-edge or short-edge length is outside the range will be filtered out	Instance Point Cloud short edge	Instance Point Cloud short edge；Instance Point Cloud long edge；Instance Point Cloud long edge and short edge	/

Example

1.3.5 Filter by Classifier-based Category ID

Feature Description

Filter instances based on classifier category ID. Instances not in the reference categories will be filtered out.

Use Cases

In multi-category Target Object scenarios, the vision model may detect multiple types of Target Objects, but the actual task may require only one specific category. In this case, this function can be used to filter out unwanted Target Objects

Parameter Description

The default value is [0], which means that instances with category ID 0 are retained by default. Category IDs not in the list will be filtered out.

1.3.6 Filter by Three-channel Color

Feature Description

Instances can be filtered out using three-channel color thresholds (HSV or RGB).

Use Cases

When incorrect instances and correct instances have clearly distinguishable colors.

Parameter Description

Parameter	Description	Default Value	Value Range
Maximum color range value	Maximum color value	[180,255,255]	[[0,0,0],[255,255,255]]
Minimum color range value	Minimum color value	[0,0,0]	[[0,0,0],[255,255,255]]
Filtering percentage threshold	Color pass-rate threshold	0.05	[0,1]
Reverse filtering	If checked, instances whose proportion outside the color range is lower than the threshold are removed; if unchecked, instances whose proportion within the color range in the instance image is lower than the threshold are removed	Unchecked	/
Color mode	Color space selected for color filtering	HSV color space	RGB color spaceHSV color space

Example

1.3.7 Filter by Confidence

Feature Description

Filter according to the instance Confidence score

Use Cases

Applicable to scenarios where the Confidence values of instances differ greatly

Parameter Description

Parameter	Description	Default Value	Parameter Range
Reference Confidence	Retain instances with Confidence higher than the threshold, and filter out instances with Confidence lower than the threshold.	0.5	[0,1]
Reverse filtering result	After reversal, instances with visibility Confidence lower than the threshold are retained, and instances with Confidence higher than the threshold are filtered out.	Unchecked	/

Example

1.3.8 Filter by Point Cloud Count

Feature Description

Filter based on the number of points in the downsampled instance Point Cloud

Use Cases

The instance Point Cloud contains a large amount of noise

Parameter Description

Parameter	Description	Default Value	Parameter Range
Minimum Point Cloud count	Minimum Point Cloud count	3500	[1, 10000000]
Maximum Point Cloud count	Maximum Point Cloud count	8500	[2, 10000000]
Filter instances whose count is within the interval	If checked, instances whose Point Cloud count is between the minimum and maximum values are filtered out; if unchecked, instances whose Point Cloud count is outside the interval are filtered out	Unchecked	/

1.3.9 Filter by Mask Area

Feature Description

Filter image Masks based on the sum of pixels in the detected instance Mask (that is, pixel area).

Use Cases

Applicable to scenarios where instance Mask areas differ greatly. By setting upper and lower limits for the Mask area, noise in image Masks can be filtered out, improving image recognition accuracy and preventing noise from increasing subsequent processing time.

Parameter Setting Description

Parameter Name	Description	Default Value	Parameter Range	Unit
Reference minimum area	This parameter sets the minimum filtering area for the Mask. Instances whose Mask area is below this value will be filtered out	1	[1, 10000000]	pixels
Reference maximum area	This parameter sets the maximum filtering area for the Mask. Instances whose Mask area is above this value will be filtered out	10000000	[2, 10000000]	pixels

Example

1.3.10 Filter by Visibility

Feature Description

Filter according to the instance visibility score

Use Cases

Applicable to scenarios where instance visibility differs greatly

Parameter Description

Parameter	Description	Default Value	Parameter Range
Reference visibility threshold	Retain instances whose visibility is greater than the threshold, and filter out instances whose visibility is less than the threshold. Visibility is used to determine how visible an instance is in the image. The more the Target Object is occluded, the lower its visibility.	0.5	[0,1]
Reverse filtering result	After reversal, instances with visibility lower than the threshold are retained, and instances with visibility higher than the threshold are filtered out.	Unchecked	/

1.3.11 Filter Instances with Overlapping Bounding Boxes

Feature Description

Filter instances whose bounding boxes intersect and overlap

Use Cases

Applicable to scenarios where instance bounding boxes intersect each other

Parameter Description

Parameter	Description	Default Value	Parameter Range
Bounding box overlap ratio threshold	The threshold for the ratio of the intersecting area of bounding boxes to the area of the instance bounding box	0.05	[0, 1]
Filter the instance with the larger bounding box area	If checked, the instance with the larger area is filtered out when two intersecting bounding boxes overlap; if unchecked, the instance with the smaller area is filtered out	Checked	/

Example

Newly added filter enclosed instances. Run with the default values and view bounding box intersection among instances in the log. After instance filtering, 2 instances remain

The log shows that 12 instances were filtered out because their bounding boxes intersected, leaving 2 instances whose bounding boxes do not intersect

Set Bounding box overlap ratio threshold to 0.1 and check Whether to filter larger instances. View the instance filtering process in the log. 9 instances are filtered out because the ratio of their bounding box intersection area to their bounding box area is greater than 0.1; 3 instances are retained because the ratio is less than 0.1; and 2 instances have no bounding box intersection.

Set Bounding box overlap ratio threshold to 0.1 and uncheck Whether to filter larger instances. View the instance filtering process in the log. The ratio of the bounding box intersection area to the instance bounding box area is greater than 0.1 for 9 instances, but 2 of those instances are retained because their bounding box area is smaller than that of the intersecting instance, so 7 instances are filtered out; 3 instances are retained because the ratio is less than 0.1; and 2 instances have no bounding box intersection.

1.3.12 [Expert] Filter Instances with Uneven Mask Shapes Based on the Area Ratio of the Mask to the Mask Circumscribed Polygon

Feature Description

Calculate the area ratio of the Mask to the polygon circumscribed around the Mask. If it is less than the set threshold, the instance will be filtered out

Use Cases

Applicable when the Target Object Mask has jagged or uneven edges.

Parameter Description

Parameter	Description	Default Value	Value Range
Area ratio threshold	Threshold for the Mask/convex hull area ratio. If it is less than the set threshold, the instance will be filtered out.	0.1	[0,1]

1.3.13 [Expert] Filter by Average Point Cloud Distance

Feature Description

Filter based on the average distance from points in the Point Cloud to the fitted plane, removing uneven instance Point Clouds

Use Cases

Applicable to scenarios where the Point Cloud of planar Target Objects is bent

Parameter Description

Parameter	Description	Default Value	Parameter Range	Unit
Plane segmentation distance threshold (mm)	Extract a plane from the bent instance Point Cloud. Points whose distance to the plane is less than this threshold are regarded as points on the plane	10	[-1000, 1000]	mm
Average distance threshold (mm)	The average value of the distances from points in the instance Point Cloud to the extracted plane	20	[-1000, 1000]	mm
Remove instances whose average distance is less than the threshold	If checked, instances whose average distance from points to the extracted plane is less than the average distance threshold are filtered out; if unchecked, instances whose average distance is greater than the threshold are filtered out	Unchecked	/	/

1.3.14 [Expert] Filter Occluded Instances Based on the Area Ratio of the Mask to the Bounding Box

Feature Description

Calculate the Mask/bounding box area ratio. Instances whose ratio is outside the minimum and maximum range will be filtered out

Use Cases

Used to filter instances of occluded Target Objects

Parameter Description

, conversely, it indicates possible occlusion.

Parameter	Description	Default Value	Value Range
Minimum area ratio	Lower limit of the Mask/bounding box area ratio range. The smaller the ratio, the more severely the instance is occluded	0.1	[0,1]
Maximum area ratio	Upper limit of the Mask/bounding box area ratio range. The closer the ratio is to 1, the lower the degree of occlusion of the instance	1.0	[0,1]

1.3.15 [Expert] Determine Whether All Top-layer Instances Have Been Fully Detected

Feature Description

One of the fail-safe mechanisms: determine whether all top-layer instances have been detected. If any top-layer instance has not been detected, an error is reported and the Workflow ends

Use Cases

Applicable to scenarios where one image is used for multiple picks or where picking must be performed in sequence, to prevent missed picks caused by incomplete instance detection from affecting subsequent tasks

Parameter Description

Parameter	Description	Default Value	Parameter Range	Unit	Parameter Tuning
Distance threshold	Used to determine the top-layer Target Object. If the distance from a point to the highest point of the Target Object Point Cloud is less than the distance threshold, the point is considered part of the top-layer Point Cloud; otherwise, it is not considered part of the top-layer Point Cloud.	5	[0.1, 1000]	mm	It should be smaller than the height of the Target Object

1.4 Instance Sorting

Feature Description

Group, sort, and extract instances according to the selected strategy

Use Cases

Applicable to both unordered picking scenarios and ordered loading/unloading scenarios

If sorting is not required, you do not need to configure a specific strategy.

1.4.1Reference Coordinate System

Feature Description

Set a unified coordinate system for all instances to perform grouping and sorting

Use Cases

Applicable to depalletizing scenarios, unordered picking scenarios, and ordered loading/unloading scenarios

Coordinate-related strategies should be used only after setting the reference coordinate system

Parameter Description

Parameter	Description	Illustration
Camera coordinate system	The origin of the coordinate system is above the object, with the positive Z-axis pointing downward; XYZ values are the values of the object center point in this coordinate system
ROI coordinate system	The origin of the coordinate system is approximately at the center of the stack, with the positive Z-axis pointing upward; XYZ values are the values of the object center point in this coordinate system
Robot coordinate system	The origin of the coordinate system is on the robot itself, and the positive Z-axis generally points upward; XYZ values are the values of the object center point in this coordinate system
Pixel coordinate system	The origin of the coordinate system is at the upper-left vertex of the RGB image and is a 2D plane coordinate system; X and Y values are the x and y values of the bbox detection box, and Z is 0

1.4.2 General Picking Strategy

Parameter Description

Parameter	Description	Default Value
Strategy	Select which value is used for grouping and sorting and how to sort, including but not limited to the XYZ coordinates of the instance Point Cloud center, the bounding box aspect ratio, and the distance from the instance Point Cloud center to the ROI center. Multiple items can be superimposed and are executed in sequence	Instance Point Cloud center X coordinate from small to large (mm)
Grouping step	According to the selected strategy, instances are divided into several groups by the step size. The grouping step is the interval between two groups of instances. For example, if the strategy selected is ”Instance Point Cloud center Z coordinate from large to small (mm)“, then the Z coordinates of all instance Point Cloud centers are sorted from large to small and grouped by the step size, and the corresponding instances are divided into several groups	/
Extract first several groups	After grouping and sorting, how many groups of instances need to be retained	10000

Strategy Name*	Description	Grouping step		Extract first several groups
Strategy Name*	Description	Default Value	Value Range	Default Value
Instance Point Cloud center XYZ coordinate values from large to small/from small to large (mm)	Use the XYZ coordinate values of each instance Point Cloud center for grouping and sorting The reference coordinate system should be set before using this strategy for sorting	200.000	(0, 10000000]	10000
From the middle to both sides of the XY coordinate axes of the instance Point Cloud center/from both sides to the middle of the XY coordinate axes of the instance Point Cloud center (mm)	Use the XY coordinate values of each instance Point Cloud center to perform grouping and sorting in the direction of “from the middle to both sides” or “from both sides to the middle” The reference coordinate system should be set before using this strategy for sorting	200.000	(0, 10000000]	10000
Bounding box center XY coordinate values from large to small/from small to large (mm)	Use the XY coordinate values of the center point of each instance bounding box in the pixel coordinate system for grouping and sorting	200.000	(0, 10000000]	10000
Bounding box aspect ratio from large to small/from small to large	Use the ratio of the long side to the wide side of the bounding box for grouping and sorting	1	(0, 10000]	10000
From the middle to both sides of the bounding box center XY coordinate axes/from both sides to the middle (mm)	Use the XY coordinate values of the center point of the bounding box to perform grouping and sorting in the direction of “from the middle to both sides” or “from both sides to the middle”	200.000	(0, 10000000]	10000
Target Object type ID from large to small/from small to large	Use the Target Object type ID for grouping and sorting, applicable to multi-category Target Object scenarios	1	[1, 10000]	10000
Local feature ID from large to small/from small to large	Use the local feature ID for grouping and sorting	1	[1, 10000]	10000
Confidence from large to small/from small to large	Use the Confidence of each instance for grouping and sorting	1	(0, 1]	10000
Visibility from small to large/from large to small	Use the visibility of each instance for grouping and sorting	1	(0, 0.1]	10000
Mask area from large to small/from small to large	Use the Mask area of each instance for grouping and sorting	10000	[1, 10000000]	10000
Instance Point Cloud center distance to ROI center from near to far/from far to near (mm)	Use the distance from each instance Point Cloud center to the center of the ROI coordinate system for grouping and sorting	200.000	(0, 10000000]	10000
Instance Point Cloud center distance to the robot coordinate origin from near to far/from far to near (mm)	Use the distance from each instance Point Cloud center to the origin of the robot coordinate system for grouping and sorting	200.000	(0, 10000000]	10000

Example

1.4.3 Custom Picking Strategy

(1) Function Description

Switch Grasping Strategy to Custom Grasping Strategy, click Add to add a custom picking strategy.

Customize the picking order for each Target Object. If the general picking strategy is difficult to use for picking, or it is hard to tune appropriate parameters because of Point Cloud noise and other issues, consider using a custom picking strategy
Custom picking strategies are suitable for depalletizing scenarios and ordered loading/unloading scenarios, but not for unordered picking scenarios, because the Target Objects used in a custom picking strategy must be ordered (that is, the Target Object order is fixed)
A custom picking strategy can only be used in combination with a single general picking strategy, and the strategy can only be Z coordinate from small to large

(2) Parameter Description

Parameter	Description	Default Value	Value Range	Parameter Tuning
IOU threshold	Represents the overlap threshold between the annotated bbox and the detected bbox. The overlap is used to determine which image's sorting method should be selected for the current Target Object instance during sorting.	0.7	[0,1]	The larger the threshold, the stricter the matching and the poorer the anti-interference capability. Minor shape or position changes may cause matching to fail, possibly matching the wrong custom strategy and sorting in the wrong order
Pixel distance threshold	Represents the dimensional difference between a matchable bbox and the detected bbox.	100	[0,1000]	The smaller the threshold, the stricter the matching and the better the anti-interference capability. If the placement of Target Objects across different layers is similar, however, the custom strategy may still be mismatched, causing the sorting order to be incorrect.

(3) Select the Reference Coordinate System

When using a custom picking strategy, only the Camera coordinate system or pixel coordinate system can be selected

If there are multiple layers of Target Objects, select the Camera coordinate system; if there is only one layer of Target Objects, select the pixel coordinate system

(4) Strategy, Grouping Step, Extract First Several Groups

Parameter	Description	Default Value
Strategy	Only Instance Point Cloud center Z coordinate from large to small/from small to large (mm) can be selected	/
Grouping step	According to the strategy of sorting Z coordinates from small to large, the Z coordinates of instances are sorted from small to large and divided into several groups according to the step size	10000
Extract first several groups	After grouping and sorting, how many groups of instances need to be retained	10000

(5) Capture Image/Add Local Image

Click Capture Image to obtain images from the currently connected Camera, or click Add Local Image to import images locally. For each layer or each different arrangement of Target Objects, you need to capture or add the corresponding number of images. If every layer is the same, only one image is needed. Right-click the image to delete it.

On the obtained image, click and hold the left mouse button and drag to annotate a bbox. The DELETE key can be used to delete annotated bboxes one by one.

2. 3D Computation

2.1 Preprocessing

The preprocessing for 3D computation processes 3D Point Clouds before the Deep Learning model performs computation

2.1.1 Point Cloud Clustering Denoising

Function

Remove noise by Point Cloud clustering

Use Cases

There is much noise in the instance Point Cloud

Point Cloud clustering denoising may filter out part of the Target Object Point Cloud because the Target Object Point Cloud is connected to the noise Point Cloud

Parameter Description

Parameter Name	Description	Default Value	Value Range	Unit	Tuning Recommendation
Distance threshold for Point Cloud clustering (mm)	Determine whether Point Clouds in space belong to the same category. If the distance between Point Clouds is less than this threshold, they are considered the same category	5	[0.1, 1000]	mm	Generally does not need to be changed; it should be greater than the point spacing of the Target Object Point Cloud and less than the minimum distance between the Target Object Point Cloud and the noise Point Cloud
Minimum point count threshold	Point Cloud clusters with fewer points than this value will be filtered out	100	[1,10000000]	/	Generally does not need to be changed; increase the minimum point count threshold according to the noise size in the instance Point Cloud
Maximum point count threshold	Point Cloud clusters with more points than this value will be filtered out	100000	[1,10000000]	/	Generally does not need to be changed. If the number of points in the Target Object Point Cloud exceeds 100000, increase the maximum point count threshold
Whether to select the top Point Cloud in the ROI	If checked, the average Z coordinate of Point Clouds of the same category is calculated and sorted in the ROI coordinate system, and the Point Cloud category with the largest average Z coordinate (top Point Cloud) is retained; if unchecked, all Point Clouds that meet the conditions are retained	Unchecked	/	/	If the Target Object Point Cloud is above the noise Point Cloud, checking this option retains the Target Object Point Cloud; if the Target Object Point Cloud is below the noise Point Cloud, check this option and set the Z axis of the ROI coordinate system downward to retain the Target Object Point Cloud
Whether to visualize process data	If checked, the denoised Point Cloud is saved and can be found in C:_data	Unchecked	/	/	In debug mode, check this option if you need to save visualization data

Example

2.1.2 Point Cloud Downsampling

Function

Sample the Point Cloud according to the specified point spacing to reduce the number of computed points and improve model Inference speed, although accuracy may decrease

Use Cases

When there are too many Point Cloud points, check Point Cloud Downsampling to reduce the number of Point Cloud points. This is a default function for surface-type target object loading and unloading (materials isolated from each other) and cannot be deleted.

Parameter Description

Parameter	Description	Default Value	Parameter Range	Unit
Downsampling point spacing (mm)	Sample the Point Cloud according to the specified point spacing	5.0	[0.1, 1000]	mm

Parameter Tuning
- The larger the value of Downsampling point spacing, the fewer points remain after downsampling, the faster Pick Point calculation becomes, but accuracy may decrease
- The smaller the value of Downsampling point spacing, the more points remain after downsampling, the slower Pick Point calculation becomes, but accuracy improves

2.1.3 Calculate Normals

Function

Calculate Point Cloud Normals for use in the subsequent cylinder fitting process

Use Cases

Default function for surface-type target object loading and unloading (materials isolated from each other); cannot be deleted.

Parameter Description

Parameter Name	Description	Default Value	Value Range
Fix normal orientation	Whether to fix the orientation when calculating Normals. When enabled, Normal orientation is determined by the orientation reference vector	Checked	/
Neighborhood point count for Normal calculation	The larger the value, the more neighboring points are referenced, but local changes may be ignored; the opposite applies when the value is smaller	30	[1,200]
Orientation reference vector	Orientation reference vector for Normal calculation	[0,0,1]	/

Parameter Tuning

Cannot be changed

2.1.4 Point Cloud Contour Extraction

Function

Extract the Target Object contour from the instance Point Cloud

Use Cases

When using 2.2.4 **Enable Contour Mode **, Point Cloud Contour Extraction should also be checked

Parameter Description

Parameter Name	Description	Default Value	Value Range	Unit	Tuning Recommendation
Reference radius (mm)	Search radius for extracting contours from the instance Point Cloud	10	[0.1,10000000000]	mm	The reference radius is recommended to be set to 2.1.2half of the downsampling point spacing in Point Cloud Downsampling, and it must be greater than the point spacing
Point Cloud contour search mode	Mode for searching Point Cloud contours	Normal mode	Normal mode；Plane mode	/	Generally select Normal mode; for planar Target Objects, select Plane mode

Example

2.1.5 Filter Point Cloud by HSV (Hue, Saturation, Value)

Function

Filter the Point Cloud based on hue, saturation, and brightness in the Point Cloud image, and select Point Cloud regions matching the target range

Parameter Description

Parameter Name	Description	Default Value	Value Range
Filter depth by HSV - maximum color range value	Maximum color value for filtering the Point Cloud	[0.9,0.9,0.9]	[[0,0,0],[1,1,1]]
Filter depth by HSV - minimum color range value	Minimum color value for filtering the Point Cloud	[0.0,0.0,0.0]	[[0,0,0],[1,1,1]]

Example

2.1.6 Filter Point Cloud by Three-channel Color

Function

Filter the Point Cloud by three-channel color and select Point Cloud regions matching the target range

Parameter Description

Parameter Name	Description	Default Value	Value Range
Filter Point Cloud by three-channel color - maximum color value	Maximum color value for filtering the Point Cloud	[0.9,0.9,0.9]	[[0,0,0],[1,1,1]]
Filter depth by three-channel color - minimum color value	Minimum color value for filtering the Point Cloud	[0.0,0.0,0.0]	[[0,0,0],[1,1,1]]

Example

2.1.7 Select Point Clouds Within the ROI Region

Function

Select Point Clouds within the ROI 3D region from the instance Point Cloud. This default function cannot be deleted

Example

2.1.8 Remove Points Whose Normal Exceeds the Angle Threshold

Function

Remove Point Cloud points whose angle between the Normal vector and the axis direction of the reference Normal vector is greater than the Normal angle threshold

Use Cases

Surface-type target object loading and unloading (materials isolated from each other)

Parameter Description

Parameter Name	Description	Default Value	Value Range	Unit
Angle threshold	Point Clouds with an angle greater than this threshold are considered different instances	15	[-360, 360]
Reference Normal axis	The angle formed between the Point Cloud Normal and the axis direction of the reference Normal vector	Z axis	X/Y/Z axis	/
Use ROI coordinate system	If checked, the angle between the Normal and the axes of the ROI coordinate system is calculated; otherwise, the angle between the Normal and the axes of the Camera coordinate system is calculated	Unchecked	/	/

Parameter Tuning

2.1.9 Point Cloud Plane Segmentation

Function

Retain or remove the plane with the largest number of points in the instance Point Cloud

Use Cases

The instance Point Cloud contains a noisy plane

Parameter Description

Parameter	Description	Default Value	Value Range	Unit	Tuning Recommendation
Reference distance for plane fitting (mm)	If the distance from a point to the plane is less than the reference distance, it is considered an in-plane point; otherwise, it is considered an out-of-plane point	3	[0.001，10000]	mm	Generally unchanged
Remove plane	If checked, the plane with the largest number of Point Cloud points is removed; if unchecked, the plane with the largest number of Point Cloud points is retained	Unchecked	/	/	If the plane with the largest number of Point Cloud points is the Target Object, retain the plane and leave it unchecked; if it is noise, remove the plane and check this option

Example

2.1.10 Remove Outliers from the Point Cloud

Function

Identify and remove outlier noise in the Point Cloud to improve Point Cloud quality

Use Cases

The instance Point Cloud contains many outlier noise points. This is a default function for surface-type target object loading and unloading (materials isolated from each other) and cannot be deleted.

Parameter Description

Parameter Name	Description	Default Value	Value Range
Reference neighborhood point count	The number of neighboring points around each point in the Point Cloud, that is, the neighborhood size. For dense Point Clouds, even a small neighborhood is sufficient to reflect Target Object features, so a smaller value can be used; for sparse Point Clouds, a larger neighborhood is needed to reflect Target Object features, so a larger value should be used.	30	[1, 10000000]
Standard deviation multiplier	Used to identify outlier noise. If the deviation of a point's coordinates from the average coordinates of the instance Point Cloud exceeds the standard deviation multiplier, the point is considered an outlier. The smaller the value, the more points are treated as outliers and removed, but this may cause false judgments and remove important Target Object features; the larger the value, the fewer points are treated as outliers and removed, but some outliers may be retained and affect Target Object recognition accuracy.	0.005	[0.0001, 2]

Parameter Tuning

Generally unchanged. If the Point Cloud becomes too sparse after Remove Outliers from the Point Cloud, you should increase the standard deviation multiplier

Example

2.1.11 Filter Out Point Clouds with Out-of-limit Object Distance

Function

Filter Point Clouds in the specified direction to remove noise and improve image recognition accuracy

Parameter Description

Parameter	Description	Default Value	Parameter Range	Unit	Tuning Recommendation
Specified axis	The specified axis of the Point Cloud, used to filter Point Clouds in the specified direction	Z axis	X/Y/Z axis	/	Specified axis generally does not need to be changed
Threshold (mm)	Along the specified axis, if the distance between the lower-layer Point Cloud and the Target Object Point Cloud is greater than this threshold, the lower-layer Point Cloud will be filtered out; if the distance is less than this threshold, the lower-layer Point Cloud will be retained	750	[0, 1000]	mm	Adjust Threshold according to the actual scenario. The larger the Threshold, the fewer Point Clouds are filtered out; the smaller the Threshold, the more Point Clouds are filtered out
Select coordinate system	Filter Point Clouds in the selected coordinate system	ROI coordinate system	Camera coordinate system；ROI coordinate system；object coordinate system	/

Example

2.1.12 Point Cloud Downsampling (Target Object Point Cloud)

Feature Description

You can downsample the scene Point Cloud used for fine matching (different from coarse matching). When using it, it is recommended to add it and move it to the top of the preprocessing function list.

Use Cases

Scenarios where fine matching is used, but the processing time is high and does not meet the overall takt time.

Parameter Description

Parameter	Description	Default Value	Value Range	Unit
Downsampling point spacing (mm)	Sample the Point Cloud according to the specified point spacing to reduce the number of Point Cloud points and improve vision computation speed	1	[0.1，1000]	mm

The larger the value, the larger the downsampling point spacing, the fewer Point Cloud points remain after downsampling, and the faster vision computation becomes, but accuracy may decrease；

The smaller the value, the smaller the downsampling point spacing, the more Point Cloud points remain after downsampling, and the slower vision computation becomes, but accuracy may improve.

After running, the Point Cloud Template in the Target Object configuration can be updated to the downsampled Target Object Point Cloud in the historical data to appropriately reduce coarse matching time.

Example

2.1.13 [Expert] Read Instance Point Cloud

Feature Description

Read the instance Point Cloud

Use Cases

Surface-type Target Object scenarios

Parameter Description

Parameter	Description	Default Value	Value Range	Unit
Point Cloud path	Target Object Point Cloud path. If left blank, the Point Cloud uploaded on the Target Object page is used	/	/	/

2.1.14 Optimize the Mask Based on the Point Cloud

Function

Based on the Point Cloud within ROI 3D, remove Point Clouds from the Mask that are not within ROI 3D to improve Mask precision

2.2 Point Cloud Matching Pose Estimation

2.2.1 Model Point Cloud Downsampling Size (mm)

Function

Before coarse matching, the size of the sampling box used when downsampling the template Point Cloud

Use Cases

Parameter Description

Default value: 5

Value range: [0.001, 500]

Unit: mm

Parameter Tuning
- The larger the size, the fewer points are retained during downsampling, and the fewer points remain in the downsampled template Point Cloud
It is recommended that the number of points in the downsampled template Point Cloud be less than 300
- If the log reports the error "模型点数大于1000，请适当增大3D匹配中的模型点云降采样大小参数", you should increase Model Point Cloud Downsampling Size to reduce the number of points in the downsampled template Point Cloud；

2.2.2 Repeated Sampling Ratio Coefficient

Function

The degree of repeated sampling of point pairs in the processed template Point Cloud

Use Cases

Parameter Description

Default value: 1

Value range: [0.1,1]

Parameter Tuning
- The larger the value, the higher the success rate of coarse matching, but the longer the coarse matching time
- If the log reports the error "请调整3D匹配中降采样的点间距参数以及模型点云降采样大小、重复采样比例系数等参数, 详情参考CPFV调参指南", you should increase Repeated Sampling Ratio Coefficient. It is generally recommended to start adjusting from 1. A value of 1 is the slowest but gives the highest fine matching success rate, while 0.1 is the fastest but gives the worst fine matching success rate.
- If the log reports the error "触发超时警告，请参考CPFV调参指南修改参数", you should decrease Repeated Sampling Ratio Coefficient

2.2.3 Pose Angle Prior

Function

Before coarse matching, provide the approximate orientation of the Target Object in the ROI coordinate system in advance to improve the accuracy of coarse matching

Use Cases

Surface-type Target Object ordered loading/unloading, surface-type Target Object positioning and assembly, and surface-type Target Object positioning and assembly (matching only). Due to the rotational symmetry of the surface-type Target Object itself, coarse matching and fine matching may perform poorly

Not applicable to surface-type Target Object unordered picking scenarios

Parameter Description

The value consists of 4 digits. The first 3 digits represent the direction of the Target Object, and the 4th digit uses 0/1/2 to represent the X/Y/Z axis of the ROI coordinate system respectively

Default value: []

Value range: [1,0,0,0] indicates the positive X-axis direction, and [-1,0,0,0] indicates the negative X-axis direction；

[0,1,0,1] indicates the positive Y-axis direction, and [0,-1,0,1] indicates the negative Y-axis direction；

[0,0,1,2] indicates the positive Z-axis direction, and [0,0,-1,2] indicates the negative Z-axis direction。

Format: [[1,0,0,0]], with two sets of square brackets; directions on the XYZ axes can be combined, for example [[1,0,0,0],[0,1,0,1],[0,0,1,2]] indicates the positive X-axis direction, positive Y-axis direction, and positive Z-axis direction

Example

Determine it according to the orientation of the surface-type Target Object. As shown below, the surface-type Target Object faces the positive X-axis direction, so Pose Angle Prior can be set to [[1,0,0,0]]

2.2.4 Enable Contour Mode

Function

Extract the contour of the instance Point Cloud and use the contour of the downsampled template Point Cloud and the contour of the downsampled instance Point Cloud for coarse matching

Use Cases

Surface-type Target Object ordered loading/unloading, surface-type Target Object unordered picking, surface-type Target Object positioning and assembly, and surface-type Target Object positioning and assembly (matching only). Because the contour features of the Target Object are relatively obvious, using contour mode can avoid poor coarse matching caused by the fact that surface-type Target Objects are prone to slipping

When using contour mode, 2.1.4 Point Cloud Contour Extraction should also be checked to extract contours from the instance Point Cloud

Example
- When using contour mode, the log will display the message "使用轮廓模式，找到轮廓点数量"

Contour Mode

Function

Extract the contour of the template Point Cloud, and use the contour of the downsampled template Point Cloud and the contour of the downsampled instance Point Cloud for coarse matching

Use Cases

Surface-type Target Object ordered loading/unloading, surface-type Target Object unordered picking, surface-type Target Object positioning and assembly, and surface-type Target Object positioning and assembly (matching only). If coarse matching performs poorly, check this function to use contour Point Cloud for another round of coarse matching

Parameter Tuning

Normal mode: default value

Plane mode: planar Target Objects should use Plane mode

Example

2.2.5 Coarse Registration Contour Search Radius

Function

When using the contour of the downsampled template Point Cloud and the contour of the downsampled instance Point Cloud for coarse matching, this is the search radius for extracting contour Point Clouds in the template Point Cloud and the instance Point Cloud

Use Cases

Parameter Description

Default value: 0.005

Value range: [0.0001,0.5]

Unit: m

2.2.6 Object Pose Correction

Fine Matching Search Radius (mm)

Function

During fine matching, the template Point Cloud is matched with the instance Point Cloud, and each point in the template Point Cloud needs to search for the nearest point in the instance Point Cloud. The fine matching search radius represents both the search radius in the instance Point Cloud and the distance threshold between each point in the template Point Cloud and its nearest point in the instance Point Cloud. If the distance between a point and its nearest point is less than the fine matching search radius, the two points are considered matchable; otherwise, they are considered not matchable.

Use Cases

Surface-type Target Object ordered loading/unloading, surface-type Target Object unordered picking, and surface-type Target Object positioning and assembly scenarios

Parameter Description

Default value: 10

Value range: [1, 500]

Unit: mm

Parameter Tuning

Usually unchanged

Fine Matching Search Mode

Function

How the template Point Cloud retrieves nearest points in the instance Point Cloud during fine matching

Use Cases

If fine matching between the template Point Cloud and the instance Point Cloud performs poorly, this function should be adjusted

Parameter Description

Parameter	Description
Point-to-point	Each point in the template Point Cloud searches for the nearest point in the instance Point Cloud (the point with the shortest straight-line distance within the search radius), which is suitable for all Target Objects
Point-to-plane	Each point in the template Point Cloud searches for the nearest point in the instance Point Cloud along its Normal, which is suitable for Target Objects with obvious geometric features
Combination of point-to-point and point-to-plane	First use point-to-point mode to optimize the Target Object pose in the instance Point Cloud, then use point-to-plane mode to optimize the Target Object pose in the instance Point Cloud. This is suitable for Target Objects with obvious geometric features Using this method increases takt time

Use Contour Mode

Function

Extract contour Point Clouds from the template Point Cloud and instance Point Cloud for coarse matching

Use Cases

In surface-type Target Object ordered loading/unloading, unordered picking, and positioning/assembly scenarios, if coarse matching based on keypoints performs poorly, this function should be checked to use contour Point Clouds for another round of coarse matching

Parameter Tuning

The result of coarse matching affects the result of fine matching. If the fine matching result is poor, check Use Contour Mode

Contour Search Range (mm)

Function

The search radius for extracting contour Point Clouds in the template Point Cloud and instance Point Cloud

Use Cases

Applicable to general Target Object ordered loading/unloading, general Target Object unordered picking, and general Target Object positioning/assembly scenarios

Parameter Description

Default value: 5

Value range: [0.1, 500]

Unit: mm

Parameter Tuning

A smaller value means a smaller radius for searching contour Point Clouds, which is suitable for extracting detailed Target Object contours, but the extracted contours may include outlier noise；

A larger value means a larger radius for searching contour Point Clouds, which is suitable for extracting broader Target Object contours, but some detailed features may be ignored.

Save Pose Estimation[Fine Matching] Data

Function

If checked, fine matching data is saved

Use Cases

Example

The fine matching data is saved in \Project Folder\data\PickLight\Historical Data Timestamp\Builder\pose\output folder under the project save path.

2.2.7 Axis-based Rotation Pose Adjustment

Function

Rotate and translate the instance Point Cloud around the given axis based on the first Pick Point of the Target Object, calculate the matching score between the instance Point Cloud and the template Point Cloud after each rotation and translation, and select the instance Point Cloud with the highest matching score as the final pose of the rotationally symmetric Target Object.

Use Cases

Deviation occurs when matching the instance Point Cloud and template Point Cloud of a rotationally symmetric Target Object, and full matching can be achieved only after rotating by a certain angle

Cannot be used together with the recognition type, identify front/back (by Point Cloud Template), or identify local feature options in the feature options

Parameter Description

Parameter	Description	Default Value	Parameter Range	Tuning Recommendation
Rotation angle interval	Rotate the instance Point Cloud at equal angle intervals. The angle difference between two adjacent rotations. For example, if the first rotation is 30°, the second is 60°, and the third is 90°, then the rotation angle interval is 30°.	5	\[1, 180\]	If the Target Object has many features and matching is difficult and high-precision matching is required, set a smaller angle interval to perform more matches, but the computation increases. If the Target Object shape is simple and has few features, set a larger angle interval to improve computational efficiency.
Rotation angle range	How large an angle range the instance Point Cloud can rotate from its initial state.	90	\[1, 180\]	The Point Cloud rotation range is the positive and negative interval of the value. For example, if the angle range is set to 120°, then the Point Cloud rotation range is \[-120°,120°\], and the instance Point Cloud rotates from -120° to 120° according to the angle interval. The number of rotations = rotation angle range / rotation angle interval. Common values for the rotation angle range are 180, 120, and 90, set according to the characteristic angle of the rotationally symmetric Target Object. For example, if an equilateral triangle Target Object overlaps with itself after rotating 120° then the rotation angle range should be ≥120°. If it is set to 180°, more rotated instance Point Clouds will be matched with the template Point Cloud. For example, if a regular quadrilateral Target Object overlaps with itself after rotating 90°, then the rotation angle range should be ≥90°. If Pose Angle Prior is filled in and the approximate orientation of the Target Object is already known, the rotation angle range can be reduced according to the possible deviation range to reduce computation.
Template Point Cloud file path	Upload the template Point Cloud file of the Target Object. If not uploaded, the Point Cloud Template uploaded on the Target Object page is used	/	/
Evaluation mode	Evaluate the quality of the matching result from different perspectives	Iso-target	Iso-target；Iso-source；Average；Strict；Loose；Fast	Iso - target: Evaluate the matching result from the perspective of the template Point Cloud, checking whether the features of the template Point Cloud are well matched. Iso - source: Evaluate the matching result from the perspective of the instance Point Cloud, checking whether the changes in the instance Point Cloud during matching and the features after matching are reasonable. Average: Comprehensively consider factors such as errors and fitting quality of the template Point Cloud and instance Point Cloud during the matching process, and provide an overall evaluation result. Strict: Evaluate the matching result in a stricter way. Only when the Point Cloud matching degree is very high will the matching result be considered good. Suitable for scenarios with many Target Object features and high matching accuracy requirements, such as precision parts. Loose: The evaluation is relatively lenient and allows a certain range of matching error. Suitable for scenarios with lower accuracy requirements and a preference for obtaining matching results quickly. Fast: Emphasizes rapid evaluation and is suitable for scenarios with high matching speed requirements and low matching accuracy requirements.
ICP threshold	The criterion for determining whether registration is successful. If the matching result error is less than this threshold, the match is successful; otherwise, it is unsuccessful	0.005	\[0.000001, 1\]	Generally unchanged. If the actual scenario requires higher matching accuracy, decrease this threshold; if the actual scenario requires higher matching speed and lower accuracy, increase this threshold. If the Target Object Point Cloud quality is good, decrease this threshold; if it is poor, increase this threshold.
Rotation axis selection	The axis around which the instance Point Cloud rotates	Z axis	X/Y/Z axis	Generally unchanged. If a certain axis of the rotationally symmetric Target Object is important for recognizing the Target Object pose, set the rotation axis to that feature axis
Save visualization data	Whether to save visualization data	Unchecked	/
Use edges for optimization	Use the edge contour of the Target Object to optimize matching, reduce matching result errors, and make the match between the instance Point Cloud and the template Point Cloud more refined	Unchecked	/	If the edge contour of the Target Object has unique geometric features, check this function to improve matching accuracy. For example, for Target Objects with complex shapes and large contour differences, the Point Cloud of the edge contour can more accurately recognize the Target Object pose.
Edge Point Cloud file path	Upload the edge Point Cloud file of the Target Object. If not uploaded, the edge Point Cloud is extracted from the Point Cloud Template uploaded on the Target Object page	/	/
Optimize rotation result	After the best pose is found during matching, optimize it again to reduce matching result errors and make the match between the instance Point Cloud and the template Point Cloud more refined	Checked	/	Checked by default to improve matching accuracy and generally does not need to be changed
Optimization mode	Mode for optimizing the matching result	Point	Point；Plane；Full	Point (point mode): Optimization is based on point-to-point correspondence, calculating point distance errors and iteratively adjusting them. Suitable for Target Objects with simple Point Clouds and few features. Plane (plane mode): The planar features of the Point Cloud are considered, and the point-to-plane relationship is included in registration calculation. Suitable for Target Objects with many planes, such as plate-like Target Objects. Full (full mode): Registration optimization comprehensively considers multiple geometric features such as points, lines, and planes, making fuller use of Point Cloud information but with relatively higher computation. Suitable for Target Objects with complex Point Clouds, rich geometric features, and high accuracy requirements.
Optimization threshold	During optimization, this is the criterion for determining whether registration has reached the expected accuracy. If the registration error is less than this threshold, optimization succeeds; otherwise, optimization fails and iteration needs to continue.	0.002	\[0.0001, 1\]	Generally unchanged. Decrease this threshold for scenarios requiring higher registration accuracy; increase it for scenarios requiring lower registration accuracy.
Enable center movement mode	When enabled, translation mode is also added, so translation is superimposed while rotating the instance Point Cloud	Unchecked		If the registration result has an offset along an axis, center movement mode can be enabled to optimize the registration result Note that enabling center movement mode requires the evaluation mode to be set to Fast
x-axis movement range	Range of movement along the Pick Point x-axis in center movement mode (mm)	0	\[0,100\]	The default value is 0, which means the x axis does not move during evaluation; a non-zero value means x-axis movement is enabled Note that the specified axis direction refers to the direction before Pick Point direction adjustment. Functions in the Pick Point processing function may change the axis direction. You can disable the Pick Point processing function to check the correct axis direction
x-axis movement step	Step size of movement along the Pick Point x-axis in center movement mode (mm)	2	\[0.01,10\]	The step size should be set reasonably according to the offset and required accuracy. Setting too small a step size will significantly increase takt time
y-axis movement range	Range of movement along the Pick Point y-axis in center movement mode (mm)	0	\[0,100\]	The default value is 0, which means the y axis does not move during evaluation; a non-zero value means y-axis movement is enabled Note that the specified axis direction refers to the direction before Pick Point direction adjustment. Functions in the Pick Point processing function may change the axis direction. You can disable the Pick Point processing function to check the correct axis direction
y-axis movement step	Step size of movement along the Pick Point y-axis in center movement mode (mm)	2	\[0.01,10\]	The step size should be set reasonably according to the offset and required accuracy. Setting too small a step size will significantly increase takt time
z-axis movement range	Range of movement along the Pick Point z-axis in center movement mode (mm)	0	\[0,100\]	The default value is 0, which means the z axis does not move during evaluation; a non-zero value means z-axis movement is enabled Note that the specified axis direction refers to the direction before Pick Point direction adjustment. Functions in the Pick Point processing function may change the axis direction. You can disable the Pick Point processing function to check the correct axis direction
z-axis movement step	Step size of movement along the Pick Point z-axis in center movement mode (mm)	2	\[0.01,10\]	The step size should be set reasonably according to the offset and required accuracy. Setting too small a step size will significantly increase takt time

Example

Axial matching offset

After enabling center movement mode

Description of the number of poses evaluated by axis-based rotation pose adjustment

Total number of pose calculations = number of x-axis movement points * number of y-axis movement points * number of z axis movement points * number of angle points

Excessive movement range or too small a step size will increase the total number of poses to be evaluated and lengthen takt time, so the range and step parameters must be set reasonably

2.3 Empty ROI Determination

Function

Determine whether there are any Target Objects (Point Clouds) remaining in ROI 3D. If the number of 3D points in ROI 3D is less than this value, it indicates that no Target Object Point Cloud remains, and no Point Cloud is returned in this case

Parameter Description

Default value: 1000

Value range: [0, 100000]

Usage Process

Set the minimum point count threshold for ROI 3D. If it is less than this threshold, the Target Object Point Cloud in ROI 3D is insufficient, and it is therefore judged that there is no Target Object in ROI 3D；

In the robot configuration, add a new vision status code to facilitate subsequent robot signal processing.

3. Pick Point Processing

This section mainly explains functions related to Pick Point filtering and adjustment, along with Parameter tuning recommendations.

3.1 Pick Point Adjustment

3.1.1 Rotate the Picking Pose when it is outside the angle range

Function description

When the Picking Pose is outside the configured angle range, it is rotated counterclockwise by a certain angle around a fixed axis. If it is still outside the configured angle range after rotation, a warning is issued.

Usage scenario

This function is only applicable to depalletizing scenarios. It can keep the robot's approach direction stable during picking and prevent the end effector from repeatedly rotating during the picking process. In 180° cases, it can prevent exceptions such as cable twisting.

Parameter description

Parameter	Description	Default	Range	Unit
Fixed axis	An axis of the Picking Pose. The pose is rotated counterclockwise around this fixed axis	Z-axis	X/Y/Z-axis	/
Rotation angle	The angle by which the pose is rotated counterclockwise around the fixed axis. Adjust this angle so the Picking Pose satisfies the angle range	0	[-360,360]	degree
Angle range	The angle range of the Picking Pose. Set the angle range according to factors such as material placement, end effector type, and cycle time	[0,180]	[-180,180]	degree
Use current robot Euler Angles	By default, pose calculation uses Euler Angles "XYZ". When selected, the Euler Angles configured for the current robot are used so the pose remains consistent with the robot teach pendant.	Unchecked	/	/
Custom coordinate system	The coordinate system of the Picking Pose	Robot arm coordinate system	Default coordinate system; camera coordinate system; ROI coordinate system; robot arm coordinate system	/

Example

Without using this function, the generated Pick Points are shown below.

When this function is used with the default values, the RZ angles of the Picking Poses for instances 0, 1, and 2 are all within the angle range [0,180], so no processing is performed. The RZ angle of the Picking Pose for instance 4 is -90°, which is outside the angle range [0,180], so the Picking Pose of instance 4 is rotated by 0° around the fixed Z-axis.

If you want to adjust the RZ angle of the Picking Pose for instance 4 into the angle range, you can change the rotation angle to 180 and rotate the Picking Pose of instance 4 by 180° around the fixed Z-axis.

3.1.2 Rotate the Picking Pose so the rotation axis direction matches the target axis direction

Function description

Rotate the Picking Pose once around the fixed axis so that the direction of the rotation axis (determined by the right-hand rule) matches the positive or negative direction of the target axis in the target coordinate system.

Usage scenario

Avoid collisions between the robot end effector and the bin.

Parameter description

Parameter	Description	Default	Range
Rotation axis	An axis of the Picking Pose. Determined by the right-hand rule, the Picking Pose is rotated counterclockwise once around the fixed axis so that the direction of the rotation axis matches the positive or negative direction of the target axis in the target coordinate system	X-axis	X/Y/Z-axis
Fixed axis	The Picking Pose is rotated counterclockwise once around the fixed axis so that the direction of the rotation axis matches the positive or negative direction of the target axis in the target coordinate system	Z-axis	X/Y/Z-axis
Target axis	An axis of the target coordinate system. The Picking Pose is rotated counterclockwise once around the fixed axis so that the direction of the rotation axis matches the positive or negative direction of the target axis in the target coordinate system	X-axis	X/Y/Z-axis
Negative target axis direction	If selected, the direction of the rotation axis is aligned with the negative direction of the target axis in the target coordinate system; otherwise, it is aligned with the positive direction of the target axis in the target coordinate system	Unchecked	/
Custom coordinate system	The coordinate system of the Picking Pose	Default coordinate system	Default coordinate system; camera coordinate system; ROI coordinate system; robot arm coordinate system

Example

3.1.3 Rotate the Picking Pose so the angle between the rotation axis and the target axis is minimized

Function description

Rotate the Picking Pose around the fixed axis by 0, 90, 180, and 270 degrees respectively, calculate the angle between the rotated rotation axis and the positive or negative direction of the target axis in the camera coordinate system, and finally output the Picking Pose with the smallest angle after rotation.

Usage scenario

Avoid collisions between the robot end effector and the bin.

Parameter description

Parameter	Description	Default	Range
Fixed axis	An axis of the Picking Pose. Rotate the pose counterclockwise around this fixed axis	Z-axis	X/Y/Z-axis
Rotation axis	An axis of the Picking Pose. When rotating the pose, calculate the angle between this rotation axis and the positive or negative direction of the target axis	X-axis	X/Y/Z-axis
Target axis	An axis of the camera coordinate system. When rotating the pose, calculate the angle between the rotation axis and the positive or negative direction of this target axis	X-axis	X/Y/Z-axis
Negative target axis direction	If selected, calculate the angle between the rotation axis and the negative direction of the target axis; otherwise, calculate the angle between the rotation axis and the positive direction of the target axis	Selected	/
Custom coordinate system	The coordinate system of the Picking Pose	Default coordinate system	Default coordinate system; camera coordinate system; ROI coordinate system; robot arm coordinate system

Example

3.1.4 Flip the Picking Pose so the angle between the rotation axis and the target axis is minimized

Function description

Rotate the Picking Pose once around the fixed axis so that the angle formed between the rotation axis and the positive or negative direction of the target axis in the ROI coordinate system is acute.

Usage scenario

Avoid collisions between the robot end effector and the bin.

Parameter description

Parameter	Description	Default	Range
Fixed axis	An axis of the Picking Pose. Rotate the Picking Pose counterclockwise around this fixed axis	Z-axis	X/Y/Z-axis
Rotation axis	An axis of the Picking Pose. Rotate the Picking Pose so that the direction of this rotation axis matches the positive or negative direction of the target axis	X-axis	X/Y/Z-axis
Target axis	An axis in the ROI coordinate system. Rotate the Picking Pose so that the direction of the rotation axis matches the positive or negative direction of this target axis	X-axis	X/Y/Z-axis
Negative target axis direction	If selected, rotate the Picking Pose so that the direction of the rotation axis matches the negative direction of the target axis; otherwise, rotate the Picking Pose so that the direction of the rotation axis matches the positive direction of the target axis	Selected	/
Custom coordinate system	The coordinate system of the Picking Pose	Default coordinate system	Default coordinate system; camera coordinate system; ROI coordinate system; robot arm coordinate system

Example

3.1.5 Point a Picking Pose axis toward the ROI center

Function

Rotate the Picking Pose around a fixed axis so that the pointing axis of the Picking Pose points to the ROI center.

Usage scenario

Avoid collisions between the robot end effector and the bin.

Parameter description

Parameter	Description	Default	Range
Pointing axis	The axis in the Picking Pose that needs to be adjusted	X-axis	X/Y/Z-axis
Fixed axis	The axis that remains unchanged during rotation	Z-axis	X/Y/Z-axis
Reverse align	If selected, reverse-align the pointing axis to the ROI center; otherwise, align the pointing axis to the ROI center	Selected	/
Strict pointing	If selected, force the Picking Pose to rotate so the pointing axis points to the ROI center	Unchecked	/
Custom coordinate system	The coordinate system of the Picking Pose	Default coordinate system	Default coordinate system; camera coordinate system; ROI coordinate system; robot arm coordinate system

Example

3.1.6 Rotate the Picking Pose so the Z-axis direction matches the Z-axis of the target coordinate system

Function description

Rotate the Picking Pose so that its Z-axis direction matches the Z-axis of the target coordinate system.

Usage scenario

Usually this is used by default only in depalletizing scenarios and cannot be deleted. It is used to make the Z-axis of the Picking Pose perpendicular to the Z-axis of the ROI coordinate system (4-axis) or consistent with the direction of the Target Object surface (6-axis).

Parameter description

Parameter	Description	Default	Range
Robot configuration	Set according to the on-site robot configuration. You can choose 4-axis or 6-axis. If a 6-axis robot is actually used as a 4-axis robot, it should be set to 4-axis	4-axis	4-axis/6-axis
Use ROI Z-axis as target direction	When the robot configuration is set to 4-axis, if selected, the pose is rotated around the X-axis so that the Z-axis direction of the rotated pose matches the positive direction of the ROI Z-axis ; if not selected, the pose is rotated around the X-axis so that the Z-axis direction of the rotated pose matches the positive direction of the Z-axis of the camera coordinate system . When the robot configuration is set to 6-axis, regardless of whether it is selected, the pose is rotated around the X-axis so that the Z-axis direction of the rotated pose matches the positive direction of the Z-axis of the object's own coordinate system	Unchecked	/
Custom coordinate system	The coordinate system of the Picking Pose	Camera coordinate system	Default coordinate system; camera coordinate system; ROI coordinate system; robot arm coordinate system

Example

3.1.7 Rotate the Picking Pose around a fixed axis

Function description

Rotate the Picking Pose by a certain angle around a fixed axis.

Usage scenario

Avoid collisions between the robot end effector and the bin.

Parameter description

Parameter	Description	Default	Range	Unit
Rotation angle	The angle by which the pose is rotated counterclockwise around the fixed axis	90	[-360, 360]	degree°
Fixed axis	An axis of the Picking Pose. Rotate the pose counterclockwise around this fixed axis	Z-axis	X/Y/Z-axis	/
Custom coordinate system	The coordinate system of the Picking Pose	Default coordinate system	Default coordinate system; camera coordinate system; ROI coordinate system; robot arm coordinate system	/

Example

3.1.8 Translate the Picking Pose

Function description

Move the Picking Pose by a certain distance along the translation axis.

Usage scenario

Avoid collisions between the robot end effector and the bin.

Parameter description

Parameter	Description	Default	Range	Unit
Translation amount (mm)	The distance the Picking Pose moves along the translation axis. A positive translation amount means translating in the positive direction of the translation axis, and a negative translation amount means translating in the negative direction of the translation axis	0	[-1000, 1000]	mm
Translation axis	The direction in which the Picking Pose moves	X-axis	X/Y/Z-axis	/
Custom coordinate system	The coordinate system of the Picking Pose	Robot arm coordinate system	Default coordinate system; camera coordinate system; ROI coordinate system; robot arm coordinate system	/

Example

3.1.9 Pick Point Teaching

Function description

Record the Pick Point coordinates generated by the software and the Pick Point coordinates taught under the current operating condition, then output the transformed Picking Pose based on the offset between the two.

Usage scenario

When the Pick Points generated by the vision system have an obvious systematic offset and the robot TCP coordinate accuracy is limited or difficult to calibrate, this method can be used to directly map the same offset pattern to subsequent Pick Points, thereby avoiding robot TCP calibration.

Parameter description

Parameter	Description	Default	Range
Vision Pose	Pick coordinates of the detection result
X(mm)	X coordinate of the Vision Pose	0.00	±10000000, meaning no limit.
Y(mm)	Y coordinate of the Vision Pose	0.00	±10000000, meaning no limit.
Z(mm)	Z coordinate of the Vision Pose	0.00	±10000000, meaning no limit.
RX(°)	X-axis rotation amount of the Vision Pose	0.00	±180
RY(°)	Y-axis rotation amount of the Vision Pose	0.00	±180
RZ(°)	Z-axis rotation amount of the Vision Pose	0.00	±180
Picking Pose	Manually taught Pick Point
X(mm)	X coordinate of the Picking Pose	0.00	±10000000, meaning no limit.
Y(mm)	Y coordinate of the Picking Pose	0.00	±10000000, meaning no limit.
Z(mm)	Z coordinate of the Picking Pose	0.00	±10000000, meaning no limit.
RX(°)	X-axis rotation amount of the Picking Pose	0.00	±180
RY(°)	Y-axis rotation amount of the Picking Pose	0.00	±180
RZ(°)	Z-axis rotation amount of the Picking Pose	0.00	±180

3.1.10 Refine Object Pose based on plane Normal

Function description

Correct the Object Pose by fitting the plane Normal so that the Z-axis direction of the Object Pose remains consistent with the direction of the plane Normal of the Target Object.

Usage scenario

When the Target Object contains a plane and there is a tilt deviation in the plane when the template Point Cloud is matched with the actual Point Cloud, use this function to fine-tune the Target Object plane and improve picking accuracy.

Not applicable to depalletizing scenarios

Parameter description

Parameter	Description	Default	Range	Unit
Distance Threshold	Distance Threshold for fitting a plane from the Point Cloud	10	[-1000, 1000]	mm
Save visualization data	If selected, the visualization data will be saved under the historical data timestamp	Selected	/	/
Custom coordinate system	The coordinate system of the Picking Pose	Camera coordinate system	Default coordinate system; camera coordinate system; ROI coordinate system; robot arm coordinate system	/

Example

3.1.11 Sort Pick Points by inter-axis angle

Function

Sort Pick Points according to the angle between an axis of the Picking Pose and the target axis of the ROI.

Parameter description

Parameter	Description	Default	Range
Axis selection	An axis of the Picking Pose	Z-axis	X/Y/Z-axis
Target axis selection	An axis of the ROI coordinate system	Z-axis	X/Y/Z-axis
Select reverse direction	If selected, calculate the angle with the negative direction of the target axis; otherwise, calculate the angle with the positive direction of the target axis	Unchecked	/
Select descending order	If selected, sort Pick Points from small to large by angle; otherwise, sort Pick Points from large to small by angle	Unchecked	/

3.1.12 [Advanced] Rotate the Picking Pose and automatically compensate for excessive angles to the specified axis

Function description

Determine whether the angle formed between the specified axis of the Picking Pose and the target axis is within the specified range. If not, adjust the Picking Pose into the specified range.

Usage scenario

Avoid collisions between the robot end effector and the bin.

Parameter description

Parameter	Description	Default	Range	Unit
Angle range	Adjust the Picking Pose into the angle range	30	[0, 180]	degree°
Specified axis	An axis of the Picking Pose. Adjust this axis so that it falls within the angle range relative to the target axis of the ROI coordinate system	Z-axis	X/Y/Z-axis	/
Target axis	An axis of the ROI coordinate system. Compare the angle range with the specified axis of the Picking Pose	Z-axis	X/Y/Z-axis	/
Compare with the negative half-axis of the ROI	If not selected, compare the angle range with the positive direction of the target axis of the ROI coordinate system; if selected, compare the angle range with the negative direction of the target axis of the ROI coordinate system	Unchecked	/	/
Custom coordinate system	The coordinate system of the Picking Pose	Default coordinate system	Default coordinate system; camera coordinate system; ROI coordinate system; robot arm coordinate system	/

3.1.13 [Advanced] Symmetry center Object Pose optimization

Function

Search for the symmetry center of the Target Object based on the instance Mask, then combine it with the plane of the instance or the pose of the ROI 3D center point to calculate the optimal Picking Pose.

Before using this function, first make sure the instance Mask is symmetrical

Usage scenario

Applicable when the instance Mask of a symmetrical Target Object is also symmetrical, but the Picking Pose is not near the expected center; at the same time, the Target Object has a plane that can be used as a reference, for example, there is a plane on the top of the object, or ROI 3D can be used as a reference for the projected pose.

Applicable project scenarios include brake discs (general circles), refractory bricks (depalletizing), symmetrical irregular parts, fuel fillers, and so on.

Parameter description

Parameter	Description	Default	Range	Tuning recommendation
Target Object Symmetry type	Target Object Symmetry type of the instance Mask	Rotational symmetry	Rotational symmetry: after the Target Object rotates by a certain angle around the center point, its shape completely overlaps with the original position; mirror symmetry: the Target Object uses a certain axis / plane as the mirror, and the left-right or upper-lower sides are completely symmetrical.	Circles and rectangles are both rotationally symmetrical and mirror-symmetrical, so rotational symmetry is preferred; for trapezoids and other shapes that are symmetrical only along a certain axis or plane, choose mirror symmetry.
Gaussian blur level	Tolerance for determining whether the actual Point Cloud overlaps after rotation	3	\[1,99\]	The larger the value, the stronger the blur, the more smoothing is applied, and the more lenient the overlap judgment after rotation; The smaller the value, the weaker the blur, the less smoothing is applied, and the stricter the overlap judgment. An odd number must be entered
Rotation angle setting	When the symmetry mode is rotational symmetry, it indicates the rotation angle interval, that is, the angle difference between two adjacent rotations. When the symmetry mode is mirror symmetry, it indicates the rotation range, that is, the angle interval within which the Point Cloud can rotate around the symmetry axis.	180	\[1,360\]	For rotational symmetry, set it based on the symmetry of the Target Object. A circle is symmetrical at any angle, so 60 degrees is recommended; A square has 90-degree symmetry, so it can be set to 90 degrees or 180 degrees; A rectangle has 180-degree symmetry, so set it to 180. For mirror symmetry, adjust it within 5° to ensure fine tuning near the symmetry axis
Image scaling ratio	Adjusts the size of the Point Cloud image. The larger this ratio, the smaller the Point Cloud image size and the lower the GPU memory usage, but image detail loss increases, resulting in reduced calculation accuracy .	2	\[1,10000000\]
Search range	Based on the initially determined center of the Target Object, this defines the range expanded outward to search for Point Cloud features. The actual range is (search range2image scaling ratio)	10	\[1,10000000\]	For example, for a square Target Object, the initially determined center position of the Target Object is point O. If the search range is set to 10 and the image scaling ratio is 1, then the actual search range is a square region centered at point O with a side length of 10×2×1=20. Point Cloud features are searched within this region to further determine the symmetry center of the Target Object and the optimal Picking Pose. As another example, for a circular Target Object, if the search range is set to 8 and the image scaling ratio is 2, then the actual search range is a circular region centered at the initially determined center of the Target Object with a diameter of 8×2×2=32. Point Cloud features are searched within this region to further determine the Object Pose of the Target Object and the optimal Picking Pose.
Use ROI3D as the reference projection plane	If selected, ROI3D is used as the reference projection plane	Unchecked	/	Select this when the Point Cloud has no obvious plane and the projection plane is difficult to determine; leave it unchecked when the Point Cloud has a clear plane.
Save symmetry center process data	If selected, the debug data generated during the symmetry center process is saved. You can view it in the `\ProjectName`\data`\PickLight`\HistoricalDataTimestamp`\find`\_symmetry_center folder	Unchecked	/	Select this when you need to inspect the detailed process images
Symmetry axis prior type	Effective in ``{=html}mirror symmetry``{=html} mode. Specifies the known Target Object Symmetry type and fixes the asymmetric orientation	Automatic search	Automatic searchSymmetric along the long axisSymmetric along the short axis	If the symmetry axis of the Target Object is the long axis, choose "Symmetric along the long axis". If the symmetry axis of the Target Object is the short axis, choose "Symmetric along the short axis". If uncertain, choose "Automatic search"
Pose adjustment type	Whether to inherit pose-related information from the input pose	Default pose	Default poseInherit rotationInherit translation	/
Symmetry score Threshold	Symmetry results with a symmetry score lower than this Threshold are abnormal results. When set to 0, no filtering is performed	0.0	\[0.0, 1.0\]	/

Example

3.2 Pick Point Filtering

3.2.1 Filter by fine matching score

Function description

Filter Pick Points based on the pose fine matching score.

Parameter description

Parameter	Description	Default	Range
Score Threshold	Retain Pick Points whose fine matching score is greater than this Threshold	0.5	[0, 1]

Example

3.2.2 Filter Pick Points of occluded Target Objects

Function description

Determine whether there are too many occluding object Point Clouds in the target detection area along the specified ROI axis or the Picking Pose axis at the Pick Point of the grasped Target Object. If so, the Target Object is considered occluded and the Pick Point is filtered out.

Usage scenario

Applicable to depalletizing and ordered scenarios in which Target Objects are picked layer by layer, but the model recognizes lower-layer Target Objects. When picking a lower-layer Target Object, the gripper may collide with the upper-layer Target Object.

Parameter description

Parameter	Description	Default	Range	Unit
Cuboid length in X direction	Set the cuboid length in the X direction of the Picking Pose	1500	[1, 10000]	mm
Cuboid length in Y direction	Set the cuboid length in the Y direction of the Picking Pose	1500	[1, 10000]	mm
Cuboid length in Z direction	Set the cuboid length in the Z direction of the Picking Pose	800	[1, 10000]	mm
Distance Threshold between detection area and Pick Point origin	Along the ROI axis, the nearby cuboid surface area farther than this distance Threshold from the Pick Point origin is regarded as the target detection area	50	[1, 1000]	mm
Point Cloud count Threshold in detection area	If the number of occluding object Point Clouds in the target detection area exceeds this Threshold, the Pick Point is considered occluded	1000	[0, 100000]	/
Specified axis direction	Based on the pose reference specified axis direction, set the specific location of the target detection area within the cuboid space (for example, near the front/back/left/right/top/bottom surface of the cuboid)	[0,0,-1]	[1,0,0]: positive X-axis[-1,0,0]: negative X-axis[0,1,0]: positive Y-axis[0,-1,0]: negative Y-axis[0,0,1]: positive Z-axis[0,0,-1]: negative Z-axis	/
Use ROI 3D pose reference	If selected, adjust the collision detection area according to the ROI 3D pose reference	Unchecked	/	/
Save visualization data	If selected, the visualization data is stored according to the saved data path to help observe whether the generated cuboid is reasonable; if not selected, it is not saved	Unchecked	/	/

Example

3.2.3 Filter by Picking Pose angle range

Function description

Determine whether the angle of the Picking Pose is within the constrained angle range, and filter out all Pick Points that do not meet the condition.

Usage scenario

Prevent collisions caused by abnormal robot arm Picking Pose angles.

Parameter description

Parameter	Description	Default	Range	Unit
Angle filtering Threshold	Calculate the maximum angle between the specified axis of the ROI and the specified axis of the Picking Pose. Pick Points whose angle is greater than the current Threshold will be filtered out	30	[-360, 360]	degree°
Invert ROI specified axis direction	If selected, use the negative direction of the specified ROI axis for angle calculation; otherwise, use the positive direction of the specified ROI axis for angle calculation	Selected	/	/
Specified Picking Pose axis	Specify an axis of the Picking Pose for angle calculation	Z-axis	X/Y/Z-axis	/
Specified ROI axis	Specify an axis of the ROI coordinate system for angle calculation	Z-axis	X/Y/Z-axis	/

Example

3.2.4 Filter Pick Points outside the ROI 3D type region

Function description

Determine whether the Pick Point is within the ROI 3D range, and remove Pick Points that are outside the ROI 3D area.

Usage scenario

Prevent picking outside the ROI area, which may cause collisions between the robot arm and the target object.

Parameter description

Parameter	Description	Default
ROI3D type region	Usually "workspace"; "pick area" is a smaller ROI region than "workspace", which can restrict Pick Points to an ROI region smaller than the "workspace" to avoid some collision cases.	Workspace

Example

As shown in the figure below, when the ROI3D area and ROI2D are area a, the corresponding Pick Point is in the upper-right corner.

When the ROI3D area and ROI2D are changed to area b, the original Pick Point is outside the ROI area, so that Pick Point is removed and a new Pick Point is generated within area b.

3.2.5 [New] Filter Pick Points where the Target Object collides with the gripper (including the original function)

[New] Filter Pick Points where the Target Object collides with the gripper

Function description

Collision detection between the gripper and the Point Cloud near the Pick Point. If the number of Point Clouds in contact with the gripper exceeds the pick collision Threshold, the Pick Point of the Target Object is considered to have a collision risk.

Usage scenario

Used when collision detection is required between the gripper and the Point Cloud near the Target Object being picked.

Parameter description

Parameter	Description	Default	Range
Collision Threshold	Collision distance Threshold. If the distance between the scene and the gripper surface is smaller than this Threshold, it is considered a collision. The larger the Threshold, the stricter it is. Unit: mm	7	1-1000
Collision Point Cloud sampling	Sampling size for collision Point Clouds. The larger the value, the faster the cycle time; the smaller the value, the slower the cycle time. Effective only in "Target Object scene Point Cloud only" and "bin + Target Object scene Point Cloud" modes. Unit: mm	5	1 - 1000
Save visualization data for gripper collision detection	Save visualization data for collision detection between the gripper and the picked Target Object	Unchecked	Selected/Unchecked

Filter Pick Points where the Target Object collides with the gripper

Function description

Usage scenario

Used when collision detection is required between the gripper and the Point Cloud near the Target Object being picked.

Parameter description

Parameter	Description	Default	Range
Pick collision Threshold	The maximum number of Point Clouds the gripper may contain near the Pick Point. For example, 20 means that if the number of scene Point Clouds contained by the gripper exceeds 20, it is considered a collision	20	0-10000
Collision Point Cloud sampling (m)	Downsampling size of the Point Cloud in the collision area. The larger the value, the faster the detection speed, but the lower the accuracy. Applicable scenario: scenarios requiring high cycle rates	0.002	0.0001 - 0.5000
Save visualization data for gripper collision detection	Save visualization data for collision detection between the gripper and the picked Target Object	Unchecked	Selected/Unchecked
Import gripper model	Select and import the gripper model used for collision detection from a folder	/	/

**The gripper should be simplified to fewer than 500 faces**

3.2.6 [Advanced] Retain the one Pick Point with the largest/smallest pose value among instance Pick Points and filter the remaining Pick Points

Function description

Convert the pose to the specified coordinate system, sort poses according to the value of the specified sorting axis, and retain the pose with the maximum or minimum value. This is suitable for cylindrical Target Objects when keeping the top or bottom Pick Point.

Parameter description

Parameter	Description	Default	Range
Specified coordinate system	Select which coordinate system the pose should be converted to for processing	ROI coordinate system	ROI coordinate system/camera coordinate system
Specified sorting axis	Select which axis value of the pose to sort by	Z-axis	X/Y/Z-axis
Take minimum value	If selected, retain the pose with the minimum value on the sorting axis; otherwise, retain the pose with the maximum value on the sorting axis	Unchecked	/

Example

3.2.7 [Advanced] Filter Pick Points close to the previous N Pick Points

Function description

If the variation between the current Pick Point and any Pick Point in the cache is within the Threshold range, the Pick Point will be filtered out.

Parameter description

Parameter	Description	Default	Range	Unit
Upper limit of Pick Point change (+)
X(mm)	Upper limit of X coordinate	2	[0, 10000000]	mm
Y(mm)	Upper limit of Y coordinate	2	[0, 10000000]	mm
Z(mm)	Upper limit of Z coordinate	2	[0, 10000000]	mm
RX(°)	Upper limit of RX rotation amount	1	[0, 180]	degree°
RY(°)	Lower limit of RY rotation amount	1	[0, 180]	degree°
RZ(°)	Lower limit of RZ rotation amount	1	[0, 180]	degree°
Lower limit of Pick Point change (-)
X(mm)	Lower limit of X coordinate	2	[0, 10000000]	mm
Y(mm)	Lower limit of Y coordinate	2	[0, 10000000]	mm
Z(mm)	Lower limit of Z coordinate	2	[0, 10000000]	mm
RX(°)	Lower limit of RX rotation amount	1	[0, 180]	degree°
RY(°)	Lower limit of RY rotation amount	1	[0, 180]	degree°
RZ(°)	Lower limit of RZ rotation amount	1	[0, 180]	degree°
Pick Point cache count	Number of Pick Points cached. After the current Pick Point comparison is completed, it will be added to the cache in real time	5	[1, 100]	/

3.2.8 [Advanced] Filter Object Poses close to the previous N Object Poses

Function description

If the variation between the current Object Pose and any Object Pose in the cache is within the Threshold range, the Object Pose will be filtered out. When an Object Pose is determined to be similar, all Pick Points on that Target Object will be filtered out.

Parameter description

Parameter	Description	Default	Range	Unit
Upper limit of Object Pose change (+)
X(mm)	Upper limit of X coordinate	2	[0, 10000000]	mm
Y(mm)	Upper limit of Y coordinate	2	[0, 10000000]	mm
Z(mm)	Upper limit of Z coordinate	2	[0, 10000000]	mm
RX(°)	Upper limit of RX rotation amount	1	[0, 180]	degree°
RY(°)	Lower limit of RY rotation amount	1	[0, 180]	degree°
RZ(°)	Lower limit of RZ rotation amount	1	[0, 180]	degree°
Lower limit of Object Pose change (-)
X(mm)	Lower limit of X coordinate	2	[0, 10000000]	mm
Y(mm)	Lower limit of Y coordinate	2	[0, 10000000]	mm
Z(mm)	Lower limit of Z coordinate	2	[0, 10000000]	mm
RX(°)	Lower limit of RX rotation amount	1	[0, 180]	degree°
RY(°)	Lower limit of RY rotation amount	1	[0, 180]	degree°
RZ(°)	Lower limit of RZ rotation amount	1	[0, 180]	degree°
Object Pose cache count	Number of vision Object Poses cached. After the comparison of the current Object Pose is completed, it will be added to the cache in real time	5	[1, 100]	/

3.2.9 [Advanced] Filter Pick Points outside the upper and lower limits of Pick coordinates

Function description

Retain other Pick Points within the specified range of a reference Pick Point and filter out abnormal Pick Points.

Usage scenario

Prevent incorrect robot picking and ensure picking accuracy.

This function is not applicable to depalletizing scenarios

Parameter description

Parameter	Description	Default	Unit
Reference Pick coordinates
X(mm)	X coordinate of the reference Pick Point	0	mm
Y(mm)	Y coordinate of the reference Pick Point	0	mm
Z(mm)	Z coordinate of the reference Pick Point	0	mm
RX(°)	RX rotation amount of the reference Pick Point	0	degree
RY(°)	RY rotation amount of the reference Pick Point	0	degree
RZ(°)	RZ rotation amount of the reference Pick Point	0	degree
Upper limit of Pick coordinates (+)
X(mm)	Upper limit of the X coordinate. For example, if the X coordinate of the reference Pick Point is 100 and the upper limit is set to 10, the allowed range is: [100-lower limit, 110]	10000000, meaning no limit.	mm
Y(mm)	Upper limit of the Y coordinate. For example, if the Y coordinate of the reference Pick Point is 100 and the upper limit is set to 10, the allowed range is: [100-lower limit, 110]	10000000	mm
Z(mm)	Upper limit of the Z coordinate. For example, if the Z coordinate of the reference Pick Point is 100 and the upper limit is set to 10, the allowed range is: [100-lower limit, 110]	10000000	mm
RX(°)	Upper limit of the RX rotation amount. For example, if the RX rotation amount of the reference Pick Point is 180 and the upper limit is set to 10, the allowed range is (default angle wraparound applies): [[-180, -170], [180-lower limit, 180]]	180, meaning no limit.	degree°
RY(°)	Upper limit of the RY rotation amount. For example, if the RY rotation amount of the reference Pick Point is 180 and the upper limit is set to 10, the allowed range is (default angle wraparound applies): [[-180, -170], [180-lower limit, 180]]	180	degree°
RZ(°)	Upper limit of the RZ rotation amount. For example, if the RZ rotation amount of the reference Pick Point is 180 and the upper limit is set to 10, the allowed range is (default angle wraparound applies): [[-180, -170], [180-lower limit, 180]]	180	degree°
Lower limit of Pick coordinates (-)
X(mm)	Lower limit of the X coordinate. For example, if the X coordinate of the reference Pick Point is 100 and the lower limit is set to 10, the allowed range is: [100-lower limit value, 110]	10000000	mm
Y(mm)	Lower limit of the Y coordinate. For example, if the Y coordinate of the reference Pick Point is 100 and the lower limit is set to 10, the allowed range is: [100-lower limit, 110]	10000000	mm
Z(mm)	Lower limit of the Z coordinate. For example, if the Z coordinate of the reference Pick Point is 100 and the lower limit is set to 10, the allowed range is: [100-lower limit, 110]	10000000	mm
RX(°)	Lower limit of the RX rotation amount. For example, if the RX rotation amount of the reference Pick Point is 180 and the lower limit is set to 10, the allowed range is (default angle wraparound applies): [[-180, -180+upper limit], [170, 180]]	180, meaning no limit.	degree°
RY(°)	Lower limit of the RY rotation amount. For example, if the RY rotation amount of the reference Pick Point is 180 and the lower limit is set to 10, the allowed range is (default angle wraparound applies): [[-180, -180+upper limit], [170, 180]]	180	degree°
RZ(°)	Lower limit of the RZ rotation amount. For example, if the RZ rotation amount of the reference Pick Point is 180 and the lower limit is set to 10, the allowed range is (default angle wraparound applies): [[-180, -180+upper limit], [170, 180]]	180	degree°

3.3 Pick Point Sorting

3.3.1 Reference coordinate system

Function description

Set a unified coordinate system for all instances to group and sort instances.

Usage scenario

Common to depalletizing scenarios, random picking scenarios, and ordered loading/unloading scenarios

Strategies related to coordinates should first set the reference coordinate system

Parameter description

Parameter	Description	Illustration
Camera coordinate system	The coordinate system origin is above the object, and the positive Z-axis direction points downward; the XYZ values are the values of the center point of the object in this coordinate system
ROI coordinate system	The coordinate system origin is approximately at the center of the pallet stack, and the positive Z-axis direction points upward; the XYZ values are the values of the center point of the object in this coordinate system
Robot arm coordinate system	The coordinate system origin is on the robot arm itself, and the positive Z-axis direction generally points upward; the XYZ values are the values of the center point of the object in this coordinate system
Pixel coordinate system	The coordinate system origin is at the top-left vertex of the RGB image and is a 2D planar coordinate system; the X and Y values are the x value of the bbox detection box and the y value of the bbox detection box, and Z is 0

3.3.2 General picking strategy

Parameter description

Parameter	Description
Strategy	Select which value is used for grouping and sorting and how to sort it, including Pick Point center X/Y/Z coordinate values from large to small/from small to large (mm), from the middle to the sides / from the sides to the middle along the Pick Point XY coordinate axis (mm). Multiple items can be superimposed and executed in order.
Grouping step size	According to the selected strategy, divide Pick Points into several groups based on the step size. The grouping step size is the interval between two groups of Pick Points
Number of leading groups to keep	After grouping and sorting, how many groups of instances need to be retained

Strategy name*	Description	Grouping step size		Number of leading groups to keep
Strategy name*	Description	Default	Range	Default
Pick Point center X/Y/Z coordinate values from large to small / from small to large (mm)	Use the X/Y/Z coordinate values of the Pick Point center for grouping and sorting	200.000	[0, 10000000]	10000
From the middle to the sides / from the sides to the middle along the Pick Point XY coordinate axis (mm)	Use the X/Y coordinate values of the Pick Point center and perform grouping and sorting in the direction of "middle to sides" or "sides to middle"	200.000	[0, 10000000]	10000

3.3.3 Carton combination strategy

To solve the problems of low efficiency and limited applicable scenarios in traditional single-pick depalletizing, PickWiz adds a carton combination strategy in depalletizing scenarios to support picking multiple Target Objects in a single operation. It supports the core scenarios of "cartons with consistent dimensions" and "rectangular suction cups", covering more real project scenarios.

3.3.3.1 Multi-pick runtime configuration

（1）In sack single depalletizing or carton single depalletizing scenarios, enable Vision computation configuration - Vision computation acceleration.

（2）Under the Pick Point sorting module, select the carton combination strategy;

（3）Strategy selection: available options are the default combination strategy or combination along a specified carton pose axis. These are two methods for finding the largest number of cartons that can be combined.

Default combination strategy: Find the largest number of cartons that can be combined along the X-axis and Y-axis directions of a carton Picking Pose.
Combine along a certain carton pose axis: Find the largest number of cartons that can be combined along the X-axis or Y-axis direction of a carton Picking Pose. This is suitable for scenarios where cartons are arranged in a straight line. When using this strategy, you need to choose the carton combination direction, namely the Picking Pose X-axis or the Picking Pose Y-axis.

Note:
The combination direction is related to the positive and negative axis directions. Cartons can be combined only when they are on the same axis and in the same direction, and after combination, the orientation of the whole stack of cartons remains consistent with the orientation of a single carton before combination. Therefore, before combining cartons, make sure all cartons to be combined are placed in the same orientation, and unify the coordinates of the cartons to be combined to the same axis direction.

（4）Combination conditions: determine which cartons can be combined and how many can be combined at most.

Maximum cartons per row: the maximum number of cartons that can be combined in one row, default is 2.
Maximum number of combination rows: the maximum number of carton rows that can be combined, default is 1.
Maximum spacing (mm): cartons to be combined cannot be too far apart in the "combination direction", In the combination direction (axis direction), when the spacing between two adjacent cartons or cartons in different rows is less than this value, they can be combined into one group. The default is 10.
- Example: when searching for the maximum number of cartons along the Picking Pose X-axis, if the spacing between two adjacent cartons in the Picking Pose X-axis direction is 8 mm (≤10), they can be combined; if the spacing is 12 mm (>10), they cannot be combined.
Maximum misalignment distance (mm): cartons to be combined cannot be too far apart in the direction "perpendicular to the combination direction" . In the direction perpendicular to the combination direction (axis direction), when the misalignment distance between two adjacent cartons or cartons in different rows is less than this value, they can be combined into one group. The default is 10.
- Example: when searching for the maximum number of cartons along the Picking Pose X-axis, if two adjacent cartons are offset in the Picking Pose Y-axis direction by 8 mm (≤10), they can be combined; if they are offset by 15 mm (>10), they are no longer aligned and cannot be picked together.
Maximum angular deviation (°): cartons to be combined should face almost the same direction. In the combination direction (axis direction), when the rotational deviation angle of the cartons is less than this value, they can be combined into one group. The default is 10.
- Example: if a carton is rotated by 5° relative to the combination direction, as long as it does not exceed 10°, it can be combined; if it is rotated by 15° (>10), the orientation differs too much, the robot will be skewed when picking, and it cannot be combined.

3.3.3.2 Robot configuration

（1）On the robot configuration page, add new placeholders in Vision computation communication message - Robot to PickWiz commands - Vision detection send command: maximum cartons per row and maximum number of combination rows, as shown below;

（2）In Vision computation communication message - PickWiz to robot commands - Pick-related information - Returned information when picking Target Objects, add Object Dimensions length, Object Dimensions width, and Target Object orientation.

After the robot configuration is completed, click the Run button.

3.3.3.3 View multi-pick runtime results

（1）In the 3D Matching window, hover the mouse over an instance to view the combined picking information of a single instance after carton combination, including 2D recognition results, Picking Pose, and instance combination information.

In the visualization window, click the Settings button in the upper right corner to set how the combined instance information is displayed.

Right-click an instance to view the combined picking information and Target Object information of the single instance.

（2）In the 2D recognition window, you can use the relevant combination buttons in the menu bar to view the combined ID, combined Mask, and combined bounding box.

How to Use Shadow Mode

Visual Parameter Adjustment Guide for Surface-type Target Object Loading and Unloading (Materials Isolated from Each Other)