This chapter mainly introduces how to adjust the visual parameters according to the actual scene in the destacking scene.

Getting Started:

Background introduction

Build project

(1) Create a new project of single disassembly of sacks/single disassembly of cartons (the project name and project address can be customized, and the project name cannot contain Chinese characters)

Target type: Single type of sack------Single bag can be dismantled separately, Single type of carton------Carton can be dismantled separately

Optional function options include Instance Optimization, visual classification, and Pallet Detection

(2) Camera and Robot configuration

(3) Add Target

The Visual Model is a universal sack model ([Single bag single disassembly-universal sack bag model, Carton single disassembly-Universal carton single disassembly model] (../3D视觉引导方案介绍.md)). If the visual segmentation effect is not good, you can train the model by yourself according to the actual situation. There are two ways to train the model: mask mode and shadow mode. If the surface pattern of the sack is small, you can use the mask mode to improve the recognition of specific textures, and the training time is about 1 hour; if the surface pattern of the sack is large, the success rate of Instance Segmentation has reached 95%, and if the success rate of Instance Segmentation is expected to be higher, you can use the shadow mode, and the training time is about 1 hour.

Mask mode: You need to enter the texture, find the unidentified data in the historical data, import the image {=tex} in the instance\input folder, select the target sack in the image, select at least 10 data, then click to start training, wait for about 1 hour. How to use Mask Mode

Shadow mode: Shadow data is a picture with good visual segmentation that is retained during the production process. Therefore, training in shadow mode requires a period of production and can only be trained after the shadow data is generated. How to use shadow mode

(4) Add End-effector, Eye-hand calibration, and ROI

(5) Function options: Instance Optimization, Pallet Detection, visual classification

Instance Optimization: Optimize the instances generated by the model and process the instance masks.

Pallet Detection: Turn on the Scene where the bottom pallet needs to be grabbed. After turning it on, the bottom pallet can be grabbed after the normal Target grabbing is completed. Pallet Detection User Guide

Visual classification: used to identify different textures, different orientations and other features of the same Target. Visual Classification User Guide

(6) Test data (historical data is provided for subsequent exercises. You can use the {=tex}2D image and 3DPoint Cloud in the foreground\input folder in the historical data instead of Camera to configure the ROI)

Sack data:

Carton data:

Visual Parameter

• 2D recognition: recognize and segment instances from the actual scene

Preprocessing: 2D image processing (edge ​​enhancement) before Instance Segmentation

Instance Segmentation: Identify segmentation instances (scaling ratio & Confidence lower limit Threshold & automatic enhancement)

Point Cloud generation: A way to generate an instance Point Cloud. Use a segmented instance mask or bounding box to generate an instance Point Cloud. Use a filtered instance mask or bounding box to generate an instance Point Cloud.

Instance filtering: Filter the split instances

Instance sorting (carton): Sort the instances. The grabbing order of destacking is the order in which the instances are sorted.

• 3D calculation: Calculate the posture of the instance in the Camera coordinate system and generate Pick Point

Preprocessing: Before calculating the Pick Point, preprocess the 3DPoint Cloud (calculate its own vertical posture, Point Cloud remove outliers)

Pick Point generation: Calculation method for Pick Point generation (rectangle fitting, selection of coordinate system, neighborhood width, neighborhood length)

• Pick Point processing: filter, adjust, and sort Pick Points

Pick Point filtering: Filter Pick Points

Pick Point adjustment: Adjust Pick Point

1. 2D recognition

This section mainly explains and provides parameter adjustment suggestions for preprocessing, instance segmentation, instance filtering, and instance sorting related functions that affect 2D image recognition results.

1.1 Pretreatment

The preprocessing of 2D recognition is to process the 2D image before Instance Segmentation

1.1.1 bilateral filtering

• Function

Image smoothing function based on bilateral filtering

• • • Parameter description * *

Parameter	Description	Default value	value range
Maximum depth difference	Maximum depth difference for bilateral filtering	0.03	[0.01, 1]
filter core size	Bilateral filtering convolution kernel size	7	[1, 3000]

1.1.2 depth to normal vector diagram

• Function

Calculate the pixel normal vector through the Depth image and convert the image into a Normal vector diagram

1.1.3 Image Enhancement

• Function

Commonly used image enhancements, such as color saturation, contrast, brightness, and sharpness

• • • Parameter description * *

Parameter	Description	Default value	value range
Image enhancement type	Enhance an element of the image	Contrast	Color saturation, contrast, brightness, sharpness
picture enhancementThreshold	How much does enhance a certain element of the image	1.5	[0.1, 100]

1.1.4 Histogram equalization

• Function

Improve image contrast

• • • Parameter description * *

Parameter	Description	Default value	value range
local mode	Local or global histogram equalization. If checked, the local histogram will be equalized. If unchecked, the global histogram will be equalized.	Check	/
Contrast Threshold	Contrast Threshold	3	[1,1000]

1.1.5 Filter Depth image by color

• Function

Filter Depth image based on color value

• • • Parameter description * *

Parameter	Description	Default value	value range
padding core size	color fill size	3	[1,99]
Filter Depth by hsv - Gamut Max	Maximum color value	[180,255,255]	[[0,0,0],[255,255,255]]
filter depth according to hsv - color gamut minimum	minimum color value	[0,0,0]	[[0,0,0],[255,255,255]]
saves the area within the color range	If checked, the area within the color range will be saved. If not checked, the area outside the color range will be saved.	/	/

1.1.6 Gamma picture correction

• Function

Gamma correction changes picture brightness

• • • Parameter description * *

Parameter	Description	Default value	value range
gamma compensation coefficient	The value of is less than 1, the value of the picture becomes darker, the value is greater than 1, the picture variable	1	[0.1,100]
gamma correction coefficient	If the value is less than 1, the picture becomes darker, which is suitable for pictures with too high brightness. If the value is greater than 1, the picture becomes brighter, which is suitable for pictures with too low brightness.	2.2	[0.1,100]

1.1.7 fills the Depth image hole

• Function

Fill the hole area in the Depth image and smooth the filled Depth image

• Use Scene

Due to problems such as structural occlusion and uneven lighting of the Target itself, the Depth image may be missing some areas of the Target.

• • • Parameter description * *

Parameter	Description	Default value	value range
padding core size	Size of hole filling	3	[1,99]

The filling core size can only be filled in with odd numbers

• Adjust parameters

Adjust according to the detection results. If the filling is excessive, the Parameter should be adjusted smaller; if the filling is insufficient, the Parameter should be increased.

• Example

1.1.8 edge enhancement

• Function

Set the edge of the texture in the image to the background color or a color that is significantly different from the background color to highlight the edge information of the Target

• Use Scene

Targets block or overlap each other, resulting in unclear edges

• • • Parameter description * *

Parameter	Description	Default value	Parameter range	Parameter adjustment recommendation
normal z direction filter Threshold	Depth image The angle between the normal vector corresponding to each point and the positive Z-axis direction of the Camera coordinate system is filtered Threshold. If the angle between the normal vector of the point and the positive Z-axis direction of the Camera coordinate system is greater than this Threshold, then the point in the 2D image corresponds to The color of the position will be set to the background color or the color that is significantly different from the background color	30	[0,180]	For flat Target surfaces, the Threshold can be stricter. Curved surfaces such as sacks can be appropriately increased according to the tilt of the Target surface.
Background color	RGB color of background color Threshold	128	[0,255]
Automatically adjust contrast background	check After automatically adjusting the contrast background, set the color of the points in the 2D image with an angle greater than the filter Threshold to a color that is significantly different from the background color . Uncheck Automatically adjust the contrast background, and set the color of the point in the 2D image with an angle greater than the filter Threshold to , the color corresponding to the background color ,	Uncheck	/

• Example

In the scene of a pile of sacks, the sacks block each other. Check Edge Enhancement to distinguish the edges of each sack, as shown in the figure below.

1.1.9 extracts the top texture

• Function

Extract the top or bottom Target texture, and set other areas to the background color or a color that is significantly different from the background color.

• Use Scene

Suitable for single carton unpacking scene. Factors such as poor lighting conditions, similar color and texture, close stacking, staggered stacking, or occlusion may make it difficult for the model to distinguish the texture differences between the upper and lower layers of the carton, making it easy to detect errors.

• • • Parameter description * *

Parameter	Description	Default value	Parameter range	unit	Parameter adjustment recommendation
distance Threshold(mm)	If the distance between point and the uppermost plane (bottom plane) is lower than the Threshold, it is considered to be a point in the uppermost plane (bottom plane) and should be retained. Otherwise, it is considered to be a point in the lower layer (upper layer). The color of the point in the lower layer (upper layer) is set to the background color or a color that is significantly different from the background color	50	[0.1, 1000]	mm	is generally adjusted to 1/2 of the carton height
Clustering Point Cloud Number	The number of points expected to participate in clustering, the number of sampling Point Clouds in the ROI 3D area	10000	[1，10000000]	/	The more the number of clustering Point Clouds
Minimum number of category points	Minimum number of points used to filter categories	1000	[1, 10000000]	/	/
automatically calculates the contrast background	check After automatically calculating the contrast background, set other areas except the top layer (bottom layer) in the 2D image to a color that is significantly different from the background color Threshold. Uncheck Automatically calculate the contrast background, and set other areas except the top layer (bottom layer) in the 2D image to the color corresponding to the background color Threshold	Check	/	/	/
Background ColorThreshold	Background ColorRGB ColorThreshold	128	[0, 255]	/	/

• Example

1.1.10 Remove picture background outside roi3d

• Function

Remove the background from the 2D image except the ROI3D area

• Use Scene

Too much background noise in the image affects the detection results

• • • Parameter description * *

Parameter name	Description	Default value	value range
padding core size	Size of hole filling	5	[1,99]
Number of iterations	Number of iterations of image expansion	1	[1,99]
automatically calculates the contrast background	check After automatically calculating the contrast background, set the area outside the roi in the 2D image to a color that is significantly different from the background color Threshold and uncheck it. Automatically calculate the contrast background and set the area outside the roi in the 2D image to the color corresponding to the background color Threshold	Check
Background ColorThreshold	RGB color of background color Threshold	128	[0,255]

The filling core size can only be filled in with odd numbers

• Adjust parameters

If you need to remove more background noise from the image, you should reduce the fill kernel size

• Example

---

1.2 Instance Segmentation

1.2.1 Scale

• Function

By scaling the original image proportionally and then inference, the accuracy and recall rate of 2D recognition can be improved.

• Use Scene

This function should be adjusted if the detection effect is poor (such as no instance detected, missing recognition, bounding box bounding box to multiple instances or the box is not full of instances).

• • • Parameter description * *

  • Default value: 1.0

  • Value range: [0.01, 3.00]

  • Step size: 0.01

• Adjust parameters

  • Run with default values ​​and view the detection results in the visualization window. If instances are not detected, recognition is missed, the bounding box is bounded to multiple instances, or the box is not full of instances, the function should be adjusted.

• The percentage of instances in 2D recognition is the Confidence score, and the number is the Instance ID (the recognition order of the instances).

• In 2D recognition, the colored shadow on the instance is the mask, and the rectangular box surrounding the instance is the bounding box.

If you cannot obtain better detection results after trying all zoom ratios, you can adjust the ROI area.

As shown in the figure below, the detection effect is significantly improved when the scaling ratio is 0.7, so it can be determined that 0.7 is the lower limit of the scaling range

The detection effect is significantly reduced when the scaling ratio is 1.8, so it can be determined that 1.8 is the upper limit of the scaling range.

---

1.2.2 ConfidenceLower Limit Threshold

• Function

Only the recognition results whose Confidence score of the deep learning model is higher than the Confidence lower limit Threshold are retained.

• Use Scene

This function can be adjusted when the instance selected by the detection result box does not meet expectations.

• • • Parameter description * *

Default value: 0.5

Value range: [0.01, 1.00]

• Adjust parameters

  • If fewer instances are detected, the Threshold should be reduced; if the value is too small, it may affect the accuracy of image recognition.

  

  

  

  • If erroneous instances are detected because the Confidence lower limit Threshold is too small, and these erroneous instances need to be removed, the Threshold should be increased; if the value is too large, the retained detection results may be zero and no results will be output.

• Example

When Confidence lower limit Threshold is set to 0.5, the retained instance detection results are shown in the figure below (left). When Confidence lower limit Threshold is set to XQ The detection results of the instances retained when ZPZ00001XQZ are shown in the figure below (right). Instances with scores of 77.60%, 74.61%, and 69.77% (less than 80%) are filtered out.

---

1.2.3 Enable Auto Enhance

• Function

Inference is achieved by combining all the values ​​in the input scaling ratio and rotation angle, and returning all results that are greater than the set Confidence lower limit Threshold after the combination, which can improve the model inference accuracy. But it adds time.

• Use Scene

A single scaling ratio cannot meet the actual scene requirements, resulting in incomplete detection or high inclination of object placement.

• Example

If the auto-enhancement-scaling ratio is set to [0.8, 0.9, 1.0] and the auto-enhancement-rotation angle is set to [0, 90.0] , then combine the values ​​in the scaling ratio and rotation angle, and the model will automatically generate 6 types of images for Inference. Finally, these 6 types of Inference results will be unified together, and a result greater than the Confidence lower limit Threshold will be output.

Auto Enhance-Zoom

• Function

Scale the original image multiple times and perform multiple inferences, and output the comprehensive inference results.

• Use Scene

A single scaling ratio cannot meet the actual scene requirements, resulting in incomplete detection.

• • • Parameter description * *

Default value: [1.0]

Value range: The range of each scaling ratio is [0.1, 3.0]

Multiple scaling ratios can be set, and each scaling ratio is separated by a comma.

• Adjust parameters

Fill in multiple 1.2.1 scaling Obtained scaling ratio with better detection effect

Auto Enhancement-Rotation Angle

• Function

Rotate the original image multiple times and perform multiple inferences, and output the comprehensive inference results.

• Use Scene

Used when objects are placed far away from the coordinate axis

• • • Parameter description * *

Default value: [0.0]

Value range: The value range of each rotation angle is [0, 360]

Multiple rotation angles can be set, and each rotation angle is separated by a comma.

• Adjust parameters

Adjust the automatic enhancement-rotation angle according to the angle of the object in the actual scene. The tilt angle can be determined based on the pattern and mouth shape of the sack, the edges of the carton and the brand logo.

---

1.3 Point Cloud generation

Destacking Scene generally uses mask form (after segmentation) or mask form (after filtering) to generate instance Point Cloud

instance Point Cloud generated form	mask form (after segmentation)	—	uses the split instance mask to generate Point Cloud
	bounding box form (after segmentation)	bounding box scaling (after segmentation)	uses the split instance bounding box to generate Point Cloud
		Whether color is needed to generate Point Cloud (after segmentation)	Does the instance Point Cloud generated by need to be attached with the color
	mask form (after filtering)	—	uses the filtered instance mask to generate Point Cloud
	bounding box form (after filtering)	bounding box scaling (after filtering)	uses the filtered instance bounding box to generate Point Cloud
		Whether color is required to generate Point Cloud (after filtering)	Does the instance Point Cloud generated by need to be attached with the color

If acceleration is not required, there is no need to use the instance filter function. Use the mask form (after splitting) to generate the instance Point Cloud, which can be found in Project storage folder \Project name\data\PickLight\Historical data timestamp\Builder\pose\input folder View the generated instance Point Cloud;

If acceleration is required, you can use the instance filtering function to filter the instances, and use the mask form (after filtering) to generate the instance Point Cloud, which can be found in Project storage folder\Project name\data\PickLight\Historical data timestamp\Builder\pose\input folder View the generated instance Point Cloud

1.4 instance filtering

1.4.1 filtering based on bounding box area

• Function introduction

Filter based on the pixel area of ​​the bounding box of the detected instance.

• Use Scene

It is suitable for scenes with large differences in instance bounding box area. By setting the upper limit and lower limit of the bounding box area, the noise in the image is filtered to improve the accuracy of image recognition and avoid noise adding time to subsequent processing.

• • • Parameter description * *

Parameter	Description	Default value	Parameter range	unit
Minimum area (pixels)	This Parameter is used to set the minimum filter area of ​​the bounding box. Instances with a bounding box area lower than this value will be filtered	1	[1, 10000000]	pixels
Maximum area (pixels)	This Parameter is used to set the maximum filtering area of ​​the bounding box. Instances with a bounding box area higher than this value will be filtered	10000000	[2, 10000000]	pixels

• Example

Running by default, you can view the bounding box area of ​​each instance in the log, as shown in the figure below.

Adjust minimum area based on the bounding box area of ​​each instance and Maximum area, if you set Minimum area to 20000 and Maximum area to 30000, you can filter out instances with a pixel area less than 20000 or greater than 30000. You can view the instance filtering process in the log.

1.4.2 filtering based on bounding box aspect ratio

• Function introduction

Instances with bounding box aspect ratios outside the specified range will be filtered out

• Use Scene

Suitable for Scenes where the aspect ratio of the bounding box of the instance is greatly different.

• • • Parameter description * *

Parameter	Description	Default value	Parameter range
Minimum aspect ratio	The minimum value of the bounding box aspect ratio. Instances with a bounding box aspect ratio lower than this value will be filtered	0	[0, 10000000]
Maximum aspect ratio	The maximum value of the bounding box aspect ratio. Instances with a bounding box aspect ratio higher than this value will be filtered	10000000	[0, 10000000]
uses the X/Y axis side length as the aspect ratio	is not checked by default and uses the length ratio of the longer side/shorter side of the bounding box as the aspect ratio, which is suitable for situations where the length difference between the long and short sides of the bounding box is large; When checked, the length ratio of the X-axis/Y-axis sides of the bounding box in the pixel coordinate system is used as the aspect ratio. This is suitable for the situation where the long side/short side ratio of most normal instance bounding boxes is similar, but the ratio of the X-axis length/Y-axis length of some abnormally identified instance bounding boxes is quite different.	Uncheck	/

1.4.3 Filter instances based on category ID

• Function introduction

Filter by instance category

• Use Scene

Suitable for scenes with multiple types of targets for incoming materials

• • • Parameter description * *

Parameter	Description	Default value
Reserved category ID	Instances with category IDs in the list are retained, and instances with category IDs not in the list will be filtered	[0]

• Example

1.4.4 Edge length filtering based on instance Point Cloud

• Function introduction

Filter based on the long and short sides of the instance Point Cloud

• Use Scene

It is suitable for scenes where the distance between the instance Point Clouds on the x-axis or the y-axis is quite different. By setting the distance range of the instance Point Cloud, you can filter the noise in the image, improve the accuracy of image recognition, and avoid noise adding time to subsequent processing.

• • • Parameter description * *

Parameter	Description	Default value	Parameter range	unit
short side length range (mm)	Point The side length range of the short side of the Cloud is	[0, 10000]	[0, 10000]	mm
long side length range (mm)	Point Cloud's long side length range	[0, 10000]	[0, 10000]	mm
edge denoising lower limit (%)	Extract the lower limit of the percentage of the X/Y value (Camera coordinate system) in the instance Point Cloud, remove the Point Cloud outside the upper and lower limits, and avoid noise affecting the length calculation	5	[0, 100]	/
edge denoising upper limit (%)	Extract the upper limit of the percentage of the X/Y value (Camera coordinate system) in the instance Point Cloud, remove the Point Cloud outside the upper and lower limits, and avoid noise affecting the length calculation	95	[0, 100]	/
side length type	Filter by the long and short sides of the instance Point Cloud. Instances whose lengths of the long and short sides are not within the range will be filtered	instance Point Cloud short side	instance Point Cloud short side; instance Point Cloud long side; instance Point Cloud long side and short side	/

• Example

1.4.5 Classifier based category ID filtering

• Function introduction

Instances are filtered based on the class ID of the classifier, instances not within the reference class will be filtered.

• Use Scene

In a multi-type TargetScene, the Visual Model may detect multiple types of Targets, but the actual Task may only require one type of Target. In this case, you can use this function to filter out unnecessary Targets.

• • • Parameter description * *

The default value is [0], that is, instances with category ID 0 are retained by default, category ID Instances not in the list will be filtered.

1.4.6 based on three-channel color filtering

• Function introduction

Instances can be filtered out by three-channel color threshold (HSV or RGB).

• Use Scene

There is a clear distinction between the colors of incorrect instances and correct instances.

• • • Parameter description * *

Parameter	Description	Default value	value range
color gamut maximum value	Maximum color value	[180,255,255]	[[0,0,0],[255,255,255]]
color gamut minimum value	minimum color value	[0,0,0]	[[0,0,0],[255,255,255]]
filter percentageThreshold	color pass rateThreshold	0.05	[0,1]
Reverse filtering	If checked, instances with a ratio outside the color range lower than the Threshold will be eliminated. If not checked, instances with a ratio within the color range lower than the Threshold in the instance image will be eliminated.	Uncheck	/
color mode	color space selected in color filtering	HSV color space	RGB color spaceHSV color space

• Example

1.4.7 based on Confidence filtering

• Function introduction

Filter based on the instance’s Confidence score

• Use Scene

Applicable to Scenes with large differences in instance confidence

• • • Parameter description * *

Parameter	Description	Default value	Parameter range
ReferenceConfidence	retains instances with Confidence greater than Threshold and filters instances with Confidence less than Threshold.	0.5	[0,1]
Invert filter result	After inversion, instances with visibility Confidence less than Threshold are retained, and instances with Confidence greater than Threshold are filtered.	Uncheck	/

• Example

1.4.8 Filter based on the number of Point Clouds

• Function introduction

Filter based on the number of instance Point Clouds after downsampling

• Use Scene

Instance Point Cloud has more noise

• • • Parameter description * *

Parameter	Description	Default value	Parameter range
Point Cloud quantity minimum	Point Cloud quantity minimum value	3500	[1, 10000000]
Point Cloud maximum number	Point Cloud maximum number	8500	[2, 10000000]
filters instances whose number is within the interval	If checked, the instances whose Point Cloud quantity is within the minimum and maximum range will be filtered. If not checked, the instances whose Point Cloud quantity is not within the range will be filtered.	Uncheck	/

1.4.9 filtering based on mask area

• Function introduction

Filter the image mask based on the mask pixel sum (i.e. pixel area) of the detected instance.

• Use Scene

It is suitable for Scenes with large differences in instance mask area. By setting the upper and lower limits of the mask area, the noise in the image mask is filtered to improve the accuracy of image recognition and avoid noise adding time to subsequent processing.

• Parameter setting instructions

Parameter name	Description	Default value	Parameter range	unit
reference minimum area	This Parameter is used to set the minimum filtering area of ​​the mask. Instances with a mask area lower than this value will be filtered	1	[1, 10000000]	pixels
Reference maximum area	This Parameter is used to set the maximum filtering area of ​​the mask. Instances with a mask area higher than this value will be filtered	10000000	[2, 10000000]	pixels

• Example

1.4.10 filtering based on visibility

• Function introduction

Filter based on the visibility score of the instance

• Use Scene

Suitable for Scenes where the visibility of instances varies greatly.

• • • Parameter description * *

Parameter	Description	Default value	Parameter range
Reference Visibility Threshold	retains instances with visibility greater than Threshold and filters instances with visibility less than Threshold. Visibility is used to determine the degree to which instances in an image are visible. The more the Target is occluded, the lower the visibility.	0.5	[0,1]
Invert filter result	After inversion, instances with visibility less than Threshold are retained and instances with visibility greater than Threshold are filtered.	Uncheck	/

1.4.11 filter bounding box overlap example

• Function introduction

Filter instances where bounding boxes intersect and overlap

• Use Scene

Applies to Scenes where the instance's bounding boxes intersect each other

• • • Parameter description * *

Parameter	Description	Default value	Parameter range
bounding box overlap ratio Threshold	The ratio of the intersection area of ​​the bounding box to the area of ​​the instance bounding box Threshold	0.05	[0, 1]
Example of filtering bounding box with large area	If checked, the instance with the larger area among the instances where the two bounding boxes intersect will be filtered. If not checked, the instance with the smaller area among the instances where the two bounding boxes intersect will be filtered.	Check	/

• Example

New Filter the surrounded instances and run with the default value. Check the intersection of instance bounding boxes in the log. After instance filtering, there are 2 instances left.

It can be seen from the log that 12 instances were filtered out because of bounding box intersections, and the remaining 2 instances had no intersections.

Set Bounding Box Overlap Ratio Threshold to 0.1 and check Whether to filter larger instances, check the instance filtering process in the log. 9 instances were filtered out because the ratio of the intersection area of ​​the bounding box to the area of ​​the instance bounding box was greater than 0.1, 3 instances were retained because the ratio of the intersection area of ​​the bounding box to the area of ​​the instance bounding box was less than 0.1, and 2 instance bounding boxes did not intersect.

Set the Bounding Box Overlap Ratio Threshold to 0.1 and uncheck it Whether to filter larger instances, check the instance filtering process in the log. The ratio of the intersection area of ​​the bounding box to the area of ​​the instance bounding box of 9 instances is greater than 0.1, but 2 instances are retained because the area of ​​the bounding box is smaller than the instance with which it intersects, so 7 instances are filtered, and 3 instances are retained because the ratio of the intersection area of ​​the bounding box to the area of ​​the instance bounding box is less than 0.1. The bounding boxes of 2 instances do not intersect.

1.4.12 [Master] An example of filtering mask bumps based on the mask/mask external polygon area ratio

• Function introduction

Calculate the area ratio of the mask/mask circumscribed polygon. If it is less than the set Threshold, the instance will be filtered out.

• Use Scene

Suitable for situations where the Target mask has jagged/bumpy edges.

• • • Parameter description * *

Parameter	Description	Default value	value range
area ratio Threshold	mask/convex hull area ratio Threshold. If it is less than the set Threshold, the instance will be filtered out.	0.1	[0,1]

1.4.13 [Master] Filtering based on Point Cloud average distance

• Function introduction

Filter based on the average distance from the midpoint of the Point Cloud to the fitting plane to eliminate uneven instance Point Clouds.

• Use Scene

Suitable for flat TargetPoint Cloud curved Scene

• • • Parameter description * *

Parameter	Description	Default value	Parameter range	unit
plane division distance Threshold (mm)	Extracts a plane from the curved instance Point Cloud. Points whose distance from the plane is less than the Threshold are regarded as points of the plane	10	[-1000, 1000]	mm
Average distance Threshold (mm)	The average distance from the points in the Point Cloud of the instance to the extraction plane	20	[-1000, 1000]	mm
Eliminate instances with an average distance less than Threshold	If checked, the average distance from the filter point to the extraction plane is less than the average distance Threshold. If not checked, the average distance from the filter point to the extraction plane is greater than the average distance Threshold.	Uncheck	/	/

1.4.14 [Master] Filter occluded instances based on the mask/bounding box area ratio

• Function introduction

Calculate the mask/bounding box area ratio. Instances whose ratio is not within the maximum and minimum range will be filtered.

• Use Scene

Used to filter instances of occluded Targets

• • • Parameter description * *

Parameter	Description	Default value	value range
minimum area ratio	The lower limit of the mask/bounding box area ratio range. The smaller the ratio, the higher the degree of occlusion of the instance.	0.1	[0,1]
maximum area ratio	The upper limit of the mask/bounding box area ratio range. The closer the ratio is to 1, the less occluded the instance is.	1.0	[0,1]

1.4.15 [Master] Determine whether the top instance is fully detected

• Function introduction

One of the anti-fool mechanisms, it determines whether all the top-level instances have been checked out. If there are top-level instances that have not been checked out, an error will be reported and the workflow will end.

• Use Scene

It is suitable for capturing scenes once and capturing multiple times or scenes that must be captured in order to prevent missing captures from incomplete instance detection and affecting subsequent tasks.

• • • Parameter description * *

Parameter	Description	Default value	Parameter range	unit	Parameter adjustment
distance Threshold	is used to determine the top-level Target. If the distance between a point and the highest point of the TargetPoint Cloud is less than the distance Threshold, then the point is considered to be the top-level Point Cloud. Otherwise, the point is considered not to be the top-level Point Cloud.	5	[0.1, 1000]	mm	should be less than the height of Target

1.5 instance sorting

• Function introduction

Group, sort, and extract instances according to the selected strategy

• Use Scene

Universal scene for depalletizing, disordered grabbing, and orderly loading and unloading

**If sorting is not required, no specific strategy can be configured. **

1.5.1 datum coordinate system

• Function introduction

Set a unified coordinate system for all instances and group and sort the instances

• Use Scene

Common to destacking Scene, disordered grabbing Scene, and orderly loading and unloading Scene

When using coordinate-related strategies, you should first set the base coordinate system

• • • Parameter description * *

Parameter	Description	Illustration
Camera coordinate system	The origin of the coordinate system is above the object, and the positive Z-axis direction is downward; the XYZ value is the value of the center point of the object in this coordinate system
ROI coordinate system	The origin of the coordinate system is roughly at the center of the stack, with the positive direction of the Z axis pointing upward; the XYZ value is the value of the center point of the object in this coordinate system
Robot arm coordinate system	The origin of the coordinate system is on the robot arm itself, and the positive direction of the Z axis is generally upward; the XYZ value is the value of the center point of the object in this coordinate system
pixel coordinate system	The origin of the coordinate system is at the left vertex of the RGB image, which is a two-dimensional plane coordinate system; the values ​​of

1.5.2 general crawling strategy

• • • Parameter description * *

Parameter	Description	Default value
Strategy	Select the value used for group sorting and the sort order. Multiple criteria can be stacked, including instance Point Cloud center XYZ coordinate values, bounding box aspect ratio, and distance from the instance Point Cloud center to ROI center. They are executed sequentially.	instance Point Cloud center X coordinate value from small to large (mm)
grouping step size	Divide the instances into several groups according to the selected strategy and step size. The grouping step size is the interval between the two groups of instances. If the strategy selects "Instance Point Cloud center Z coordinate value from large to small (mm)", then the Z coordinates of all instance Point Cloud centers are sorted from large to small, and then the Z coordinates are grouped according to the step size. The corresponding instances are also divided into several groups.	/
extracts the first few groups	After is grouped and sorted, how many groups of instances need to be retained?	10000

policy name	Description	grouping step size		extracts the first few groups
		Default value	value range	Default value
instance Point Cloud center XYZ coordinate value from large to small/small to large (mm)	Use the XYZ coordinate value of the Point Cloud center of each instance for group sorting Before using this strategy for sorting, you should first set the base coordinate system	200.000	(0, 10000000]	10000
From the middle to both sides of the XY coordinate axis of the instance Point Cloud center/From both sides of the XY coordinate axis of the instance Point Cloud center to the middle (mm)	Use the XY coordinate value of the Point Cloud center of each instance to group and sort according to the direction of "middle to both sides" or "both sides to the middle" Before using this strategy for sorting, you should first set the reference coordinate system	200.000	(0, 10000000]	10000
Bounding box aspect ratio from large to small/small to large	Use the ratio of the long edge/wide edge of the bounding box for group sorting	1	(0, 10000]	10000
Mask area from large to small/small to large	Group sorting using mask area of ​​each instance	10000	[1, 10000000]	10000
instance Point Cloud center distance from ROI center near to far/far to near (mm)	Use the distance between the Point Cloud center of each instance and the center of the ROI coordinate system for group sorting	200.000	(0, 10000]	10000

• Example

1.5.3 Custom crawling strategy

(1) Function description

Switch the crawl strategy to custom crawl strategy and click New A custom crawling strategy can be added.

• Customize the order in which each Target is crawled. If it is difficult to achieve crawling using a general crawling strategy or it is difficult to adjust the appropriate Parameter due to issues such as Point Cloud noise, you can consider using a custom crawling strategy.

• The custom fetching strategy is applicable to the depalletizing Scene and the ordered loading and unloading Scene. It is not applicable to the unordered fetching Scene because the Targets of the custom fetching strategy must be in order (that is, the order of the Targets is fixed).

• Custom crawling strategies can only be used in combination with a single general crawling strategy, and the strategy can only select Z coordinates from small to large.

(2)Parameter description

Parameter	Description	Default value	value range	Parameter adjustment
IOUThreshold	represents the overlap degree Threshold between the marked bbox box and the detected bbox box. The overlap degree is used to determine which sorting method on the picture should be selected when the current Target instance is sorted.	0.7	[0,1]	Threshold The larger the match, the stricter the matching, and the worse the anti-interference performance will be. Small changes in shape or position may cause matching failure, and the wrong custom strategy may be matched and sorted in the wrong order.
Pixel Distance Threshold	represents the difference in size between the matched bbox box and the detected bbox box.	100	[0,1000]	The smaller the Threshold, the stricter the matching and the better the anti-interference performance. If the Target placement between different layers is similar, the custom strategy may be mismatched, resulting in an incorrect sorting order.

(3)Select the datum coordinate system

When using a custom crawling strategy, only Camera coordinate system or pixel coordinate system can be selected.

If there are multiple layers of Target, select the Camera coordinate system; if there is only one layer of Target, select the Pixel coordinate system.

(4) Strategy, grouping step size, extracting the first few groups

Parameter	Description	Default value
Strategy	can only select instance Point Cloud center Z coordinate value from large to small/small to large (mm)	/
grouping step size	Sort the Z coordinates of the instances from small to large according to the Z coordinate strategy from small to large, and divide the instances into several groups according to the step size	10000
extracts the first few groups	After is grouped and sorted, how many groups of instances need to be retained?	10000

(5) Take pictures/add local images

Click Take a picture to get an image from the currently connected Camera, or click Add local image to import an image from the local. How many layers or how many different placements of Target there are, depends on how many pictures you need to take a picture or add a local image. If each layer is the same, you only need one. Right-click the image to delete it.

Long press and drag the left mouse button on the acquired image to mark the bbox box. The DELETE key can gradually delete the marked bbox box.

2. 3D calculations

This section mainly explains the related functions that affect Pick Point generation and provides parameter adjustment suggestions.

2.1 Pretreatment

2.1.1 Point Cloud Clustering Denoising

• Function

Remove noise through Point Cloud clustering method

• Use Scene

There is a lot of noise in the instance Point Cloud.

• • • Parameter description * *

Parameter name	Description	Default value	value range	unit	Parameter adjustment recommendation
Point Cloud clustering distance Threshold (mm)	determines whether the Point Clouds in the space are of the same category. If the distance between Point Clouds is lower than the Threshold, they are of the same category.	5	[0.1, 1000]	mm	Generally, there is no need to change the point spacing that is greater than the TargetPoint Cloud and less than the minimum distance between the TargetPoint Cloud and the noise Point Cloud
Minimum point Threshold	Point Cloud clusters smaller than this number of points will be filtered	100	[1,10000000]	/	generally does not need to be changed. According to the noise level in the instance Point Cloud, increase the minimum number of points Threshold
Maximum pointsThreshold	Point Cloud clusters larger than this number will be filtered	100000	[1,10000000]	/	Generally does not need to be changed. If the number of TargetPoint Cloud is greater than 100000, increase the maximum point Threshold
Select the top point of ROI Cloud	If is checked, the Z coordinate average of the same category Point Cloud in the ROI coordinate system is calculated and sorted, and the Point Cloud category with the largest Z coordinate average (top Point Cloud) is retained. If not checked, all Point Clouds that meet the conditions are retained	Uncheck	/	/	If the TargetPoint Cloud is above the noise Point Cloud, check to retain the TargetPoint Cloud; if the TargetPoint Cloud is below the noise Point Cloud, you should adjust the Z axis of the ROI coordinate system downward to retain the TargetPoint Cloud
Visualizing process data	If checked, the denoised Point Cloud will be saved, which can be found in C:_data	Uncheck	/	/	In debugging mode, if you need to save visual data, you can check

2.1.2 Point Clouddownsampling

• Function

Sampling Point Cloud according to the specified point spacing to reduce the number of calculation points and improve model inference speed, but the accuracy may decrease

• Use Scene

There are too many actual ScenePoint Clouds, so check Point Clouddownsampling.

• • • Parameter description * *

Parameter	Description	Default value	Parameter range	unit
Point spacing of downsampling (mm)	samples Point Cloud according to the specified point spacing	5.0	[0.1, 1000]	mm

• Adjust parameters

  • The larger the value of downsampling point spacing is, the smaller the number of Point Clouds after downsampling will be, and the Pick Point calculation speed will be accelerated, but the accuracy may be reduced.

  • The smaller the value of downsampling point spacing is, the greater the number of Point Clouds after downsampling will be, and the Pick Point calculation speed will be reduced and the accuracy will be improved.

2.1.3 Calculate its own vertical posture

• Function

Calculate the attitude of the tilted sack in the vertical direction

• Use Scene

Depalletizing Scene Sacks placed at an angle

• • • Parameter description * *

Parameter	Description	Default value	Parameter range	unit
neighborhood range	The neighborhood range of the center point on the surface of the sack, the distance from the center point to the edge. This value affects the processing and analysis of the Point Cloud around the center point	20.00	[0, 100]	/
Point Cloud accuracy (mm)	In the 3DPoint Cloud image collected by Camera, the distance between adjacent points	10	[1, 1000]	mm
uses obb to calculate	If checked, use the z vector of the Point Cloudobb (directed bounding box) as the vertical direction of the sack itself. Otherwise, traverse the Point Cloud within the neighborhood of the center point, and select the largest axis among the projection axes from these Point Clouds as the vertical direction of the sack itself.	Uncheck	/	/

• Example

2.1.4 Filter Point Cloud based on color gamut hsv (hue, saturation, brightness)

• Function

Filter the Point Cloud according to the hue, saturation, and brightness in the Point Cloud map to select the Point Cloud area that matches the target range.

• • • Parameter description * *

Parameter name	Description	Default value	value range
Filter Depth by hsv - Gamut Max	Filters the maximum color value of Point Cloud	[0.9,0.9,0.9]	[[0,0,0],[1,1,1]]
filter depth according to hsv - color gamut minimum	Filters the minimum color value of Point Cloud	[0.0,0.0,0.0]	[[0,0,0],[1,1,1]]

Filter Point Cloud based on tee color

• Function

Filter the Point Cloud based on the tee color to select the Point Cloud area that matches the target range.

• • • Parameter description * *

Parameter name	Description	Default value	value range
Filter Point Cloud based on tee color - color maximum	Filters the maximum color value of Point Cloud	[0.9,0.9,0.9]	[[0,0,0],[1,1,1]]
Filter depth based on tee color - color minimum	Filters the minimum color value of Point Cloud	[0.0,0.0,0.0]	[[0,0,0],[1,1,1]]

2.1.5 Select Point Cloud in ROI area

• Function

Select the Point Cloud in the ROI 3D area from the instance Point Cloud. The default function cannot be deleted.

2.1.6 Optimize mask according to Point Cloud

• Function

Based on the Point Cloud in ROI 3D, eliminate the Point Cloud in the mask that is not in ROI 3D to improve the accuracy of the mask.

• Use Scene

Applicable to the destacking scene, the model recognizes the situation of adhesion outside the ROI 3D

2.1.7 Remove points whose normal is greater than the angle Threshold

• Function

Remove the Point Cloud where the angle between the normal vector and the standard normal vector axis is greater than the normal vector angle Threshold.

• Use Scene

Surface Target loading and unloading (materials are isolated from each other)

• • • Parameter description * *

Parameter name	Description	Default value	value range	unit
AngleThreshold	Point Cloud that is greater than the angle Threshold of is considered to be a different instance	15	[-360, 360]
standard normal vector axis	The angle between the normal vector of the Point Cloud and the axis of the standard normal vector	Z axis	X/Y/Z axis	/
Whether to use the ROI coordinate system	If checked, the angle between the normal vector and the axis of the ROI coordinate system is calculated. If not checked, the angle between the normal vector and the axis of the Camera coordinate system is calculated.	Uncheck	/	/

2.1.8 Point Cloud Plane Split

• Function

When performing plane fitting on a Point Cloud, if the distance between a point and the plane is lower than the reference distance, it is considered a point within the plane; otherwise, it is considered a point outside the plane, and the plane obtained by fitting can be retained or removed.

• Use Scene

Applicable to carton single disassembly scene, not applicable to sack single disassembly scene

• • • Parameter description * *

Default value: 3.000

Value range: [0.001, 10000]

Unit: mm

• Adjust parameters

The larger the reference distance, the thicker the fitted plane; the smaller the reference distance, the thinner the fitted plane.

If Remove Plane is checked, the plane with the largest number of Point Clouds will be removed. If unchecked, the plane with the largest number of Point Clouds will be retained.

• Example

2.1.9 Point Cloud removes outliers

• Function

Identify and remove outlier point noise in the adjacent point area (i.e. neighborhood) of each point in the Point Cloud

• Use Scene

The Point Cloud of the object contains more outlier noise

• • • Parameter description * *

Parameter	Description	Default value	Parameter range	Parameter adjustment recommendation
reference neighborhood points	Point Cloud The number of adjacent points to each point, that is, the neighborhood size. For a dense Point Cloud, even a small neighborhood is enough to reflect the characteristics of the object, so the value can be smaller; for a sparser Point Cloud, a larger neighborhood is needed to reflect the characteristics of the object, so the value should be larger.	30	[1, 10000000]
Standard deviation multiples	is used to identify outliers. If the coordinates of a point deviate from the average coordinate of the object Point Cloud by more than a multiple of the standard deviation, the point is considered an outlier. The smaller the value, the more points are considered outliers, and the more outlier points are removed, but this may lead to misjudgment and the removal of important features of the object; the larger the value, the fewer points are considered outliers, and the fewer outlier points are removed, but some outlier points may be retained and affect the accuracy of image recognition.	0.005	[0.0001, 2]	If Point Cloud is too sparse after removing outliers, the standard deviation multiple should be increased

• Example

2.1.10 Filter out Point Cloud whose object distance exceeds the limit

• Function

Filter out Point Cloud in the specified direction, remove noise, and improve the accuracy of Pick Point calculation.

• Use Scene

Usually used in single-disassembly scene of sacks

• • • Parameter description * *

Parameter	Description	Default value	Parameter range	unit	Parameter adjustment recommendation
specifies the axis	Point Cloud specifies the axis, used to filter out Point Cloud in the specified direction	Z axis	X/Y/Z axis	/	specifies the axis generally does not need to be changed
Threshold(mm)	In the specified axis direction, if the distance between the lower Point Cloud and the TargetPoint Cloud is greater than the Threshold, the lower Point Cloud will be filtered out; if the distance between the lower Point Cloud and the TargetPoint Cloud is less than the Threshold, the lower Point Cloud will be retained	750	[0, 1000]	mm	is adjusted according to the actual Scene. Threshold The smaller the Cloud; the smaller the Threshold, the more Point Clouds are filtered out
Select coordinate system	filters out Point Cloud in the selected coordinate system	ROI coordinate system	Camera coordinate system; ROI coordinate system; object own coordinate system	/

• Example

The Point Cloud carries the lower Point Cloud as shown in the figure below (left). Check Filter Point Cloud with object distance exceeding the limit and the Point Cloud after filtering out the lower Point Cloud is as shown in the figure below (right)

As shown in the figure below, comparing the effect of a Threshold of 100 and the detection results of a Threshold of 400, the larger the Threshold, the less Point Cloud is filtered out.

2.2 Sack pose generation/pose estimation

2.2.1 Rectangular Fitting-2D Rectangular Fitting Threshold(m)

• Function

The model calculates an ideal rectangle based on the instance Point Cloud. If the maximum distance between the ideal rectangle and the Target outline does not exceed the fitting Threshold, the rectangle fitting is successful. Points whose distance is smaller than the fitting Threshold will be fitted into a rectangle. If the distance exceeds the fitting Threshold, the rectangle fitting fails.

• Use Scene

Applicable to depalletizing Scene

• • • Parameter description * *

Default value: 2.0

Value range: [0,1000]

Unit: mm

• Adjust parameters

Generally, there is no adjustment. The Pick Point of the sack is generated at the center of the outline, and there is no need to fit the rectangle; the Pick Point of the carton is generated at the center of the fitting rectangle, and the fitting Threshold of the carton is generally 2 mm

• Example

2.2.2 Select coordinate system

• Function

The coordinate system when the Pick Point is generated.

• Use Scene

Universal destacking scene

• • • Parameter description * *

Default value: ROI coordinate system

Value range: Camera coordinate system, ROI coordinate system, object own coordinate system

• Adjust parameters

The default ROI coordinate system usually does not need to be changed. In the case of tilted stacking in the sack destacking scene, you need to check the object's own coordinate system. At the same time, calculate its own vertical posture should be added to the preprocessing of 3D calculations.

2.2.3 Neighborhood width (mm)

• Function

Search for points within the width range near the Pick Point, and use points in the nearby neighborhood to adjust the coordinate Z value of the Pick Point to prevent abnormal noise from affecting the capture.

• Use Scene

Common to destacking Scene, adjusted when there are bulges on the surface of the object

• • • Parameter description * *

Default value: 50

Value range: [0,1000]

Unit: mm

• Adjust parameters

  • The larger the value, the more accurate the Pick Point coordinates; if the value is too large, the model may mistakenly use the underlying layer or other objects to adjust the Pick Point coordinate z value.

  • If the value is too small, the Pick Point coordinates will be affected by noise, affecting the grabbing accuracy.

  • If the surface of the object in the actual scene is convex or concave, in order to make the Pick Point more stable, the width should be increased.

  • If the surface of the object in the actual Scene is smooth, no changes are needed. If you need to increase the model Inference speed, you can slightly reduce the width.

2.2.4 neighborhood length (mm)

• Function

Search for points within the length range near the Pick Point, and use points in the nearby neighborhood to adjust the coordinate Z value of the Pick Point to prevent abnormal noise from affecting the capture.

• Use Scene

Common to destacking Scene, adjusted when there are bulges on the surface of the object

• • • Parameter description * *

Default value: 50

Value range: [0,1000]

Unit: mm

• Adjust parameters

  • The larger the value, the more accurate the Pick Point coordinates; if the value is too large, the model may mistakenly use the underlying layer or other objects to adjust the Pick Point coordinate z value.

  • If the value is too small, the Pick Point coordinates will be affected by noise, affecting the grabbing accuracy.

  • If the surface of the Target Object in the actual Scene is convex or concave, in order to make the Pick Point more stable, the length should be increased.

  • If the surface of the Target Object in the actual Scene is smooth, no changes are needed. If you need to increase the model Inference speed, you can slightly adjust the length.

2.3 empty ROI judgment

• Function

Determine whether there is Target (Point Cloud) remaining in ROI 3D. If ROI If the number of 3D points in 3D is less than this value, it means that there is no TargetPoint Cloud remaining, and the Point Cloud will not be returned at this time.

• • • Parameter description * *

Default value: 1000

Value range: [0, 100000]

• Usage process

Set the minimum number of ROI 3D points to determine the Threshold. If it is less than the Threshold, it is the ROI. There is insufficient TargetPoint Cloud in 3D, so it is judged that there is no Target in ROI 3D;

In the Robot configuration, a new visual status code is added to facilitate subsequent signal processing by the Robot.

---

3. Pick Point

This section mainly explains functions related to Pick Point filtering and adjustment, along with Parameter tuning recommendations.

3.1 Pick Point Adjustment

3.1.1 Bound Euler Angle

• Function description

When the Picking Pose is outside the configured angle range, it is rotated counterclockwise by a certain angle around a fixed axis. If it is still outside the configured angle range after rotation, a warning is issued.

• Usage scenario

This function is only applicable to depalletizing scenarios. It can keep the robot's approach direction stable during picking and prevent the end effector from repeatedly rotating during the picking process. In 180° cases, it can prevent exceptions such as cable twisting.

• Parameter description

Parameter	Description	Default	Range	Unit
Fixed axis	An axis of the Picking Pose. The pose is rotated counterclockwise around this fixed axis	Z-axis	X/Y/Z-axis	/
Rotation angle	The angle by which the pose is rotated counterclockwise around the fixed axis. Adjust this angle so the Picking Pose satisfies the angle range	0	[-360,360]	degree
Angle range	The angle range of the Picking Pose. Set the angle range according to factors such as material placement, end effector type, and cycle time	[0,180]	[-180,180]	degree
Use current robot Euler Angles	By default, pose calculation uses Euler Angles "XYZ". When selected, the Euler Angles configured for the current robot are used so the pose remains consistent with the robot teach pendant.	Unchecked	/	/
Custom coordinate system	The coordinate system of the Picking Pose	Robot arm coordinate system	Default coordinate system; camera coordinate system; ROI coordinate system; robot arm coordinate system	/

• Example

Without using this function, the generated Pick Points are shown below.

When this function is used with the default values, the RZ angles of the Picking Poses for instances 0, 1, and 2 are all within the angle range [0,180], so no processing is performed. The RZ angle of the Picking Pose for instance 4 is -90°, which is outside the angle range [0,180], so the Picking Pose of instance 4 is rotated by 0° around the fixed Z-axis.

If you want to adjust the RZ angle of the Picking Pose for instance 4 into the angle range, you can change the rotation angle to 180 and rotate the Picking Pose of instance 4 by 180° around the fixed Z-axis.

---

3.1.2 Rotate the pose to align the rotation axis direction with the target axis direction

• Function description

Rotate the Picking Pose once around the fixed axis so that the direction of the rotation axis (determined by the right-hand rule) matches the positive or negative direction of the target axis in the target coordinate system.

• Usage scenario

Avoid collisions between the robot end effector and the bin.

• Parameter description

Parameter	Description	Default	Range
Rotation axis	An axis of the Picking Pose. Determined by the right-hand rule, the Picking Pose is rotated counterclockwise once around the fixed axis so that the direction of the rotation axis matches the positive or negative direction of the target axis in the target coordinate system	X-axis	X/Y/Z-axis
Fixed axis	The Picking Pose is rotated counterclockwise once around the fixed axis so that the direction of the rotation axis matches the positive or negative direction of the target axis in the target coordinate system	Z-axis	X/Y/Z-axis
Target axis	An axis of the target coordinate system. The Picking Pose is rotated counterclockwise once around the fixed axis so that the direction of the rotation axis matches the positive or negative direction of the target axis in the target coordinate system	X-axis	X/Y/Z-axis
Negative target axis direction	If selected, the direction of the rotation axis is aligned with the negative direction of the target axis in the target coordinate system; otherwise, it is aligned with the positive direction of the target axis in the target coordinate system	Unchecked	/
Custom coordinate system	The coordinate system of the Picking Pose	Default coordinate system	Default coordinate system; camera coordinate system; ROI coordinate system; robot arm coordinate system

3.1.3 Rotate Pose to Align Axis

• Function description

Rotate the Picking Pose around the fixed axis by 0, 90, 180, and 270 degrees respectively, calculate the angle between the rotated rotation axis and the positive or negative direction of the target axis in the camera coordinate system, and finally output the Picking Pose with the smallest angle after rotation.

• Usage scenario

Avoid collisions between the robot end effector and the bin.

• Parameter description

Parameter	Description	Default	Range
Fixed axis	An axis of the Picking Pose. Rotate the pose counterclockwise around this fixed axis	Z-axis	X/Y/Z-axis
Rotation axis	An axis of the Picking Pose. When rotating the pose, calculate the angle between this rotation axis and the positive or negative direction of the target axis	X-axis	X/Y/Z-axis
Target axis	An axis of the camera coordinate system. When rotating the pose, calculate the angle between the rotation axis and the positive or negative direction of this target axis	X-axis	X/Y/Z-axis
Negative target axis direction	If selected, calculate the angle between the rotation axis and the negative direction of the target axis; otherwise, calculate the angle between the rotation axis and the positive direction of the target axis	Selected	/
Custom coordinate system	The coordinate system of the Picking Pose	Default coordinate system	Default coordinate system; camera coordinate system; ROI coordinate system; robot arm coordinate system

• Example

---

3.1.4 Flip Pose to Align Axis

• Function description

Rotate the Picking Pose once around the fixed axis so that the angle formed between the rotation axis and the positive or negative direction of the target axis in the ROI coordinate system is acute.

• Usage scenario

Avoid collisions between the robot end effector and the bin.

• Parameter description

Parameter	Description	Default	Range
Fixed axis	An axis of the Picking Pose. Rotate the Picking Pose counterclockwise around this fixed axis	Z-axis	X/Y/Z-axis
Rotation axis	An axis of the Picking Pose. Rotate the Picking Pose so that the direction of this rotation axis matches the positive or negative direction of the target axis	X-axis	X/Y/Z-axis
Target axis	An axis in the ROI coordinate system. Rotate the Picking Pose so that the direction of the rotation axis matches the positive or negative direction of this target axis	X-axis	X/Y/Z-axis
Negative target axis direction	If selected, rotate the Picking Pose so that the direction of the rotation axis matches the negative direction of the target axis; otherwise, rotate the Picking Pose so that the direction of the rotation axis matches the positive direction of the target axis	Selected	/
Custom coordinate system	The coordinate system of the Picking Pose	Default coordinate system	Default coordinate system; camera coordinate system; ROI coordinate system; robot arm coordinate system

• Example

3.1.5 Flip Axis to ROI Center

• Function

Rotate the Picking Pose around a fixed axis so that the pointing axis of the Picking Pose points to the ROI center.

• Usage scenario

Avoid collisions between the robot end effector and the bin.

• Parameter description

Parameter	Description	Default	Range
Pointing axis	The axis in the Picking Pose that needs to be adjusted	X-axis	X/Y/Z-axis
Fixed axis	The axis that remains unchanged during rotation	Z-axis	X/Y/Z-axis
Reverse align	If selected, reverse-align the pointing axis to the ROI center; otherwise, align the pointing axis to the ROI center	Selected	/
Strict pointing	If selected, force the Picking Pose to rotate so the pointing axis points to the ROI center	Unchecked	/
Custom coordinate system	The coordinate system of the Picking Pose	Default coordinate system	Default coordinate system; camera coordinate system; ROI coordinate system; robot arm coordinate system

• Example

---

3.1.6 Transform to Target Pose

• Function description

Rotate the Picking Pose so that its Z-axis direction matches the Z-axis of the target coordinate system.

• Usage scenario

Usually this is used by default only in depalletizing scenarios and cannot be deleted. It is used to make the Z-axis of the Picking Pose perpendicular to the Z-axis of the ROI coordinate system (4-axis) or consistent with the direction of the Target Object surface (6-axis).

• Parameter description

Parameter	Description	Default	Range
Robot configuration	Set according to the on-site robot configuration. You can choose 4-axis or 6-axis. If a 6-axis robot is actually used as a 4-axis robot, it should be set to 4-axis	4-axis	4-axis/6-axis
Use ROI Z-axis as target direction	When the robot configuration is set to 4-axis, if selected, the pose is rotated around the X-axis so that the Z-axis direction of the rotated pose matches the positive direction of the ROI Z-axis ; if not selected, the pose is rotated around the X-axis so that the Z-axis direction of the rotated pose matches the positive direction of the Z-axis of the camera coordinate system . When the robot configuration is set to 6-axis, regardless of whether it is selected, the pose is rotated around the X-axis so that the Z-axis direction of the rotated pose matches the positive direction of the Z-axis of the object's own coordinate system	Unchecked	/
Custom coordinate system	The coordinate system of the Picking Pose	Camera coordinate system	Default coordinate system; camera coordinate system; ROI coordinate system; robot arm coordinate system

• Example

3.1.7 Rotate Pose with Angle

• Function description

Rotate the Picking Pose by a certain angle around a fixed axis.

• Usage scenario

Avoid collisions between the robot end effector and the bin.

• Parameter description

Parameter	Description	Default	Range	Unit
Rotation angle	The angle by which the pose is rotated counterclockwise around the fixed axis	90	[-360, 360]	degree°
Fixed axis	An axis of the Picking Pose. Rotate the pose counterclockwise around this fixed axis	Z-axis	X/Y/Z-axis	/
Custom coordinate system	The coordinate system of the Picking Pose	Default coordinate system	Default coordinate system; camera coordinate system; ROI coordinate system; robot arm coordinate system	/

• Example

3.1.8 Move Pose With Offset

• Function description

Move the Picking Pose by a certain distance along the translation axis.

• Usage scenario

Avoid collisions between the robot end effector and the bin.

• Parameter description

Parameter	Description	Default	Range	Unit
Translation amount (mm)	The distance the Picking Pose moves along the translation axis. A positive translation amount means translating in the positive direction of the translation axis, and a negative translation amount means translating in the negative direction of the translation axis	0	[-1000, 1000]	mm
Translation axis	The direction in which the Picking Pose moves	X-axis	X/Y/Z-axis	/
Custom coordinate system	The coordinate system of the Picking Pose	Robot arm coordinate system	Default coordinate system; camera coordinate system; ROI coordinate system; robot arm coordinate system	/

• Example

3.1.9 Transform Pose by Difference

• Function description

Record the Pick Point coordinates generated by the software and the Pick Point coordinates taught under the current operating condition, then output the transformed Picking Pose based on the offset between the two.

• Usage scenario

When the Pick Points generated by the vision system have an obvious systematic offset and the robot TCP coordinate accuracy is limited or difficult to calibrate, this method can be used to directly map the same offset pattern to subsequent Pick Points, thereby avoiding robot TCP calibration.

• Parameter description

Parameter	Description	Default	Range
Vision Pose	Pick coordinates of the detection result
X(mm)	X coordinate of the Vision Pose	0.00	±10000000, meaning no limit.
Y(mm)	Y coordinate of the Vision Pose	0.00	±10000000, meaning no limit.
Z(mm)	Z coordinate of the Vision Pose	0.00	±10000000, meaning no limit.
RX(°)	X-axis rotation amount of the Vision Pose	0.00	±180
RY(°)	Y-axis rotation amount of the Vision Pose	0.00	±180
RZ(°)	Z-axis rotation amount of the Vision Pose	0.00	±180
Picking Pose	Manually taught Pick Point
X(mm)	X coordinate of the Picking Pose	0.00	±10000000, meaning no limit.
Y(mm)	Y coordinate of the Picking Pose	0.00	±10000000, meaning no limit.
Z(mm)	Z coordinate of the Picking Pose	0.00	±10000000, meaning no limit.
RX(°)	X-axis rotation amount of the Picking Pose	0.00	±180
RY(°)	Y-axis rotation amount of the Picking Pose	0.00	±180
RZ(°)	Z-axis rotation amount of the Picking Pose	0.00	±180

3.1.10 Refine Pose by Plane Normal

• Function description

Correct the Object Pose by fitting the plane Normal so that the Z-axis direction of the Object Pose remains consistent with the direction of the plane Normal of the Target Object.

• Usage scenario

When the Target Object contains a plane and there is a tilt deviation in the plane when the template Point Cloud is matched with the actual Point Cloud, use this function to fine-tune the Target Object plane and improve picking accuracy.

Not applicable to depalletizing scenarios

• Parameter description

Parameter	Description	Default	Range	Unit
Distance Threshold	Distance Threshold for fitting a plane from the Point Cloud	10	[-1000, 1000]	mm
Save visualization data	If selected, the visualization data will be saved under the historical data timestamp	Selected	/	/
Custom coordinate system	The coordinate system of the Picking Pose	Camera coordinate system	Default coordinate system; camera coordinate system; ROI coordinate system; robot arm coordinate system	/

• Example

3.1.11 Sort Pick Points by Angle with ROI Axis

• Function

Sort Pick Points according to the angle between an axis of the Picking Pose and the target axis of the ROI.

• Parameter description

Parameter	Description	Default	Range
Axis selection	An axis of the Picking Pose	Z-axis	X/Y/Z-axis
Target axis selection	An axis of the ROI coordinate system	Z-axis	X/Y/Z-axis
Select reverse direction	If selected, calculate the angle with the negative direction of the target axis; otherwise, calculate the angle with the positive direction of the target axis	Unchecked	/
Select descending order	If selected, sort Pick Points from small to large by angle; otherwise, sort Pick Points from large to small by angle	Unchecked	/

3.1.12 [Advanced] Rotate Pose, Automatically Compensate for Grasp Orientation Angles with Excessive Deviation from the Specified Axis

• Function description

Determine whether the angle formed between the specified axis of the Picking Pose and the target axis is within the specified range. If not, adjust the Picking Pose into the specified range.

• Usage scenario

Avoid collisions between the robot end effector and the bin.

• Parameter description

Parameter	Description	Default	Range	Unit
Angle range	Adjust the Picking Pose into the angle range	30	[0, 180]	degree°
Specified axis	An axis of the Picking Pose. Adjust this axis so that it falls within the angle range relative to the target axis of the ROI coordinate system	Z-axis	X/Y/Z-axis	/
Target axis	An axis of the ROI coordinate system. Compare the angle range with the specified axis of the Picking Pose	Z-axis	X/Y/Z-axis	/
Compare with the negative half-axis of the ROI	If not selected, compare the angle range with the positive direction of the target axis of the ROI coordinate system; if selected, compare the angle range with the negative direction of the target axis of the ROI coordinate system	Unchecked	/	/
Custom coordinate system	The coordinate system of the Picking Pose	Default coordinate system	Default coordinate system; camera coordinate system; ROI coordinate system; robot arm coordinate system	/

3.1.13 [Advanced] Symmetry Center Pose Optimization

• Function

Search for the symmetry center of the Target Object based on the instance Mask, then combine it with the plane of the instance or the pose of the ROI 3D center point to calculate the optimal Picking Pose.

Before using this function, first make sure the instance Mask is symmetrical

• Usage scenario

Applicable when the instance Mask of a symmetrical Target Object is also symmetrical, but the Picking Pose is not near the expected center; at the same time, the Target Object has a plane that can be used as a reference, for example, there is a plane on the top of the object, or ROI 3D can be used as a reference for the projected pose.

Applicable project scenarios include brake discs (general circles), refractory bricks (depalletizing), symmetrical irregular parts, fuel fillers, and so on.

• Parameter description

Parameter	Description	Default	Range	Tuning recommendation
Target Object Symmetry type	Target Object Symmetry type of the instance Mask	Rotational symmetry	Rotational symmetry: after the Target Object rotates by a certain angle around the center point, its shape completely overlaps with the original position; mirror symmetry: the Target Object uses a certain axis / plane as the mirror, and the left-right or upper-lower sides are completely symmetrical.	Circles and rectangles are both rotationally symmetrical and mirror-symmetrical, so rotational symmetry is preferred; for trapezoids and other shapes that are symmetrical only along a certain axis or plane, choose mirror symmetry.
Gaussian blur level	Tolerance for determining whether the actual Point Cloud overlaps after rotation	3	\[1,99\]	The larger the value, the stronger the blur, the more smoothing is applied, and the more lenient the overlap judgment after rotation; The smaller the value, the weaker the blur, the less smoothing is applied, and the stricter the overlap judgment. An odd number must be entered
Rotation angle setting	When the symmetry mode is rotational symmetry, it indicates the rotation angle interval, that is, the angle difference between two adjacent rotations. When the symmetry mode is mirror symmetry, it indicates the rotation range, that is, the angle interval within which the Point Cloud can rotate around the symmetry axis.	180	\[1,360\]	For rotational symmetry, set it based on the symmetry of the Target Object. A circle is symmetrical at any angle, so 60 degrees is recommended; A square has 90-degree symmetry, so it can be set to 90 degrees or 180 degrees; A rectangle has 180-degree symmetry, so set it to 180. For mirror symmetry, adjust it within 5° to ensure fine tuning near the symmetry axis
Image scaling ratio	Adjusts the size of the Point Cloud image. The larger this ratio, the smaller the Point Cloud image size and the lower the GPU memory usage, but image detail loss increases, resulting in reduced calculation accuracy .	2	\[1,10000000\]
Search range	Based on the initially determined center of the Target Object, this defines the range expanded outward to search for Point Cloud features. The actual range is (search range*2*image scaling ratio)	10	\[1,10000000\]	For example, for a square Target Object, the initially determined center position of the Target Object is point O. If the search range is set to 10 and the image scaling ratio is 1, then the actual search range is a square region centered at point O with a side length of 10×2×1=20. Point Cloud features are searched within this region to further determine the symmetry center of the Target Object and the optimal Picking Pose. As another example, for a circular Target Object, if the search range is set to 8 and the image scaling ratio is 2, then the actual search range is a circular region centered at the initially determined center of the Target Object with a diameter of 8×2×2=32. Point Cloud features are searched within this region to further determine the Object Pose of the Target Object and the optimal Picking Pose.
Use ROI3D as the reference projection plane	If selected, ROI3D is used as the reference projection plane	Unchecked	/	Select this when the Point Cloud has no obvious plane and the projection plane is difficult to determine; leave it unchecked when the Point Cloud has a clear plane.
Save symmetry center process data	If selected, the debug data generated during the symmetry center process is saved. You can view it in the `\ProjectName`\data`\PickLight`\HistoricalDataTimestamp`\find`\_symmetry_center folder	Unchecked	/	Select this when you need to inspect the detailed process images
Symmetry axis prior type	Effective in ``{=html}mirror symmetry``{=html} mode. Specifies the known Target Object Symmetry type and fixes the asymmetric orientation	Automatic search	Automatic searchSymmetric along the long axisSymmetric along the short axis	If the symmetry axis of the Target Object is the long axis, choose "Symmetric along the long axis". If the symmetry axis of the Target Object is the short axis, choose "Symmetric along the short axis". If uncertain, choose "Automatic search"
Pose adjustment type	Whether to inherit pose-related information from the input pose	Default pose	Default poseInherit rotationInherit translation	/
Symmetry score Threshold	Symmetry results with a symmetry score lower than this Threshold are abnormal results. When set to 0, no filtering is performed	0.0	\[0.0, 1.0\]	/

• Example

3.2 Pick Point Filtering

3.2.1 Filter by Fine Matching Score

• Function description

Filter Pick Points based on the pose fine matching score.

• Parameter description

Parameter	Description	Default	Range
Score Threshold	Retain Pick Points whose fine matching score is greater than this Threshold	0.5	[0, 1]

3.2.2 Filter grasp points of occluded artifacts

• Function description

Determine whether there are too many occluding object Point Clouds in the target detection area along the specified ROI axis or the Picking Pose axis at the Pick Point of the grasped Target Object. If so, the Target Object is considered occluded and the Pick Point is filtered out.

• Usage scenario

Applicable to depalletizing and ordered scenarios in which Target Objects are picked layer by layer, but the model recognizes lower-layer Target Objects. When picking a lower-layer Target Object, the gripper may collide with the upper-layer Target Object.

• Parameter description

Parameter	Description	Default	Range	Unit
Cuboid length in X direction	Set the cuboid length in the X direction of the Picking Pose	1500	[1, 10000]	mm
Cuboid length in Y direction	Set the cuboid length in the Y direction of the Picking Pose	1500	[1, 10000]	mm
Cuboid length in Z direction	Set the cuboid length in the Z direction of the Picking Pose	800	[1, 10000]	mm
Distance Threshold between detection area and Pick Point origin	Along the ROI axis, the nearby cuboid surface area farther than this distance Threshold from the Pick Point origin is regarded as the target detection area	50	[1, 1000]	mm
Point Cloud count Threshold in detection area	If the number of occluding object Point Clouds in the target detection area exceeds this Threshold, the Pick Point is considered occluded	1000	[0, 100000]	/
Specified axis direction	Based on the pose reference specified axis direction, set the specific location of the target detection area within the cuboid space (for example, near the front/back/left/right/top/bottom surface of the cuboid)	[0,0,-1]	[1,0,0]: positive X-axis[-1,0,0]: negative X-axis[0,1,0]: positive Y-axis[0,-1,0]: negative Y-axis[0,0,1]: positive Z-axis[0,0,-1]: negative Z-axis	/
Use ROI 3D pose reference	If selected, adjust the collision detection area according to the ROI 3D pose reference	Unchecked	/	/
Save visualization data	If selected, the visualization data is stored according to the saved data path to help observe whether the generated cuboid is reasonable; if not selected, it is not saved	Unchecked	/	/

3.2.3 Filter Pose by Cone

• Function description

Determine whether the angle of the Picking Pose is within the constrained angle range, and filter out all Pick Points that do not meet the condition.

• Usage scenario

Prevent collisions caused by abnormal robot arm Picking Pose angles.

• Parameter description

Parameter	Description	Default	Range	Unit
Angle filtering Threshold	Calculate the maximum angle between the specified axis of the ROI and the specified axis of the Picking Pose. Pick Points whose angle is greater than the current Threshold will be filtered out	30	[-360, 360]	degree°
Invert ROI specified axis direction	If selected, use the negative direction of the specified ROI axis for angle calculation; otherwise, use the positive direction of the specified ROI axis for angle calculation	Selected	/	/
Specified Picking Pose axis	Specify an axis of the Picking Pose for angle calculation	Z-axis	X/Y/Z-axis	/
Specified ROI axis	Specify an axis of the ROI coordinate system for angle calculation	Z-axis	X/Y/Z-axis	/

3.2.4 Filter Pick Points outside the ROI 3D type area

• Function description

Determine whether the Pick Point is within the ROI 3D range, and remove Pick Points that are outside the ROI 3D area.

• Usage scenario

Prevent picking outside the ROI area, which may cause collisions between the robot arm and the target object.

• Parameter description

Parameter	Description	Default
ROI3D type region	Usually "workspace"; "pick area" is a smaller ROI region than "workspace", which can restrict Pick Points to an ROI region smaller than the "workspace" to avoid some collision cases.	Workspace

• Example

As shown in the figure below, when the ROI3D area and ROI2D are area a, the corresponding Pick Point is in the upper-right corner.

When the ROI3D area and ROI2D are changed to area b, the original Pick Point is outside the ROI area, so that Pick Point is removed and a new Pick Point is generated within area b.

---

3.2.5 [New] Gripper Collision Detection

[New] Gripper Collision Detection

• Function description

Collision detection between the gripper and the Point Cloud near the Pick Point. If the number of Point Clouds in contact with the gripper exceeds the pick collision Threshold, the Pick Point of the Target Object is considered to have a collision risk.

• Usage scenario

Used when collision detection is required between the gripper and the Point Cloud near the Target Object being picked.

• Parameter description

Parameter	Description	Default	Range
Collision Threshold	Collision distance Threshold. If the distance between the scene and the gripper surface is smaller than this Threshold, it is considered a collision. The larger the Threshold, the stricter it is. Unit: mm	7	1-1000
Collision Point Cloud sampling	Sampling size for collision Point Clouds. The larger the value, the faster the cycle time; the smaller the value, the slower the cycle time. Effective only in "Target Object scene Point Cloud only" and "bin + Target Object scene Point Cloud" modes. Unit: mm	5	1 - 1000
Save visualization data for gripper collision detection	Save visualization data for collision detection between the gripper and the picked Target Object	Unchecked	Selected/Unchecked

Gripper Collision Detection

• Function description

Collision detection between the gripper and the Point Cloud near the Pick Point. If the number of Point Clouds in contact with the gripper exceeds the pick collision Threshold, the Pick Point of the Target Object is considered to have a collision risk.

• Usage scenario

Used when collision detection is required between the gripper and the Point Cloud near the Target Object being picked.

• Parameter description

Parameter	Description	Default	Range
Pick collision Threshold	The maximum number of Point Clouds the gripper may contain near the Pick Point. For example, 20 means that if the number of scene Point Clouds contained by the gripper exceeds 20, it is considered a collision	20	0-10000
Collision Point Cloud sampling (m)	Downsampling size of the Point Cloud in the collision area. The larger the value, the faster the detection speed, but the lower the accuracy. Applicable scenario: scenarios requiring high cycle rates	0.002	0.0001 - 0.5000
Save visualization data for gripper collision detection	Save visualization data for collision detection between the gripper and the picked Target Object	Unchecked	Selected/Unchecked
Import gripper model	Select and import the gripper model used for collision detection from a folder	/	/

**The gripper should be simplified to fewer than 500 faces**

3.2.6 [Master] Retain the one Pick Point with the largest/smallest pose value among instance Pick Points and filter the remaining Pick Points

• Function description

Convert the pose to the specified coordinate system, sort poses according to the value of the specified sorting axis, and retain the pose with the maximum or minimum value. This is suitable for cylindrical Target Objects when keeping the top or bottom Pick Point.

• Parameter description

Parameter	Description	Default	Range
Specified coordinate system	Select which coordinate system the pose should be converted to for processing	ROI coordinate system	ROI coordinate system/camera coordinate system
Specified sorting axis	Select which axis value of the pose to sort by	Z-axis	X/Y/Z-axis
Take minimum value	If selected, retain the pose with the minimum value on the sorting axis; otherwise, retain the pose with the maximum value on the sorting axis	Unchecked	/

3.2.7 [Master] Filter grasp points similar to the previous N grasp points

• Function description

If the variation between the current Pick Point and any Pick Point in the cache is within the Threshold range, the Pick Point will be filtered out.

• Parameter description

Parameter	Description	Default	Range	Unit
Upper limit of Pick Point change (+)
X(mm)	Upper limit of X coordinate	2	[0, 10000000]	mm
Y(mm)	Upper limit of Y coordinate	2	[0, 10000000]	mm
Z(mm)	Upper limit of Z coordinate	2	[0, 10000000]	mm
RX(°)	Upper limit of RX rotation amount	1	[0, 180]	degree°
RY(°)	Lower limit of RY rotation amount	1	[0, 180]	degree°
RZ(°)	Lower limit of RZ rotation amount	1	[0, 180]	degree°
Lower limit of Pick Point change (-)
X(mm)	Lower limit of X coordinate	2	[0, 10000000]	mm
Y(mm)	Lower limit of Y coordinate	2	[0, 10000000]	mm
Z(mm)	Lower limit of Z coordinate	2	[0, 10000000]	mm
RX(°)	Lower limit of RX rotation amount	1	[0, 180]	degree°
RY(°)	Lower limit of RY rotation amount	1	[0, 180]	degree°
RZ(°)	Lower limit of RZ rotation amount	1	[0, 180]	degree°
Pick Point cache count	Number of Pick Points cached. After the current Pick Point comparison is completed, it will be added to the cache in real time	5	[1, 100]	/

3.2.8 [Master] Filter artifact poses similar to the previous N artifact poses

• Function description

If the variation between the current Object Pose and any Object Pose in the cache is within the Threshold range, the Object Pose will be filtered out. When an Object Pose is determined to be similar, all Pick Points on that Target Object will be filtered out.

• Parameter description

Parameter	Description	Default	Range	Unit
Upper limit of Object Pose change (+)
X(mm)	Upper limit of X coordinate	2	[0, 10000000]	mm
Y(mm)	Upper limit of Y coordinate	2	[0, 10000000]	mm
Z(mm)	Upper limit of Z coordinate	2	[0, 10000000]	mm
RX(°)	Upper limit of RX rotation amount	1	[0, 180]	degree°
RY(°)	Lower limit of RY rotation amount	1	[0, 180]	degree°
RZ(°)	Lower limit of RZ rotation amount	1	[0, 180]	degree°
Lower limit of Object Pose change (-)
X(mm)	Lower limit of X coordinate	2	[0, 10000000]	mm
Y(mm)	Lower limit of Y coordinate	2	[0, 10000000]	mm
Z(mm)	Lower limit of Z coordinate	2	[0, 10000000]	mm
RX(°)	Lower limit of RX rotation amount	1	[0, 180]	degree°
RY(°)	Lower limit of RY rotation amount	1	[0, 180]	degree°
RZ(°)	Lower limit of RZ rotation amount	1	[0, 180]	degree°
Object Pose cache count	Number of vision Object Poses cached. After the comparison of the current Object Pose is completed, it will be added to the cache in real time	5	[1, 100]	/

3.2.9 [Master] Filter grasp points outside the upper and lower limits of the grasp coordinates

• Function description

Retain other Pick Points within the specified range of a reference Pick Point and filter out abnormal Pick Points.

• Usage scenario

Prevent incorrect robot picking and ensure picking accuracy.

This function is not applicable to depalletizing scenarios

• Parameter description

Parameter	Description	Default	Unit
Reference Pick coordinates
X(mm)	X coordinate of the reference Pick Point	0	mm
Y(mm)	Y coordinate of the reference Pick Point	0	mm
Z(mm)	Z coordinate of the reference Pick Point	0	mm
RX(°)	RX rotation amount of the reference Pick Point	0	degree
RY(°)	RY rotation amount of the reference Pick Point	0	degree
RZ(°)	RZ rotation amount of the reference Pick Point	0	degree
Upper limit of Pick coordinates (+)
X(mm)	Upper limit of the X coordinate. For example, if the X coordinate of the reference Pick Point is 100 and the upper limit is set to 10, the allowed range is: [100-lower limit, 110]	10000000, meaning no limit.	mm
Y(mm)	Upper limit of the Y coordinate. For example, if the Y coordinate of the reference Pick Point is 100 and the upper limit is set to 10, the allowed range is: [100-lower limit, 110]	10000000	mm
Z(mm)	Upper limit of the Z coordinate. For example, if the Z coordinate of the reference Pick Point is 100 and the upper limit is set to 10, the allowed range is: [100-lower limit, 110]	10000000	mm
RX(°)	Upper limit of the RX rotation amount. For example, if the RX rotation amount of the reference Pick Point is 180 and the upper limit is set to 10, the allowed range is (default angle wraparound applies): [[-180, -170], [180-lower limit, 180]]	180, meaning no limit.	degree°
RY(°)	Upper limit of the RY rotation amount. For example, if the RY rotation amount of the reference Pick Point is 180 and the upper limit is set to 10, the allowed range is (default angle wraparound applies): [[-180, -170], [180-lower limit, 180]]	180	degree°
RZ(°)	Upper limit of the RZ rotation amount. For example, if the RZ rotation amount of the reference Pick Point is 180 and the upper limit is set to 10, the allowed range is (default angle wraparound applies): [[-180, -170], [180-lower limit, 180]]	180	degree°
Lower limit of Pick coordinates (-)
X(mm)	Lower limit of the X coordinate. For example, if the X coordinate of the reference Pick Point is 100 and the lower limit is set to 10, the allowed range is: [100-lower limit value, 110]	10000000	mm
Y(mm)	Lower limit of the Y coordinate. For example, if the Y coordinate of the reference Pick Point is 100 and the lower limit is set to 10, the allowed range is: [100-lower limit, 110]	10000000	mm
Z(mm)	Lower limit of the Z coordinate. For example, if the Z coordinate of the reference Pick Point is 100 and the lower limit is set to 10, the allowed range is: [100-lower limit, 110]	10000000	mm
RX(°)	Lower limit of the RX rotation amount. For example, if the RX rotation amount of the reference Pick Point is 180 and the lower limit is set to 10, the allowed range is (default angle wraparound applies): [[-180, -180+upper limit], [170, 180]]	180, meaning no limit.	degree°
RY(°)	Lower limit of the RY rotation amount. For example, if the RY rotation amount of the reference Pick Point is 180 and the lower limit is set to 10, the allowed range is (default angle wraparound applies): [[-180, -180+upper limit], [170, 180]]	180	degree°
RZ(°)	Lower limit of the RZ rotation amount. For example, if the RZ rotation amount of the reference Pick Point is 180 and the lower limit is set to 10, the allowed range is (default angle wraparound applies): [[-180, -180+upper limit], [170, 180]]	180	degree°

---

3.3 Pick Point Sorting

3.3.1 Base Coords

• Function description

Set a unified coordinate system for all instances to group and sort instances.

• Usage scenario

Common to depalletizing scenarios, random picking scenarios, and ordered loading/unloading scenarios

Strategies related to coordinates should first set the reference coordinate system

• Parameter description

Parameter	Description	Illustration
Camera coordinate system	The coordinate system origin is above the object, and the positive Z-axis direction points downward; the XYZ values are the values of the center point of the object in this coordinate system
ROI coordinate system	The coordinate system origin is approximately at the center of the pallet stack, and the positive Z-axis direction points upward; the XYZ values are the values of the center point of the object in this coordinate system
Robot arm coordinate system	The coordinate system origin is on the robot arm itself, and the positive Z-axis direction generally points upward; the XYZ values are the values of the center point of the object in this coordinate system
Pixel coordinate system	The coordinate system origin is at the top-left vertex of the RGB image and is a 2D planar coordinate system; the X and Y values are the x value of the bbox detection box and the y value of the bbox detection box, and Z is 0

3.3.2 Picking strategy

• Parameter description

Parameter	Description
Strategy	Select which value is used for grouping and sorting and how to sort it, including Pick Point center X/Y/Z coordinate values from large to small/from small to large (mm), from the middle to the sides / from the sides to the middle along the Pick Point XY coordinate axis (mm). Multiple items can be superimposed and executed in order.
Grouping step size	According to the selected strategy, divide Pick Points into several groups based on the step size. The grouping step size is the interval between two groups of Pick Points
Number of leading groups to keep	After grouping and sorting, how many groups of instances need to be retained

Strategy name*	Description	Grouping step size		Number of leading groups to keep
		Default	Range	Default
Pick Point center X/Y/Z coordinate values from large to small / from small to large (mm)	Use the X/Y/Z coordinate values of the Pick Point center for grouping and sorting	200.000	[0, 10000000]	10000
From the middle to the sides / from the sides to the middle along the Pick Point XY coordinate axis (mm)	Use the X/Y coordinate values of the Pick Point center and perform grouping and sorting in the direction of "middle to sides" or "sides to middle"	200.000	[0, 10000000]	10000

3.3.3 Carton combination strategy

To solve the problems of low efficiency and limited applicable scenarios in traditional single-pick depalletizing, PickWiz adds a carton combination strategy in depalletizing scenarios to support picking multiple Target Objects in a single operation. It supports the core scenarios of "cartons with consistent dimensions" and "rectangular suction cups", covering more real project scenarios.

3.3.3.1 Multi-pick runtime configuration

（1）In sack single depalletizing or carton single depalletizing scenarios, enable Vision computation configuration - Vision computation acceleration.

（2）Under the Pick Point sorting module, select the carton combination strategy;

（3）Strategy selection: available options are the default combination strategy or combination along a specified carton pose axis. These are two methods for finding the largest number of cartons that can be combined.

• Default combination strategy: Find the largest number of cartons that can be combined along the X-axis and Y-axis directions of a carton Picking Pose.

• Combine along a certain carton pose axis: Find the largest number of cartons that can be combined along the X-axis or Y-axis direction of a carton Picking Pose. This is suitable for scenarios where cartons are arranged in a straight line. When using this strategy, you need to choose the carton combination direction, namely the Picking Pose X-axis or the Picking Pose Y-axis.

Note:

The combination direction is related to the positive and negative axis directions. Cartons can be combined only when they are on the same axis and in the same direction, and after combination, the orientation of the whole stack of cartons remains consistent with the orientation of a single carton before combination. Therefore, before combining cartons, make sure all cartons to be combined are placed in the same orientation, and unify the coordinates of the cartons to be combined to the same axis direction.

（4）Combination conditions: determine which cartons can be combined and how many can be combined at most.

• Maximum cartons per row: the maximum number of cartons that can be combined in one row, default is 2.

• Maximum number of combination rows: the maximum number of carton rows that can be combined, default is 1.

• Maximum spacing (mm): cartons to be combined cannot be too far apart in the "combination direction", In the combination direction (axis direction), when the spacing between two adjacent cartons or cartons in different rows is less than this value, they can be combined into one group. The default is 10.

  • Example: when searching for the maximum number of cartons along the Picking Pose X-axis, if the spacing between two adjacent cartons in the Picking Pose X-axis direction is 8 mm (≤10), they can be combined; if the spacing is 12 mm (>10), they cannot be combined.

• Maximum misalignment distance (mm): cartons to be combined cannot be too far apart in the direction "perpendicular to the combination direction" . In the direction perpendicular to the combination direction (axis direction), when the misalignment distance between two adjacent cartons or cartons in different rows is less than this value, they can be combined into one group. The default is 10.

  • Example: when searching for the maximum number of cartons along the Picking Pose X-axis, if two adjacent cartons are offset in the Picking Pose Y-axis direction by 8 mm (≤10), they can be combined; if they are offset by 15 mm (>10), they are no longer aligned and cannot be picked together.

• Maximum angular deviation (°): cartons to be combined should face almost the same direction. In the combination direction (axis direction), when the rotational deviation angle of the cartons is less than this value, they can be combined into one group. The default is 10.

  • Example: if a carton is rotated by 5° relative to the combination direction, as long as it does not exceed 10°, it can be combined; if it is rotated by 15° (>10), the orientation differs too much, the robot will be skewed when picking, and it cannot be combined.

3.3.3.2 Robot configuration

（1）On the robot configuration page, add new placeholders in Vision computation communication message - Robot to PickWiz commands - Vision detection send command: maximum cartons per row and maximum number of combination rows, as shown below;

（2）In Vision computation communication message - PickWiz to robot commands - Pick-related information - Returned information when picking Target Objects, add Object Dimensions length, Object Dimensions width, and Target Object orientation.

After the robot configuration is completed, click the Run button.

3.3.3.3 View multi-pick runtime results

（1）In the 3D Matching window, hover the mouse over an instance to view the combined picking information of a single instance after carton combination, including 2D recognition results, Picking Pose, and instance combination information.

In the visualization window, click the Settings button in the upper right corner to set how the combined instance information is displayed.

Right-click an instance to view the combined picking information and Target Object information of the single instance.

（2）In the 2D recognition window, you can use the relevant combination buttons in the menu bar to view the combined ID, combined Mask, and combined bounding box.

Previous pageCalibration Result Inspection and Analysis

Next pageGeneral Target Object Visual Parameter Tuning Guide