Visual Parameter Adjustment Guide for Ordered Loading/Unloading and Positioning Assembly of Planar Workpieces (Image Matching)

About 8777 wordsAbout 29 min

This article mainly explains how to adjust visual parameters according to actual scenarios for planar workpiece loading/unloading (image matching) and planar workpiece positioning assembly (image matching only)

Getting Started

Ordered planar workpiece loading/unloading scenario (image matching): uses a pre-trained large model together with template examples for detection, enabling immediate deployment without one-click model training and connection; at the same time, high-precision matching is completed through the correspondence between 2D image examples and scene image features, thereby achieving pose estimation. It is suitable for ordered planar workpiece loading/unloading scenarios.

Planar workpiece positioning assembly (image matching only): high-precision matching is completed through the correspondence between 2D image examples and scene image features, thereby achieving pose estimation. It is suitable for high-precision positioning when there is only a single planar workpiece within the field of view.

Build the Project

(1) Create a new ordered planar workpiece loading/unloading scenario (image matching) / planar workpiece positioning assembly (image matching only) Project (the Project name and Project path can be customized, but the Project name cannot contain Chinese characters)

Optional functional options include collision detection, bin and workpiece collision detection, visual classification, point cloud denoising, and instance optimization

(2) Camera and Robot configuration

(3) Add the workpiece

In the template folder of the workpiece, you need to first create a point cloud template in Menu - Point Cloud Template Creation. For details, see Point Cloud Template Creation

The template folder path is the template data generation path selected during Point Cloud Template Creation

(4) Add Tool, eye-hand calibration, and ROI

Point Cloud Template Creation

When the Project type is “Ordered Planar Workpiece Loading/Unloading (Image Matching)” or “Planar Workpiece Positioning Assembly (Image Matching Only)”, you need to create a point cloud template in “Menu” — “Point Cloud Template Creation” using “Generate Template from Historical Data”.

Source data source: historical data folder

Source data path: select a complete historical data folder, which is usually in the project folder\data\PickLight\历史数据\Builder\foreground\input

Template data generation path: by default, it is generated in the current Project's data folder, and can be changed to another local folder.

Select the required workpiece ROI in the left viewport, click Generate Workpiece Point Cloud to generate the workpiece point cloud, and a point cloud will appear in the left window. Use the point cloud editing tools in the upper-right corner to edit the required workpiece point cloud.

After the workpiece point cloud is created, click Generate Template Data to complete template generation.

Source data source: on-site acquisition

Make sure the Camera is connected and configured in the task information section, click Capture Image to capture an image and generate a point cloud image, select the ROI of the workpiece position, click Generate Workpiece Point Cloud to generate the workpiece point cloud, and a point cloud will appear in the left window. Use the point cloud editing tools in the upper-right corner to edit the required workpiece point cloud.

After the workpiece point cloud is created, click Generate Template Data to complete template creation.

After the point cloud template is generated, you need to move the Pick Point to the center of the workpiece in the workpiece configuration. Changing the coordinate system to the workpiece center in meshlab will cause a task error.

Visual Parameters

2D recognition: recognize and segment instances from the actual scene

Preprocessing: process the 2D image before instance segmentation

Template-based instance segmentation: recognize and segment instances

Point cloud generation: methods for generating instance point clouds; use the segmented instance Mask or bounding box to generate the instance point cloud, or use the filtered instance Mask or bounding box to generate the instance point cloud

Instance filtering: filter segmented instances

Instance sorting: sort instances; the picking order follows the instance sorting order

3D calculation: calculate the pose of the instance in the Camera coordinate system and generate the Pick Point

Preprocessing: preprocess the 3D point cloud before calculating the Pick Point (calculate its own vertical pose, remove outliers from the point cloud)

Image matching pose estimation: calculate the pose of the instance in the Camera coordinate system (coarse matching, fine matching) and generate the Pick Point

Empty ROI judgment: determine whether there are still workpieces (point clouds) inside ROI 3D

Pick Point processing: filter, adjust, and sort Pick Points

Pick Point filtering: filter Pick Points

Pick Point adjustment: adjust Pick Points

Pick Point sorting: sort Pick Points

1. 2D Recognition

1.1 Preprocessing

The preprocessing in 2D recognition processes the 2D image before instance segmentation

1.1.1 Bilateral Filtering

Function

Image smoothing based on bilateral filtering

Parameter description

Parameter	Description	Default value	Value range
Maximum depth difference	Maximum depth difference for bilateral filtering	0.03	[0.01, 1]
Filter kernel size	Bilateral filtering kernel size	7	[1, 3000]

1.1.2 Convert Depth to Normal Map

Function

Compute pixel Normals from the depth map and convert the image to a Normal map

1.1.3 Image Enhancement

Function

Common image enhancement, such as color saturation, contrast, brightness, and sharpness

Parameter description

Parameter	Description	Default value	Value range
Image enhancement type	Enhance a specific element of the image	Contrast	Color saturation, contrast, brightness, sharpness
Image enhancement threshold	Degree of enhancement applied to a specific image element	1.5	[0.1, 100]

1.1.4 Histogram Equalization

Function Improve image contrast
Parameter description

Parameter	Description	Default value	Value range
Local mode	Local or global histogram equalization. Selected means local histogram equalization; cleared means global histogram equalization	Selected	/
Contrast threshold	Contrast threshold	3	[1,1000]

1.1.5 Filter Depth Map by Color

Function

Filter the depth map based on color values

Parameter description

Parameter	Description	Default value	Value range
Fill kernel size	Color fill size	3	[1,99]
Filter depth by HSV - maximum color range value	Maximum color value	[180,255,255]	[[0,0,0],[255,255,255]]
Filter depth by HSV - minimum color range value	Minimum color value	[0,0,0]	[[0,0,0],[255,255,255]]
Keep regions within the color range	Selected means keep regions within the color range; cleared means keep regions outside the color range	/	/

1.1.6 Gamma Image Correction

Function

Gamma correction changes image brightness

Parameter description

Parameter	Description	Default value	Value range
Gamma compensation factor	When this value is less than 1, the image becomes darker; when it is greater than 1, the image becomes brighter	1	[0.1,100]
Gamma correction factor	When this value is less than 1, the image becomes darker, which is suitable for overly bright images; when it is greater than 1, the image becomes brighter, which is suitable for overly dark images	2.2	[0.1,100]

1.1.7 Fill Holes in the Depth Map

Function

Fill hollow regions in the depth map and smooth the filled depth map

Usage scenario

Due to occlusion from the workpiece structure itself, uneven lighting, and other issues, parts of the workpiece may be missing in the depth map

Parameter description

Parameter	Description	Default value	Value range
Fill kernel size	Hole filling size	3	[1,99]

The fill kernel size can only be an odd number

Tuning

Adjust according to the detection results. If filling is excessive, decrease the parameter; if filling is insufficient, increase the parameter

Example

1.1.8 Edge Enhancement

Function

Set the texture edge regions in the image to the Background color or to a color that differs greatly from the Background color, so as to highlight the edge information of the workpiece

Usage scenario

Workpieces occlude or overlap each other, making the edges unclear

Parameter description

Parameter	Description	Default value	Parameter range	Tuning recommendation
Normal Z-direction filtering threshold	Filtering threshold for the angle between the Normal of each point in the depth map and the positive Z-axis of the Camera coordinate system. If the angle between a point's Normal and the positive Z-axis of the Camera coordinate system is greater than this threshold, the color at the corresponding position of that point in the 2D image will be set to the Background color or to a color that differs greatly from the Background color	30	[0,180]	For flat workpiece surfaces, this threshold can be stricter; for curved surfaces such as sacks, appropriately increase it according to the surface inclination of the workpiece
Background color	RGB color threshold of the Background color	128	[0,255]
Automatically adjust contrast background	Selected After enabling automatic contrast background adjustment, the colors of points in the 2D image whose angle is greater than the filtering threshold are set to a color that differs greatly from the Background color Cleared After disabling automatic contrast background adjustment, the colors of points in the 2D image whose angle is greater than the filtering threshold are set to the color corresponding to the Background color	Cleared	/

Example

In a scene with a pile of sacks, the sacks occlude one another. Enable Edge Enhancement to distinguish the edge of each sack, as shown below.

1.1.9 Extract Highest Layer Texture

Function

Extract the texture of the Highest Layer or lowest layer workpiece, and set other regions to the Background color or to a color that differs greatly from the Background color.

Usage scenario

Suitable for single-carton depalletizing scenarios. Poor lighting, similar color textures, tight stacking, interleaved stacking, or occlusion may make it difficult for the model to distinguish the texture difference between the upper and lower layers of cartons, leading to false detections.

Parameter description

Parameter	Description	Default value	Parameter range	Unit	Tuning recommendation
Distance threshold (mm)	If the distance between a point and the Highest Layer plane (lowest layer plane) is lower than this threshold, the point is regarded as being within the Highest Layer plane (lowest layer plane) and should be retained; otherwise, it is regarded as a point in the lower layer (upper layer), and the color of the lower-layer (upper-layer) point is set to the Background color or to a color that differs greatly from the Background color	50	[0.1, 1000]	mm	Usually set to 1/2 of the carton height
Number of clustering point clouds	Expected number of points participating in clustering, that is, the number of point cloud samples in the ROI 3D region	10000	[1，10000000]	/	The larger the Number of clustering point clouds, the slower the model Inference speed but the higher the accuracy; the smaller the Number of clustering point clouds, the faster the model Inference speed but the lower the accuracy
Minimum number of points per class	Minimum number of points used to filter classes	1000	[1, 10000000]	/	/
Automatically calculate contrast background	Selected After enabling automatic contrast background calculation, regions outside the Highest Layer (lowest layer) in the 2D image are set to a color that differs greatly from the Background color threshold Cleared After disabling automatic contrast background calculation, regions outside the Highest Layer (lowest layer) in the 2D image are set to the color corresponding to the Background color threshold	Selected	/	/	/
Background color threshold	RGB color threshold of the Background color	128	[0, 255]	/	/

Example

1.1.10 Remove image background outside ROI 3D

Function

Remove the background in the 2D image outside the ROI 3D region

Usage scenario

Too much image background noise affects the detection results

Parameter description

Parameter name	Description	Default value	Value range
Fill kernel size	Hole filling size	5	[1,99]
Number of iterations	Number of image Dilation iterations	1	[1,99]
Automatically calculate contrast background	Selected After enabling automatic contrast background calculation, regions outside roi in the 2D image are set to a color that differs greatly from the Background color threshold Cleared After disabling automatic contrast background calculation, regions outside roi in the 2D image are set to the color corresponding to the Background color threshold	Selected
Background color threshold	RGB color threshold of the Background color	128	[0,255]

The fill kernel size can only be an odd number

Tuning

If you need to remove more background noise from the image, decrease Fill kernel size

Example

1.2 Template-based Instance Segmentation

1.2.1 Image size

Function

Adjust the segmentation image size

Usage scenario

Use when Inference GPU memory usage or time consumption is high

Parameter description

Image size used for segmentation, which affects the Inference results

Tuning

It can be appropriately reduced; increasing is not recommended. Reducing it can lower Inference GPU memory usage and time consumption to some extent. The value must be an integer multiple of 32

1.2.2 Segmentation standard

Function

Switch the standard mode

Usage scenario

Can be used when there are false detections of incorrect instances (unexpected workpieces)

Parameter description

Value range: 1 or 2

1: use semantic similarity

2: use semantic and appearance similarity

Tuning

When there are many false detections, adjust to 2. If there are no false detections and you want to improve speed, switch to 1

1.2.3 Instance screening threshold

Function

Filter segmented instances

Usage scenario

Use when there are missed detections or long Inference time

Parameter description

Adjust tolerance for segmentation results

Tuning

Lower this parameter when there are missed detections; increase it when there are no missed detections but processing takes too long

1.2.4 Confidence threshold

Function

Filter correct instances

Usage scenario

Use when there are many false detections or missed detections

Parameter description

Remove instances below this threshold

Tuning

Increase this parameter when there are many false detections; decrease it when there are missed detections

1.2.5 Image scaling factor

Function

Scale the image

Tuning

Modification is currently not recommended

1.2.6 Bounding box type

Function

Switch the returned bounding box type

Parameter description

"aabb"：generate an aabb box based on the Mask；“obb”：generate an obb box based on the Mask

Tuning

Select as needed

1.2.7 Instance segmentation type

Function

Switch the returned instance mode

Usage scenario

Return all instances or only the instance with the largest area as needed

Parameter description

"Return all instances"：return all detected instances that satisfy the score threshold (n instances)；“Return the instance with the largest area”：return the detected instance with the largest area that satisfies the score threshold (1 instance)

Tuning

Select as needed

1.3 Instance Filtering

1.3.1 Filter based on bounding box area

Function description

Filter based on the pixel area of detected instance bounding boxes.

Usage scenario

Applicable to scenarios where instance bounding box areas differ significantly. By setting upper and lower limits for the bounding box area, image noise can be filtered out to improve image recognition accuracy and prevent noise from increasing subsequent processing time.

Parameter description

Parameter	Description	Default value	Parameter range	Unit
Minimum area (pixels)	This parameter sets the minimum filtered area of the bounding box. Instances whose bounding box area is lower than this value will be filtered out	1	[1, 10000000]	pixels
Maximum area (pixels)	This parameter sets the maximum filtered area of the bounding box. Instances whose bounding box area is higher than this value will be filtered out	10000000	[2, 10000000]	pixels

Example

Run with the default values, and you can view the bounding box area of each instance in the log, as shown below.

Based on the bounding box area of each instance, adjust Minimum area and Maximum area. For example, set Minimum area to 20000 and Maximum area to 30000 to filter out instances whose pixel area is less than 20000 or greater than 30000. You can view the instance filtering process in the log.

1.3.2 Filter based on bounding box aspect ratio

Function description

Instances whose bounding box aspect ratio is outside the specified range will be filtered out

Usage scenario

Applicable to scenarios where instance bounding box aspect ratios differ significantly

Parameter description

Parameter	Description	Default value	Parameter range
Minimum aspect ratio	Minimum bounding box aspect ratio. Instances whose bounding box aspect ratio is lower than this value will be filtered out	0	[0, 10000000]
Maximum aspect ratio	Maximum bounding box aspect ratio. Instances whose bounding box aspect ratio is higher than this value will be filtered out	10000000	[0, 10000000]
Use X/Y-axis side lengths as the aspect ratio	By default, not selected. The ratio of the longer side to the shorter side of the bounding box is used as the aspect ratio, which is suitable when the long and short sides of the bounding box differ greatly in length； After selection, the ratio of the side length of the bounding box on the X-axis to the side length on the Y-axis in the pixel coordinate system is used as the aspect ratio, which is suitable when the long-side/short-side ratio of most normal instance bounding boxes is similar, but the ratio of X-axis length to Y-axis length differs greatly for some abnormally recognized instance bounding boxes.	Cleared	/

1.3.3 Filter instances based on class ID

Function description

Filter based on instance class

Usage scenario

Applicable to scenarios where incoming materials include multiple types of workpieces

Parameter description

Parameter	Description	Default value
Retained class IDs	Retain instances whose class IDs are in the list; instances whose class IDs are not in the list will be filtered out	[0]

Example

1.3.4 Filter based on edge lengths of the instance point cloud

Function description

Filter based on the long and short edges of the instance point cloud

Usage scenario

Applicable to scenarios where the distances of the instance point cloud along the x-axis or y-axis differ greatly. By setting the distance range of the instance point cloud, image noise can be filtered out to improve image recognition accuracy and prevent noise from increasing subsequent processing time.

Parameter description

Parameter	Description	Default value	Parameter range	Unit
Short edge length range (mm)	Edge length range of the short side of the point cloud	[0, 10000]	[0, 10000]	mm
Long edge length range (mm)	Edge length range of the long side of the point cloud	[0, 10000]	[0, 10000]	mm
Lower limit of edge denoising (%)	Extract the lower percentage limit of X/Y values in the instance point cloud (Camera coordinate system) and remove point clouds outside the upper and lower limits to avoid noise affecting length calculation	5	[0, 100]	/
Upper limit of edge denoising (%)	Extract the upper percentage limit of X/Y values in the instance point cloud (Camera coordinate system) and remove point clouds outside the upper and lower limits to avoid noise affecting length calculation	95	[0, 100]	/
Edge length type	Filter by the long edge and short edge of the instance point cloud. Instances whose long or short edge length is outside the range will be filtered out	Instance point cloud short edge	Instance point cloud short edge；instance point cloud long edge；both long and short edges of the instance point cloud	/

Example

1.3.5 Filter by classifier-based class ID

Function description

Use classifier-based class ID to filter instances. Instances outside the reference classes will be filtered out.

Usage scenario

In multi-class workpiece scenarios, the visual model may detect multiple types of workpieces, but the actual task may only require one specific class of workpiece. In this case, this function can be used to filter out unnecessary workpieces

Parameter description

The default value is [0], which means instances with class ID 0 are retained by default. Instances whose class ID is not in the list will be filtered out.

1.3.6 Filter based on three-channel color

Function description

Instances can be filtered out using three-channel color thresholds (HSV or RGB).

Usage scenario

There is a clear color difference between incorrect instances and correct instances.

Parameter description

Parameter	Description	Default value	Value range
Maximum color range value	Maximum color value	[180,255,255]	[[0,0,0],[255,255,255]]
Minimum color range value	Minimum color value	[0,0,0]	[[0,0,0],[255,255,255]]
Filtering percentage threshold	Color pass-rate threshold	0.05	[0,1]
Reverse filtering	Selected means remove instances whose proportion of colors outside the color range is lower than the threshold; cleared means remove instances whose proportion of colors within the color range in the instance image is lower than the threshold	Cleared	/
Color mode	Color space selected in color filtering	HSV color space	RGB color spaceHSV color space

Example

1.3.7 Filter based on Confidence

Function description

Filter based on the instance Confidence score

Usage scenario

Applicable to scenarios where instance Confidence differs significantly

Parameter description

Parameter	Description	Default value	Parameter range
Reference Confidence threshold	Retain instances with Confidence greater than the threshold and filter out instances with Confidence less than the threshold.	0.5	[0,1]
Invert filtering results	After inversion, retain instances with Confidence less than the threshold and filter out instances with Confidence greater than the threshold.	Cleared	/

Example

1.3.8 Filter based on point cloud count

Function description

Filter based on the number of downsampled instance point clouds

Usage scenario

The instance point cloud contains a large amount of noise

Parameter description

Parameter	Description	Default value	Parameter range
Minimum point cloud count	Minimum point cloud count	3500	[1, 10000000]
Maximum point cloud count	Maximum point cloud count	8500	[2, 10000000]
Filter instances whose count is within the interval	Selected means filter out instances whose point cloud count is within the interval between the minimum and maximum values; cleared means filter out instances whose point cloud count is outside the interval	Cleared	/

1.3.9 Filter based on Mask area

Function description

Filter image masks according to the sum of mask pixels (that is, pixel area) of detected instances.

Usage scenario

Applicable to scenarios where instance Mask areas differ significantly. By setting upper and lower limits on the Mask area, noise in image masks can be filtered out, improving image recognition accuracy and preventing noise from increasing subsequent processing time.

Parameter setting description

Parameter name	Description	Default value	Parameter range	Unit
Reference minimum area	This parameter sets the minimum filtered area of the Mask. Instances whose Mask area is lower than this value will be filtered out	1	[1, 10000000]	pixels
Reference maximum area	This parameter sets the maximum filtered area of the Mask. Instances whose Mask area is higher than this value will be filtered out	10000000	[2, 10000000]	pixels

Example

1.3.10 Filter based on visibility

Function description

Filter based on the visibility score of the instance

Usage scenario

Applicable to scenarios where instance visibility differs significantly

Parameter description

Parameter	Description	Default value	Parameter range
Reference visibility threshold	Retain instances whose visibility is greater than the threshold and filter out instances whose visibility is lower than the threshold. Visibility is used to determine how visible an instance is in the image. The more the workpiece is occluded, the lower the visibility.	0.5	[0,1]
Invert filtering results	After inversion, retain instances whose visibility is less than the threshold and filter out instances whose visibility is greater than the threshold.	Cleared	/

1.3.11 Filter instances with overlapping bounding boxes

Function description

Filter instances whose bounding boxes intersect and overlap

Usage scenario

Applicable to scenarios where instance bounding boxes cross each other

Parameter description

Parameter	Description	Default value	Parameter range
Bounding box overlap ratio threshold	Threshold for the ratio of the intersecting area of bounding boxes to the area of the instance bounding box	0.05	[0, 1]
Filter the instance with the larger bounding box area	Selected means filter the larger-area instance among two instances whose bounding boxes intersect; cleared means filter the smaller-area instance among two instances whose bounding boxes intersect	Selected	/

Example

Added Filter enclosed instances. Run with the default values and view bounding box intersection information in the log. After instance filtering, 2 instances remain

According to the log, 12 instances were filtered out because their bounding boxes intersected, leaving 2 instances whose bounding boxes do not intersect

Set Bounding box overlap ratio threshold to 0.1 and select Whether to filter the larger instance. View the instance filtering process in the log: 9 instances were filtered out because the ratio of the intersecting area of bounding boxes to the instance bounding box area was greater than 0.1, 3 instances were retained because the ratio was less than 0.1, and 2 instances had no bounding box intersections.

Set Bounding box overlap ratio threshold to 0.1 and clear Whether to filter the larger instance. View the instance filtering process in the log: for 9 instances, the ratio of the intersecting area of bounding boxes to the instance bounding box area was greater than 0.1, but 2 of these instances were retained because their bounding box area was smaller than that of the intersecting instance, so 7 instances were filtered out; 3 instances were retained because the ratio was less than 0.1, and 2 instances had no bounding box intersections.

1.3.12 【Master】Filter instances with concave-convex masks based on the Mask/outer polygon area ratio

Function description

Calculate the area ratio of the Mask to the polygon enclosing the Mask. If it is lower than the set threshold, the instance will be filtered out

Usage scenario

Applicable when the workpiece Mask has jagged/concave-convex edges.

Parameter description

Parameter	Description	Default value	Value range
Area ratio threshold	Threshold of the Mask/convex hull area ratio. If it is lower than the set threshold, the instance will be filtered out.	0.1	[0,1]

1.3.13 【Master】Filter based on average point cloud distance

Function description

Filter based on the average value of the distance from points in the point cloud to the fitted plane, removing uneven instance point clouds

Usage scenario

Applicable to scenarios where planar workpiece point clouds are bent

Parameter description

Parameter	Description	Default value	Parameter range	Unit
Plane segmentation distance threshold (mm)	Extract a plane from a bent instance point cloud. Points whose distance to the plane is smaller than this threshold are regarded as points on the plane	10	[-1000, 1000]	mm
Average distance threshold (mm)	Average value of the distances from points in the instance point cloud to the extracted plane	20	[-1000, 1000]	mm
Remove instances whose average distance is less than the threshold	Selected means filter out instances whose average distance from points to the extracted plane is less than the average distance threshold; cleared means filter out instances whose average distance from points to the extracted plane is greater than the average distance threshold	Cleared	/	/

1.3.14 【Master】Filter occluded instances based on the Mask/bounding box area ratio

Function description

Calculate the Mask/bounding box area ratio. Instances whose ratio is outside the minimum and maximum range will be filtered out

Usage scenario

Used to filter instances of occluded workpieces

Parameter description

Parameter	Description	Default value	Value range
Minimum area ratio	Lower limit of the Mask/bounding box area ratio range. The smaller the ratio, the higher the degree of occlusion of the instance	0.1	[0,1]
Maximum area ratio	Upper limit of the Mask/bounding box area ratio range. The closer the ratio is to 1, the lower the degree of occlusion of the instance	1.0	[0,1]

1.3.15 【Master】Determine whether all Highest Layer instances have been fully detected

Function description

One of the foolproof mechanisms. Determine whether all instances on the Highest Layer have been detected. If there are undetected Highest Layer instances, an error will be reported and the Workflow will end

Usage scenario

Applicable to scenarios where one image is used for multiple picks or where picking must be performed in sequence, to prevent missed picks in subsequent tasks caused by incomplete instance detection

Parameter description

Parameter	Description	Default value	Parameter range	Unit	Tuning
Distance threshold	Used to determine the Highest Layer workpiece. If the distance between a point and the highest point of the workpiece point cloud is smaller than the distance threshold, the point is regarded as belonging to the Highest Layer point cloud; otherwise, it is not regarded as belonging to the Highest Layer point cloud.	5	[0.1, 1000]	mm	It should be smaller than the height of the workpiece

2. 3D Calculation

This section mainly explains the functions related to Pick Point generation and provides Parameter tuning recommendations.

2.1 Preprocessing

2.1.1 Point Cloud Clustering Denoising

Function

Remove noise by using Point Cloud clustering.

Usage Scenario

There is a large amount of noise in the instance Point Cloud.

Parameter Description

Parameter Name	Description	Default Value	Range	Unit	Tuning Recommendation
Distance Threshold for Point Cloud Clustering (mm)	Determines whether Point Clouds in space belong to the same category. If the distance between Point Clouds is lower than this Threshold, they are treated as the same category.	5	[0.1, 1000]	mm	Generally does not need to be changed. It should be greater than the point spacing of the Target Object Point Cloud and smaller than the minimum distance between the Target Object Point Cloud and the noise Point Cloud.
Minimum Point Count Threshold	Point Cloud clusters with fewer than this number of points will be filtered out.	100	[1,10000000]	/	Generally does not need to be changed. Increase the Minimum Point Count Threshold according to the size of the noise in the instance Point Cloud.
Maximum Point Count Threshold	Point Cloud clusters with more than this number of points will be filtered out.	100000	[1,10000000]	/	Generally does not need to be changed. If the number of points in the Target Object Point Cloud exceeds 100000, increase the Maximum Point Count Threshold.
Select Top Point Cloud in ROI	If selected, the system calculates and sorts the average Z coordinate of Point Clouds of the same category in the ROI coordinate system, and retains the Point Cloud category with the largest average Z coordinate (the top Point Cloud). If not selected, all qualified Point Clouds are retained.	Not selected	/	/	If the Target Object Point Cloud is above the noise Point Cloud, select this option to retain the Target Object Point Cloud. If the Target Object Point Cloud is below the noise Point Cloud, you should also adjust the Z-axis of the ROI coordinate system to point downward when selecting this option, so that the Target Object Point Cloud can be retained.
Visualize Process Data	If selected, the denoised Point Cloud is saved and can be viewed in C:_data	Not selected	/	/	In debugging mode, select this option if you need to save visualization data.

2.1.2 Point Cloud Downsampling

Function

Sample the Point Cloud according to the specified point spacing to reduce the number of computed points and improve model inference speed, though accuracy may decrease.

Usage Scenario

If there are too many Point Cloud points in the actual scenario, select Point Cloud Downsampling.

Parameter Description

Parameter	Description	Default Value	Parameter Range	Unit
Point Spacing for Downsampling (mm)	Samples the Point Cloud according to the specified point spacing.	5.0	[0.1, 1000]	mm

Tuning
- The larger the value of Point Spacing for Downsampling, the fewer Point Cloud points remain after downsampling, the faster the Pick Point calculation, but the lower the accuracy may be.
- The smaller the value of Point Spacing for Downsampling, the more Point Cloud points remain after downsampling, the slower the Pick Point calculation, but the higher the accuracy.

2.1.3 Calculate Self Vertical Pose

Function

Calculate the vertical pose of a tilted sack.

Usage Scenario

Sacks are placed at an angle in the depalletizing scenario.

Parameter Description

Parameter	Description	Default Value	Parameter Range	Unit
Neighborhood Range	The neighborhood range of the center point on the sack surface, that is, the distance from the center point to the edge. This value affects the processing and analysis of the surrounding Point Cloud near the center point.	20.00	[0, 100]	/
Point Cloud Accuracy (mm)	The distance between adjacent points in the 3D Point Cloud captured by the camera.	10	[1, 1000]	mm
Use obb for Calculation	If selected, the z vector of the Point Cloud obb (oriented bounding box) is used as the sack's own vertical direction. Otherwise, the system traverses the Point Cloud within the center-point neighborhood range and selects the axis with the largest projection axis from those Point Clouds as the sack's own vertical direction.	Not selected	/	/

Example

2.1.4 Filter Point Cloud by HSV (Hue, Saturation, Value)

Function

Filter the Point Cloud based on hue, saturation, and value in the Point Cloud image to screen out Point Cloud regions that match the target range.

Parameter Description

Parameter Name	Description	Default Value	Range
Maximum Color Range Value for Depth Filtering by HSV	The maximum color value for filtering the Point Cloud.	[0.9,0.9,0.9]	[[0,0,0],[1,1,1]]
Minimum Color Range Value for Depth Filtering by HSV	The minimum color value for filtering the Point Cloud.	[0.0,0.0,0.0]	[[0,0,0],[1,1,1]]

2.1.5 Filter Point Cloud by RGB Color

Function

Filter the Point Cloud by RGB color to screen out Point Cloud regions that match the target range.

Parameter Description

Parameter Name	Description	Default Value	Range
Maximum Color Value for Point Cloud Filtering by RGB Color	The maximum color value for filtering the Point Cloud.	[0.9,0.9,0.9]	[[0,0,0],[1,1,1]]
Minimum Color Value for Depth Filtering by RGB Color	The minimum color value for filtering the Point Cloud.	[0.0,0.0,0.0]	[[0,0,0],[1,1,1]]

2.1.6 Select Point Cloud Within the ROI Region

Function

Select the Point Cloud within the ROI 3D region from the instance Point Cloud. This default function cannot be deleted.

2.1.7 Optimize Mask Based on Point Cloud

Function

Based on the Point Cloud within ROI 3D, remove Point Cloud data in the Mask that is outside ROI 3D to improve Mask accuracy.

Usage Scenario

Applicable to depalletizing scenarios where model recognition includes connected regions outside ROI 3D.

2.1.8 Remove Points Whose Normal Exceeds the Angle Threshold

Function

Remove Point Cloud points whose angle between the normal vector and the standard normal-vector axis exceeds the normal-vector angle Threshold.

Usage Scenario

Loading and unloading of planar Target Objects (materials are isolated from each other).

Parameter Description

Parameter Name	Description	Default Value	Range	Unit
Angle Threshold	Point Cloud points with an angle greater than this Threshold are considered different instances.	15	[-360, 360]
Standard Normal Vector Axis	The angle formed between the Point Cloud Normal and the standard normal-vector axis.	Z-axis	X/Y/Z-axis	/
Use ROI Coordinate System	If selected, the angle between the Normal and the axes of the ROI coordinate system is calculated. Otherwise, the angle between the Normal and the axes of the camera coordinate system is calculated.	Not selected	/	/

2.1.9 Point Cloud Plane Segmentation

Function

When fitting a plane to the Point Cloud, points whose distance to the plane is lower than the reference distance are considered in-plane points; otherwise, they are considered out-of-plane points. The fitted plane can be retained or removed.

Usage Scenario

Applicable to single-carton depalletizing scenarios, but not to single-sack depalletizing scenarios.

Parameter Description

Default Value: 3.000

Range: [0.001, 10000]

Unit: mm

Tuning

The larger the reference distance, the thicker the fitted plane; the smaller the reference distance, the thinner the fitted plane.

If Remove Plane is selected, the plane with the largest number of Point Cloud points is removed; otherwise, the plane with the largest number of Point Cloud points is retained.

Example

2.1.10 Remove Point Cloud Outliers

Function

Identify and remove outlier noise from the neighboring region (that is, the neighborhood) of each point in the Point Cloud.

Usage Scenario

The object's Point Cloud contains many outlier noise points.

Parameter Description

Parameter	Description	Default Value	Parameter Range	Tuning Recommendation
Reference Neighborhood Point Count	The number of neighboring points adjacent to each point in the Point Cloud, that is, the neighborhood size. For dense Point Clouds, even a small neighborhood is sufficient to reflect the object's features, so a smaller value can be used. For sparser Point Clouds, a larger neighborhood is needed to reflect the object's features, so a larger value should be used.	30	[1, 10000000]
Standard Deviation Multiplier	Used to identify outliers. If the deviation between a point's coordinates and the average coordinates of the object's Point Cloud exceeds the Standard Deviation Multiplier, the point is considered an outlier. The smaller the value, the more points are considered outliers and removed, but this may lead to misjudgment and removal of important object features. The larger the value, the fewer points are considered outliers and removed, but some outliers may be retained and affect image recognition accuracy.	0.005	[0.0001, 2]	If the Point Cloud becomes too sparse after Remove Point Cloud Outliers, you should increase the Standard Deviation Multiplier.

Example

2.1.11 Filter Out Point Cloud Beyond the Object Distance Limit

Function

Filter out Point Cloud data in the specified direction to remove noise points and improve the accuracy of Pick Point calculation.

Usage Scenario

Usually used in single-sack depalletizing scenarios.

Parameter Description

Parameter	Description	Default Value	Parameter Range	Unit	Tuning Recommendation
Specified Axis	The specified axis of the Point Cloud, used to filter out Point Cloud data in the specified direction.	Z-axis	X/Y/Z-axis	/	Specified Axis generally does not need to be changed.
Threshold (mm)	Along the specified axis direction, if the distance between the lower-layer Point Cloud and the Target Object Point Cloud is greater than this Threshold, the lower-layer Point Cloud will be filtered out; if the distance between the lower-layer Point Cloud and the Target Object Point Cloud is smaller than this Threshold, the lower-layer Point Cloud will be retained.	750	[0, 1000]	mm	Adjust Threshold according to the actual scenario. The larger the Threshold, the fewer Point Cloud points are filtered out; the smaller the Threshold, the more Point Cloud points are filtered out.
Select Coordinate System	Filter out Point Cloud data in the selected coordinate system.	ROI Coordinate System	Camera Coordinate System; ROI Coordinate System; Object Coordinate System	/

Example

As shown in the figure below (left), the Point Cloud contains lower-layer Point Cloud data. After selecting Filter Out Point Cloud Beyond the Object Distance Limit, the Point Cloud after filtering out the lower-layer Point Cloud is shown in the figure below (right).

As shown in the figure below, comparing the detection results with a Threshold of 100 and a Threshold of 400, the larger the Threshold, the fewer Point Cloud points are filtered out.

2.2 Image Matching Pose Estimation

2.2.1 Matching Confidence threshold (mm)

Function

Confidence score of feature point matching. The higher the score, the higher the feature point quality, but the number of matched feature points may be lower

Usage scenario

Applicable to scenarios with substantial feature detection noise

Parameter description

Feature points below this threshold will be removed

Tuning

Default value: 0.01

Value range: [0.001, 1.0]

2.2.2 Number of feature augmentation points

Function

Augment default features

Usage scenario

Applicable to scenarios where the detected scene feature points are few, or where keypoints are mainly detected at the edges

Parameter description

Number of additionally augmented points

Tuning

When matching is tilted or offset, try reducing it to 0; when keypoints are mainly detected at the edges, try increasing it to 5

2.2.3 Feature augmentation range

Function

Adjust the pixel range selected for feature augmentation

Usage scenario

Applicable to scenarios where keypoints are detected at the edges or where template and scene keypoint matching has a large offset

Parameter description

Select augmented points within a radius equal to the augmentation range around the original feature point

Tuning

Modification is currently not recommended

2.2.4 Maximum number of iterations

Function

Adjust the maximum number of iterations for matching

Tuning

Modification is currently not recommended

2.2.5 Bounding box size factor

Function

Adjust the scene instance image

Usage scenario

Applicable to scenarios where the instance segmentation Mask is smaller than the workpiece or closely fits the workpiece

Parameter description

Bounding box scale factor used when cropping scene instances

Tuning

Increase it when feature points on the outer contour may be lost

2.2.6 Enable depth features

Tuning

Adjustment is not currently supported

2.2.7 Coarse matching evaluation threshold

Function

Evaluate the matching effect

Usage scenario

Use when matching is tilted or abnormal

Parameter description

Matching point pairs smaller than this threshold are considered valid

Tuning

For tilted matching of small objects, it is recommended to decrease it; for tilted matching of large objects, it is recommended to increase it

2.2.9 Object Pose correction

Fine matching search radius(mm)

Function

During fine matching, the template point cloud is matched with the instance point cloud, and each point in the template point cloud needs to search for the nearest point in the instance point cloud. The fine matching search radius represents both the search radius in the instance point cloud and the distance threshold between each point in the template point cloud and its nearest point in the instance point cloud. If the distance between a point and its nearest point is less than the fine matching search radius, the two points are considered matchable; otherwise, they are considered unmatched.

Usage scenario

Ordered planar workpiece loading/unloading, unordered planar workpiece picking, and planar workpiece positioning assembly scenarios

Parameter description

Default value: 10

Value range：[1, 500]

Unit: mm

Tuning

Usually unchanged

Fine matching search mode

Function

Method for retrieving the nearest point in the instance point cloud for the template point cloud during fine matching

Usage scenario

If the fine matching effect between the template point cloud and the instance point cloud is poor, this function should be adjusted

Parameter description

Parameter	Description
Point to point	Each point in the template point cloud searches for its nearest point in the instance point cloud (the point with the shortest straight-line distance within the search radius)；applicable to all workpieces
Point to plane	Each point in the template point cloud searches for its nearest point in the instance point cloud along its Normal；applicable to workpieces with obvious geometric features
Combination of point to point and point to plane	First use point-to-point mode to optimize the workpiece pose in the instance point cloud, and then use point-to-plane mode to optimize the workpiece pose in the instance point cloud；applicable to workpieces with obvious geometric features Using this method increases Takt Time

Use contour mode

Function

Extract contour point clouds from the template point cloud and instance point cloud for coarse matching

Usage scenario

Ordered planar workpiece loading/unloading, unordered planar workpiece picking, and planar workpiece positioning assembly scenarios. If the coarse matching result using keypoints is poor, enable this function to perform coarse matching again using contour point clouds

Tuning

The result of coarse matching affects the result of fine matching. If the fine matching result is poor, enable Use contour mode

Contour search range(mm)

Function

Search radius for extracting contour point clouds from the template point cloud and the instance point cloud

Usage scenario

General workpiece ordered loading/unloading, general workpiece unordered picking, and general workpiece positioning assembly scenarios

Parameter description

Default value: 5

Value range：[0.1, 500]

Unit: mm

Tuning

When the value is small, the search radius for contour point clouds is small, which is suitable for extracting detailed workpiece contours, but the extracted contours may include outlier noise；

When the value is large, the search radius for contour point clouds is large, which is suitable for extracting wider workpiece contours, but some detailed features may be ignored.

Save pose estimation[fine matching] data

Function

Selected means save fine matching data

Usage scenario

Ordered planar workpiece loading/unloading, unordered planar workpiece picking, planar workpiece positioning assembly, and planar workpiece positioning assembly (matching only)

Example

The fine matching data is saved in the project save path \Project Folder\data\PickLight\Historical Data Timestamp\Builder\pose\output folder.

3. Pick Point Processing

This section mainly explains functions related to Pick Point filtering and adjustment, along with Parameter tuning recommendations.

3.1 Pick Point Adjustment

3.1.1 Rotate the Picking Pose when it is outside the angle range

Function description

When the Picking Pose is outside the configured angle range, it is rotated counterclockwise by a certain angle around a fixed axis. If it is still outside the configured angle range after rotation, a warning is issued.

Usage scenario

This function is only applicable to depalletizing scenarios. It can keep the robot's approach direction stable during picking and prevent the end effector from repeatedly rotating during the picking process. In 180° cases, it can prevent exceptions such as cable twisting.

Parameter description

Parameter	Description	Default	Range	Unit
Fixed axis	An axis of the Picking Pose. The pose is rotated counterclockwise around this fixed axis	Z-axis	X/Y/Z-axis	/
Rotation angle	The angle by which the pose is rotated counterclockwise around the fixed axis. Adjust this angle so the Picking Pose satisfies the angle range	0	[-360,360]	degree
Angle range	The angle range of the Picking Pose. Set the angle range according to factors such as material placement, end effector type, and cycle time	[0,180]	[-180,180]	degree
Use current robot Euler Angles	By default, pose calculation uses Euler Angles "XYZ". When selected, the Euler Angles configured for the current robot are used so the pose remains consistent with the robot teach pendant.	Unchecked	/	/
Custom coordinate system	The coordinate system of the Picking Pose	Robot arm coordinate system	Default coordinate system; camera coordinate system; ROI coordinate system; robot arm coordinate system	/

Example

Without using this function, the generated Pick Points are shown below.

When this function is used with the default values, the RZ angles of the Picking Poses for instances 0, 1, and 2 are all within the angle range [0,180], so no processing is performed. The RZ angle of the Picking Pose for instance 4 is -90°, which is outside the angle range [0,180], so the Picking Pose of instance 4 is rotated by 0° around the fixed Z-axis.

If you want to adjust the RZ angle of the Picking Pose for instance 4 into the angle range, you can change the rotation angle to 180 and rotate the Picking Pose of instance 4 by 180° around the fixed Z-axis.

3.1.2 Rotate the Picking Pose so the rotation axis direction matches the target axis direction

Function description

Rotate the Picking Pose once around the fixed axis so that the direction of the rotation axis (determined by the right-hand rule) matches the positive or negative direction of the target axis in the target coordinate system.

Usage scenario

Avoid collisions between the robot end effector and the bin.

Parameter description

Parameter	Description	Default	Range
Rotation axis	An axis of the Picking Pose. Determined by the right-hand rule, the Picking Pose is rotated counterclockwise once around the fixed axis so that the direction of the rotation axis matches the positive or negative direction of the target axis in the target coordinate system	X-axis	X/Y/Z-axis
Fixed axis	The Picking Pose is rotated counterclockwise once around the fixed axis so that the direction of the rotation axis matches the positive or negative direction of the target axis in the target coordinate system	Z-axis	X/Y/Z-axis
Target axis	An axis of the target coordinate system. The Picking Pose is rotated counterclockwise once around the fixed axis so that the direction of the rotation axis matches the positive or negative direction of the target axis in the target coordinate system	X-axis	X/Y/Z-axis
Negative target axis direction	If selected, the direction of the rotation axis is aligned with the negative direction of the target axis in the target coordinate system; otherwise, it is aligned with the positive direction of the target axis in the target coordinate system	Unchecked	/
Custom coordinate system	The coordinate system of the Picking Pose	Default coordinate system	Default coordinate system; camera coordinate system; ROI coordinate system; robot arm coordinate system

Example

3.1.3 Rotate the Picking Pose so the angle between the rotation axis and the target axis is minimized

Function description

Rotate the Picking Pose around the fixed axis by 0, 90, 180, and 270 degrees respectively, calculate the angle between the rotated rotation axis and the positive or negative direction of the target axis in the camera coordinate system, and finally output the Picking Pose with the smallest angle after rotation.

Usage scenario

Avoid collisions between the robot end effector and the bin.

Parameter description

Parameter	Description	Default	Range
Fixed axis	An axis of the Picking Pose. Rotate the pose counterclockwise around this fixed axis	Z-axis	X/Y/Z-axis
Rotation axis	An axis of the Picking Pose. When rotating the pose, calculate the angle between this rotation axis and the positive or negative direction of the target axis	X-axis	X/Y/Z-axis
Target axis	An axis of the camera coordinate system. When rotating the pose, calculate the angle between the rotation axis and the positive or negative direction of this target axis	X-axis	X/Y/Z-axis
Negative target axis direction	If selected, calculate the angle between the rotation axis and the negative direction of the target axis; otherwise, calculate the angle between the rotation axis and the positive direction of the target axis	Selected	/
Custom coordinate system	The coordinate system of the Picking Pose	Default coordinate system	Default coordinate system; camera coordinate system; ROI coordinate system; robot arm coordinate system

Example

3.1.4 Flip the Picking Pose so the angle between the rotation axis and the target axis is minimized

Function description

Rotate the Picking Pose once around the fixed axis so that the angle formed between the rotation axis and the positive or negative direction of the target axis in the ROI coordinate system is acute.

Usage scenario

Avoid collisions between the robot end effector and the bin.

Parameter description

Parameter	Description	Default	Range
Fixed axis	An axis of the Picking Pose. Rotate the Picking Pose counterclockwise around this fixed axis	Z-axis	X/Y/Z-axis
Rotation axis	An axis of the Picking Pose. Rotate the Picking Pose so that the direction of this rotation axis matches the positive or negative direction of the target axis	X-axis	X/Y/Z-axis
Target axis	An axis in the ROI coordinate system. Rotate the Picking Pose so that the direction of the rotation axis matches the positive or negative direction of this target axis	X-axis	X/Y/Z-axis
Negative target axis direction	If selected, rotate the Picking Pose so that the direction of the rotation axis matches the negative direction of the target axis; otherwise, rotate the Picking Pose so that the direction of the rotation axis matches the positive direction of the target axis	Selected	/
Custom coordinate system	The coordinate system of the Picking Pose	Default coordinate system	Default coordinate system; camera coordinate system; ROI coordinate system; robot arm coordinate system

Example

3.1.5 Point a Picking Pose axis toward the ROI center

Function

Rotate the Picking Pose around a fixed axis so that the pointing axis of the Picking Pose points to the ROI center.

Usage scenario

Avoid collisions between the robot end effector and the bin.

Parameter description

Parameter	Description	Default	Range
Pointing axis	The axis in the Picking Pose that needs to be adjusted	X-axis	X/Y/Z-axis
Fixed axis	The axis that remains unchanged during rotation	Z-axis	X/Y/Z-axis
Reverse align	If selected, reverse-align the pointing axis to the ROI center; otherwise, align the pointing axis to the ROI center	Selected	/
Strict pointing	If selected, force the Picking Pose to rotate so the pointing axis points to the ROI center	Unchecked	/
Custom coordinate system	The coordinate system of the Picking Pose	Default coordinate system	Default coordinate system; camera coordinate system; ROI coordinate system; robot arm coordinate system

Example

3.1.6 Rotate the Picking Pose so the Z-axis direction matches the Z-axis of the target coordinate system

Function description

Rotate the Picking Pose so that its Z-axis direction matches the Z-axis of the target coordinate system.

Usage scenario

Usually this is used by default only in depalletizing scenarios and cannot be deleted. It is used to make the Z-axis of the Picking Pose perpendicular to the Z-axis of the ROI coordinate system (4-axis) or consistent with the direction of the Target Object surface (6-axis).

Parameter description

Parameter	Description	Default	Range
Robot configuration	Set according to the on-site robot configuration. You can choose 4-axis or 6-axis. If a 6-axis robot is actually used as a 4-axis robot, it should be set to 4-axis	4-axis	4-axis/6-axis
Use ROI Z-axis as target direction	When the robot configuration is set to 4-axis, if selected, the pose is rotated around the X-axis so that the Z-axis direction of the rotated pose matches the positive direction of the ROI Z-axis ; if not selected, the pose is rotated around the X-axis so that the Z-axis direction of the rotated pose matches the positive direction of the Z-axis of the camera coordinate system . When the robot configuration is set to 6-axis, regardless of whether it is selected, the pose is rotated around the X-axis so that the Z-axis direction of the rotated pose matches the positive direction of the Z-axis of the object's own coordinate system	Unchecked	/
Custom coordinate system	The coordinate system of the Picking Pose	Camera coordinate system	Default coordinate system; camera coordinate system; ROI coordinate system; robot arm coordinate system

Example

3.1.7 Rotate the Picking Pose around a fixed axis

Function description

Rotate the Picking Pose by a certain angle around a fixed axis.

Usage scenario

Avoid collisions between the robot end effector and the bin.

Parameter description

Parameter	Description	Default	Range	Unit
Rotation angle	The angle by which the pose is rotated counterclockwise around the fixed axis	90	[-360, 360]	degree°
Fixed axis	An axis of the Picking Pose. Rotate the pose counterclockwise around this fixed axis	Z-axis	X/Y/Z-axis	/
Custom coordinate system	The coordinate system of the Picking Pose	Default coordinate system	Default coordinate system; camera coordinate system; ROI coordinate system; robot arm coordinate system	/

Example

3.1.8 Translate the Picking Pose

Function description

Move the Picking Pose by a certain distance along the translation axis.

Usage scenario

Avoid collisions between the robot end effector and the bin.

Parameter description

Parameter	Description	Default	Range	Unit
Translation amount (mm)	The distance the Picking Pose moves along the translation axis. A positive translation amount means translating in the positive direction of the translation axis, and a negative translation amount means translating in the negative direction of the translation axis	0	[-1000, 1000]	mm
Translation axis	The direction in which the Picking Pose moves	X-axis	X/Y/Z-axis	/
Custom coordinate system	The coordinate system of the Picking Pose	Robot arm coordinate system	Default coordinate system; camera coordinate system; ROI coordinate system; robot arm coordinate system	/

Example

3.1.9 Pick Point Teaching

Function description

Record the Pick Point coordinates generated by the software and the Pick Point coordinates taught under the current operating condition, then output the transformed Picking Pose based on the offset between the two.

Usage scenario

When the Pick Points generated by the vision system have an obvious systematic offset and the robot TCP coordinate accuracy is limited or difficult to calibrate, this method can be used to directly map the same offset pattern to subsequent Pick Points, thereby avoiding robot TCP calibration.

Parameter description

Parameter	Description	Default	Range
Vision Pose	Pick coordinates of the detection result
X(mm)	X coordinate of the Vision Pose	0.00	±10000000, meaning no limit.
Y(mm)	Y coordinate of the Vision Pose	0.00	±10000000, meaning no limit.
Z(mm)	Z coordinate of the Vision Pose	0.00	±10000000, meaning no limit.
RX(°)	X-axis rotation amount of the Vision Pose	0.00	±180
RY(°)	Y-axis rotation amount of the Vision Pose	0.00	±180
RZ(°)	Z-axis rotation amount of the Vision Pose	0.00	±180
Picking Pose	Manually taught Pick Point
X(mm)	X coordinate of the Picking Pose	0.00	±10000000, meaning no limit.
Y(mm)	Y coordinate of the Picking Pose	0.00	±10000000, meaning no limit.
Z(mm)	Z coordinate of the Picking Pose	0.00	±10000000, meaning no limit.
RX(°)	X-axis rotation amount of the Picking Pose	0.00	±180
RY(°)	Y-axis rotation amount of the Picking Pose	0.00	±180
RZ(°)	Z-axis rotation amount of the Picking Pose	0.00	±180

3.1.10 Refine Object Pose based on plane Normal

Function description

Correct the Object Pose by fitting the plane Normal so that the Z-axis direction of the Object Pose remains consistent with the direction of the plane Normal of the Target Object.

Usage scenario

When the Target Object contains a plane and there is a tilt deviation in the plane when the template Point Cloud is matched with the actual Point Cloud, use this function to fine-tune the Target Object plane and improve picking accuracy.

Not applicable to depalletizing scenarios

Parameter description

Parameter	Description	Default	Range	Unit
Distance Threshold	Distance Threshold for fitting a plane from the Point Cloud	10	[-1000, 1000]	mm
Save visualization data	If selected, the visualization data will be saved under the historical data timestamp	Selected	/	/
Custom coordinate system	The coordinate system of the Picking Pose	Camera coordinate system	Default coordinate system; camera coordinate system; ROI coordinate system; robot arm coordinate system	/

Example

3.1.11 Sort Pick Points by inter-axis angle

Function

Sort Pick Points according to the angle between an axis of the Picking Pose and the target axis of the ROI.

Parameter description

Parameter	Description	Default	Range
Axis selection	An axis of the Picking Pose	Z-axis	X/Y/Z-axis
Target axis selection	An axis of the ROI coordinate system	Z-axis	X/Y/Z-axis
Select reverse direction	If selected, calculate the angle with the negative direction of the target axis; otherwise, calculate the angle with the positive direction of the target axis	Unchecked	/
Select descending order	If selected, sort Pick Points from small to large by angle; otherwise, sort Pick Points from large to small by angle	Unchecked	/

3.1.12 [Advanced] Rotate the Picking Pose and automatically compensate for excessive angles to the specified axis

Function description

Determine whether the angle formed between the specified axis of the Picking Pose and the target axis is within the specified range. If not, adjust the Picking Pose into the specified range.

Usage scenario

Avoid collisions between the robot end effector and the bin.

Parameter description

Parameter	Description	Default	Range	Unit
Angle range	Adjust the Picking Pose into the angle range	30	[0, 180]	degree°
Specified axis	An axis of the Picking Pose. Adjust this axis so that it falls within the angle range relative to the target axis of the ROI coordinate system	Z-axis	X/Y/Z-axis	/
Target axis	An axis of the ROI coordinate system. Compare the angle range with the specified axis of the Picking Pose	Z-axis	X/Y/Z-axis	/
Compare with the negative half-axis of the ROI	If not selected, compare the angle range with the positive direction of the target axis of the ROI coordinate system; if selected, compare the angle range with the negative direction of the target axis of the ROI coordinate system	Unchecked	/	/
Custom coordinate system	The coordinate system of the Picking Pose	Default coordinate system	Default coordinate system; camera coordinate system; ROI coordinate system; robot arm coordinate system	/

3.1.13 [Advanced] Symmetry center Object Pose optimization

Function

Search for the symmetry center of the Target Object based on the instance Mask, then combine it with the plane of the instance or the pose of the ROI 3D center point to calculate the optimal Picking Pose.

Before using this function, first make sure the instance Mask is symmetrical

Usage scenario

Applicable when the instance Mask of a symmetrical Target Object is also symmetrical, but the Picking Pose is not near the expected center; at the same time, the Target Object has a plane that can be used as a reference, for example, there is a plane on the top of the object, or ROI 3D can be used as a reference for the projected pose.

Applicable project scenarios include brake discs (general circles), refractory bricks (depalletizing), symmetrical irregular parts, fuel fillers, and so on.

Parameter description

Parameter	Description	Default	Range	Tuning recommendation
Target Object Symmetry type	Target Object Symmetry type of the instance Mask	Rotational symmetry	Rotational symmetry: after the Target Object rotates by a certain angle around the center point, its shape completely overlaps with the original position; mirror symmetry: the Target Object uses a certain axis / plane as the mirror, and the left-right or upper-lower sides are completely symmetrical.	Circles and rectangles are both rotationally symmetrical and mirror-symmetrical, so rotational symmetry is preferred; for trapezoids and other shapes that are symmetrical only along a certain axis or plane, choose mirror symmetry.
Gaussian blur level	Tolerance for determining whether the actual Point Cloud overlaps after rotation	3	\[1,99\]	The larger the value, the stronger the blur, the more smoothing is applied, and the more lenient the overlap judgment after rotation; The smaller the value, the weaker the blur, the less smoothing is applied, and the stricter the overlap judgment. An odd number must be entered
Rotation angle setting	When the symmetry mode is rotational symmetry, it indicates the rotation angle interval, that is, the angle difference between two adjacent rotations. When the symmetry mode is mirror symmetry, it indicates the rotation range, that is, the angle interval within which the Point Cloud can rotate around the symmetry axis.	180	\[1,360\]	For rotational symmetry, set it based on the symmetry of the Target Object. A circle is symmetrical at any angle, so 60 degrees is recommended; A square has 90-degree symmetry, so it can be set to 90 degrees or 180 degrees; A rectangle has 180-degree symmetry, so set it to 180. For mirror symmetry, adjust it within 5° to ensure fine tuning near the symmetry axis
Image scaling ratio	Adjusts the size of the Point Cloud image. The larger this ratio, the smaller the Point Cloud image size and the lower the GPU memory usage, but image detail loss increases, resulting in reduced calculation accuracy .	2	\[1,10000000\]
Search range	Based on the initially determined center of the Target Object, this defines the range expanded outward to search for Point Cloud features. The actual range is (search range2image scaling ratio)	10	\[1,10000000\]	For example, for a square Target Object, the initially determined center position of the Target Object is point O. If the search range is set to 10 and the image scaling ratio is 1, then the actual search range is a square region centered at point O with a side length of 10×2×1=20. Point Cloud features are searched within this region to further determine the symmetry center of the Target Object and the optimal Picking Pose. As another example, for a circular Target Object, if the search range is set to 8 and the image scaling ratio is 2, then the actual search range is a circular region centered at the initially determined center of the Target Object with a diameter of 8×2×2=32. Point Cloud features are searched within this region to further determine the Object Pose of the Target Object and the optimal Picking Pose.
Use ROI3D as the reference projection plane	If selected, ROI3D is used as the reference projection plane	Unchecked	/	Select this when the Point Cloud has no obvious plane and the projection plane is difficult to determine; leave it unchecked when the Point Cloud has a clear plane.
Save symmetry center process data	If selected, the debug data generated during the symmetry center process is saved. You can view it in the `\ProjectName`\data`\PickLight`\HistoricalDataTimestamp`\find`\_symmetry_center folder	Unchecked	/	Select this when you need to inspect the detailed process images
Symmetry axis prior type	Effective in ``{=html}mirror symmetry``{=html} mode. Specifies the known Target Object Symmetry type and fixes the asymmetric orientation	Automatic search	Automatic searchSymmetric along the long axisSymmetric along the short axis	If the symmetry axis of the Target Object is the long axis, choose "Symmetric along the long axis". If the symmetry axis of the Target Object is the short axis, choose "Symmetric along the short axis". If uncertain, choose "Automatic search"
Pose adjustment type	Whether to inherit pose-related information from the input pose	Default pose	Default poseInherit rotationInherit translation	/
Symmetry score Threshold	Symmetry results with a symmetry score lower than this Threshold are abnormal results. When set to 0, no filtering is performed	0.0	\[0.0, 1.0\]	/

Example

3.2 Pick Point Filtering

3.2.1 Filter by fine matching score

Function description

Filter Pick Points based on the pose fine matching score.

Parameter description

Parameter	Description	Default	Range
Score Threshold	Retain Pick Points whose fine matching score is greater than this Threshold	0.5	[0, 1]

Example

3.2.2 Filter Pick Points of occluded Target Objects

Function description

Determine whether there are too many occluding object Point Clouds in the target detection area along the specified ROI axis or the Picking Pose axis at the Pick Point of the grasped Target Object. If so, the Target Object is considered occluded and the Pick Point is filtered out.

Usage scenario

Applicable to depalletizing and ordered scenarios in which Target Objects are picked layer by layer, but the model recognizes lower-layer Target Objects. When picking a lower-layer Target Object, the gripper may collide with the upper-layer Target Object.

Parameter description

Parameter	Description	Default	Range	Unit
Cuboid length in X direction	Set the cuboid length in the X direction of the Picking Pose	1500	[1, 10000]	mm
Cuboid length in Y direction	Set the cuboid length in the Y direction of the Picking Pose	1500	[1, 10000]	mm
Cuboid length in Z direction	Set the cuboid length in the Z direction of the Picking Pose	800	[1, 10000]	mm
Distance Threshold between detection area and Pick Point origin	Along the ROI axis, the nearby cuboid surface area farther than this distance Threshold from the Pick Point origin is regarded as the target detection area	50	[1, 1000]	mm
Point Cloud count Threshold in detection area	If the number of occluding object Point Clouds in the target detection area exceeds this Threshold, the Pick Point is considered occluded	1000	[0, 100000]	/
Specified axis direction	Based on the pose reference specified axis direction, set the specific location of the target detection area within the cuboid space (for example, near the front/back/left/right/top/bottom surface of the cuboid)	[0,0,-1]	[1,0,0]: positive X-axis[-1,0,0]: negative X-axis[0,1,0]: positive Y-axis[0,-1,0]: negative Y-axis[0,0,1]: positive Z-axis[0,0,-1]: negative Z-axis	/
Use ROI 3D pose reference	If selected, adjust the collision detection area according to the ROI 3D pose reference	Unchecked	/	/
Save visualization data	If selected, the visualization data is stored according to the saved data path to help observe whether the generated cuboid is reasonable; if not selected, it is not saved	Unchecked	/	/

Example

3.2.3 Filter by Picking Pose angle range

Function description

Determine whether the angle of the Picking Pose is within the constrained angle range, and filter out all Pick Points that do not meet the condition.

Usage scenario

Prevent collisions caused by abnormal robot arm Picking Pose angles.

Parameter description

Parameter	Description	Default	Range	Unit
Angle filtering Threshold	Calculate the maximum angle between the specified axis of the ROI and the specified axis of the Picking Pose. Pick Points whose angle is greater than the current Threshold will be filtered out	30	[-360, 360]	degree°
Invert ROI specified axis direction	If selected, use the negative direction of the specified ROI axis for angle calculation; otherwise, use the positive direction of the specified ROI axis for angle calculation	Selected	/	/
Specified Picking Pose axis	Specify an axis of the Picking Pose for angle calculation	Z-axis	X/Y/Z-axis	/
Specified ROI axis	Specify an axis of the ROI coordinate system for angle calculation	Z-axis	X/Y/Z-axis	/

Example

3.2.4 Filter Pick Points outside the ROI 3D type region

Function description

Determine whether the Pick Point is within the ROI 3D range, and remove Pick Points that are outside the ROI 3D area.

Usage scenario

Prevent picking outside the ROI area, which may cause collisions between the robot arm and the target object.

Parameter description

Parameter	Description	Default
ROI3D type region	Usually "workspace"; "pick area" is a smaller ROI region than "workspace", which can restrict Pick Points to an ROI region smaller than the "workspace" to avoid some collision cases.	Workspace

Example

As shown in the figure below, when the ROI3D area and ROI2D are area a, the corresponding Pick Point is in the upper-right corner.

When the ROI3D area and ROI2D are changed to area b, the original Pick Point is outside the ROI area, so that Pick Point is removed and a new Pick Point is generated within area b.

3.2.5 [New] Filter Pick Points where the Target Object collides with the gripper (including the original function)

[New] Filter Pick Points where the Target Object collides with the gripper

Function description

Collision detection between the gripper and the Point Cloud near the Pick Point. If the number of Point Clouds in contact with the gripper exceeds the pick collision Threshold, the Pick Point of the Target Object is considered to have a collision risk.

Usage scenario

Used when collision detection is required between the gripper and the Point Cloud near the Target Object being picked.

Parameter description

Parameter	Description	Default	Range
Collision Threshold	Collision distance Threshold. If the distance between the scene and the gripper surface is smaller than this Threshold, it is considered a collision. The larger the Threshold, the stricter it is. Unit: mm	7	1-1000
Collision Point Cloud sampling	Sampling size for collision Point Clouds. The larger the value, the faster the cycle time; the smaller the value, the slower the cycle time. Effective only in "Target Object scene Point Cloud only" and "bin + Target Object scene Point Cloud" modes. Unit: mm	5	1 - 1000
Save visualization data for gripper collision detection	Save visualization data for collision detection between the gripper and the picked Target Object	Unchecked	Selected/Unchecked

Filter Pick Points where the Target Object collides with the gripper

Function description

Usage scenario

Used when collision detection is required between the gripper and the Point Cloud near the Target Object being picked.

Parameter description

Parameter	Description	Default	Range
Pick collision Threshold	The maximum number of Point Clouds the gripper may contain near the Pick Point. For example, 20 means that if the number of scene Point Clouds contained by the gripper exceeds 20, it is considered a collision	20	0-10000
Collision Point Cloud sampling (m)	Downsampling size of the Point Cloud in the collision area. The larger the value, the faster the detection speed, but the lower the accuracy. Applicable scenario: scenarios requiring high cycle rates	0.002	0.0001 - 0.5000
Save visualization data for gripper collision detection	Save visualization data for collision detection between the gripper and the picked Target Object	Unchecked	Selected/Unchecked
Import gripper model	Select and import the gripper model used for collision detection from a folder	/	/

**The gripper should be simplified to fewer than 500 faces**

3.2.6 [Advanced] Retain the one Pick Point with the largest/smallest pose value among instance Pick Points and filter the remaining Pick Points

Function description

Convert the pose to the specified coordinate system, sort poses according to the value of the specified sorting axis, and retain the pose with the maximum or minimum value. This is suitable for cylindrical Target Objects when keeping the top or bottom Pick Point.

Parameter description

Parameter	Description	Default	Range
Specified coordinate system	Select which coordinate system the pose should be converted to for processing	ROI coordinate system	ROI coordinate system/camera coordinate system
Specified sorting axis	Select which axis value of the pose to sort by	Z-axis	X/Y/Z-axis
Take minimum value	If selected, retain the pose with the minimum value on the sorting axis; otherwise, retain the pose with the maximum value on the sorting axis	Unchecked	/

Example

3.2.7 [Advanced] Filter Pick Points close to the previous N Pick Points

Function description

If the variation between the current Pick Point and any Pick Point in the cache is within the Threshold range, the Pick Point will be filtered out.

Parameter description

Parameter	Description	Default	Range	Unit
Upper limit of Pick Point change (+)
X(mm)	Upper limit of X coordinate	2	[0, 10000000]	mm
Y(mm)	Upper limit of Y coordinate	2	[0, 10000000]	mm
Z(mm)	Upper limit of Z coordinate	2	[0, 10000000]	mm
RX(°)	Upper limit of RX rotation amount	1	[0, 180]	degree°
RY(°)	Lower limit of RY rotation amount	1	[0, 180]	degree°
RZ(°)	Lower limit of RZ rotation amount	1	[0, 180]	degree°
Lower limit of Pick Point change (-)
X(mm)	Lower limit of X coordinate	2	[0, 10000000]	mm
Y(mm)	Lower limit of Y coordinate	2	[0, 10000000]	mm
Z(mm)	Lower limit of Z coordinate	2	[0, 10000000]	mm
RX(°)	Lower limit of RX rotation amount	1	[0, 180]	degree°
RY(°)	Lower limit of RY rotation amount	1	[0, 180]	degree°
RZ(°)	Lower limit of RZ rotation amount	1	[0, 180]	degree°
Pick Point cache count	Number of Pick Points cached. After the current Pick Point comparison is completed, it will be added to the cache in real time	5	[1, 100]	/

3.2.8 [Advanced] Filter Object Poses close to the previous N Object Poses

Function description

If the variation between the current Object Pose and any Object Pose in the cache is within the Threshold range, the Object Pose will be filtered out. When an Object Pose is determined to be similar, all Pick Points on that Target Object will be filtered out.

Parameter description

Parameter	Description	Default	Range	Unit
Upper limit of Object Pose change (+)
X(mm)	Upper limit of X coordinate	2	[0, 10000000]	mm
Y(mm)	Upper limit of Y coordinate	2	[0, 10000000]	mm
Z(mm)	Upper limit of Z coordinate	2	[0, 10000000]	mm
RX(°)	Upper limit of RX rotation amount	1	[0, 180]	degree°
RY(°)	Lower limit of RY rotation amount	1	[0, 180]	degree°
RZ(°)	Lower limit of RZ rotation amount	1	[0, 180]	degree°
Lower limit of Object Pose change (-)
X(mm)	Lower limit of X coordinate	2	[0, 10000000]	mm
Y(mm)	Lower limit of Y coordinate	2	[0, 10000000]	mm
Z(mm)	Lower limit of Z coordinate	2	[0, 10000000]	mm
RX(°)	Lower limit of RX rotation amount	1	[0, 180]	degree°
RY(°)	Lower limit of RY rotation amount	1	[0, 180]	degree°
RZ(°)	Lower limit of RZ rotation amount	1	[0, 180]	degree°
Object Pose cache count	Number of vision Object Poses cached. After the comparison of the current Object Pose is completed, it will be added to the cache in real time	5	[1, 100]	/

3.2.9 [Advanced] Filter Pick Points outside the upper and lower limits of Pick coordinates

Function description

Retain other Pick Points within the specified range of a reference Pick Point and filter out abnormal Pick Points.

Usage scenario

Prevent incorrect robot picking and ensure picking accuracy.

This function is not applicable to depalletizing scenarios

Parameter description

Parameter	Description	Default	Unit
Reference Pick coordinates
X(mm)	X coordinate of the reference Pick Point	0	mm
Y(mm)	Y coordinate of the reference Pick Point	0	mm
Z(mm)	Z coordinate of the reference Pick Point	0	mm
RX(°)	RX rotation amount of the reference Pick Point	0	degree
RY(°)	RY rotation amount of the reference Pick Point	0	degree
RZ(°)	RZ rotation amount of the reference Pick Point	0	degree
Upper limit of Pick coordinates (+)
X(mm)	Upper limit of the X coordinate. For example, if the X coordinate of the reference Pick Point is 100 and the upper limit is set to 10, the allowed range is: [100-lower limit, 110]	10000000, meaning no limit.	mm
Y(mm)	Upper limit of the Y coordinate. For example, if the Y coordinate of the reference Pick Point is 100 and the upper limit is set to 10, the allowed range is: [100-lower limit, 110]	10000000	mm
Z(mm)	Upper limit of the Z coordinate. For example, if the Z coordinate of the reference Pick Point is 100 and the upper limit is set to 10, the allowed range is: [100-lower limit, 110]	10000000	mm
RX(°)	Upper limit of the RX rotation amount. For example, if the RX rotation amount of the reference Pick Point is 180 and the upper limit is set to 10, the allowed range is (default angle wraparound applies): [[-180, -170], [180-lower limit, 180]]	180, meaning no limit.	degree°
RY(°)	Upper limit of the RY rotation amount. For example, if the RY rotation amount of the reference Pick Point is 180 and the upper limit is set to 10, the allowed range is (default angle wraparound applies): [[-180, -170], [180-lower limit, 180]]	180	degree°
RZ(°)	Upper limit of the RZ rotation amount. For example, if the RZ rotation amount of the reference Pick Point is 180 and the upper limit is set to 10, the allowed range is (default angle wraparound applies): [[-180, -170], [180-lower limit, 180]]	180	degree°
Lower limit of Pick coordinates (-)
X(mm)	Lower limit of the X coordinate. For example, if the X coordinate of the reference Pick Point is 100 and the lower limit is set to 10, the allowed range is: [100-lower limit value, 110]	10000000	mm
Y(mm)	Lower limit of the Y coordinate. For example, if the Y coordinate of the reference Pick Point is 100 and the lower limit is set to 10, the allowed range is: [100-lower limit, 110]	10000000	mm
Z(mm)	Lower limit of the Z coordinate. For example, if the Z coordinate of the reference Pick Point is 100 and the lower limit is set to 10, the allowed range is: [100-lower limit, 110]	10000000	mm
RX(°)	Lower limit of the RX rotation amount. For example, if the RX rotation amount of the reference Pick Point is 180 and the lower limit is set to 10, the allowed range is (default angle wraparound applies): [[-180, -180+upper limit], [170, 180]]	180, meaning no limit.	degree°
RY(°)	Lower limit of the RY rotation amount. For example, if the RY rotation amount of the reference Pick Point is 180 and the lower limit is set to 10, the allowed range is (default angle wraparound applies): [[-180, -180+upper limit], [170, 180]]	180	degree°
RZ(°)	Lower limit of the RZ rotation amount. For example, if the RZ rotation amount of the reference Pick Point is 180 and the lower limit is set to 10, the allowed range is (default angle wraparound applies): [[-180, -180+upper limit], [170, 180]]	180	degree°

3.3 Pick Point Sorting

3.3.1 Reference coordinate system

Function description

Set a unified coordinate system for all instances to group and sort instances.

Usage scenario

Common to depalletizing scenarios, random picking scenarios, and ordered loading/unloading scenarios

Strategies related to coordinates should first set the reference coordinate system

Parameter description

Parameter	Description	Illustration
Camera coordinate system	The coordinate system origin is above the object, and the positive Z-axis direction points downward; the XYZ values are the values of the center point of the object in this coordinate system
ROI coordinate system	The coordinate system origin is approximately at the center of the pallet stack, and the positive Z-axis direction points upward; the XYZ values are the values of the center point of the object in this coordinate system
Robot arm coordinate system	The coordinate system origin is on the robot arm itself, and the positive Z-axis direction generally points upward; the XYZ values are the values of the center point of the object in this coordinate system
Pixel coordinate system	The coordinate system origin is at the top-left vertex of the RGB image and is a 2D planar coordinate system; the X and Y values are the x value of the bbox detection box and the y value of the bbox detection box, and Z is 0

3.3.2 General picking strategy

Parameter description

Parameter	Description
Strategy	Select which value is used for grouping and sorting and how to sort it, including Pick Point center X/Y/Z coordinate values from large to small/from small to large (mm), from the middle to the sides / from the sides to the middle along the Pick Point XY coordinate axis (mm). Multiple items can be superimposed and executed in order.
Grouping step size	According to the selected strategy, divide Pick Points into several groups based on the step size. The grouping step size is the interval between two groups of Pick Points
Number of leading groups to keep	After grouping and sorting, how many groups of instances need to be retained

Strategy name*	Description	Grouping step size		Number of leading groups to keep
Strategy name*	Description	Default	Range	Default
Pick Point center X/Y/Z coordinate values from large to small / from small to large (mm)	Use the X/Y/Z coordinate values of the Pick Point center for grouping and sorting	200.000	[0, 10000000]	10000
From the middle to the sides / from the sides to the middle along the Pick Point XY coordinate axis (mm)	Use the X/Y coordinate values of the Pick Point center and perform grouping and sorting in the direction of "middle to sides" or "sides to middle"	200.000	[0, 10000000]	10000

3.3.3 Carton combination strategy

To solve the problems of low efficiency and limited applicable scenarios in traditional single-pick depalletizing, PickWiz adds a carton combination strategy in depalletizing scenarios to support picking multiple Target Objects in a single operation. It supports the core scenarios of "cartons with consistent dimensions" and "rectangular suction cups", covering more real project scenarios.

3.3.3.1 Multi-pick runtime configuration

（1）In sack single depalletizing or carton single depalletizing scenarios, enable Vision computation configuration - Vision computation acceleration.

（2）Under the Pick Point sorting module, select the carton combination strategy;

（3）Strategy selection: available options are the default combination strategy or combination along a specified carton pose axis. These are two methods for finding the largest number of cartons that can be combined.

Default combination strategy: Find the largest number of cartons that can be combined along the X-axis and Y-axis directions of a carton Picking Pose.
Combine along a certain carton pose axis: Find the largest number of cartons that can be combined along the X-axis or Y-axis direction of a carton Picking Pose. This is suitable for scenarios where cartons are arranged in a straight line. When using this strategy, you need to choose the carton combination direction, namely the Picking Pose X-axis or the Picking Pose Y-axis.

Note:
The combination direction is related to the positive and negative axis directions. Cartons can be combined only when they are on the same axis and in the same direction, and after combination, the orientation of the whole stack of cartons remains consistent with the orientation of a single carton before combination. Therefore, before combining cartons, make sure all cartons to be combined are placed in the same orientation, and unify the coordinates of the cartons to be combined to the same axis direction.

（4）Combination conditions: determine which cartons can be combined and how many can be combined at most.

Maximum cartons per row: the maximum number of cartons that can be combined in one row, default is 2.
Maximum number of combination rows: the maximum number of carton rows that can be combined, default is 1.
Maximum spacing (mm): cartons to be combined cannot be too far apart in the "combination direction", In the combination direction (axis direction), when the spacing between two adjacent cartons or cartons in different rows is less than this value, they can be combined into one group. The default is 10.
- Example: when searching for the maximum number of cartons along the Picking Pose X-axis, if the spacing between two adjacent cartons in the Picking Pose X-axis direction is 8 mm (≤10), they can be combined; if the spacing is 12 mm (>10), they cannot be combined.
Maximum misalignment distance (mm): cartons to be combined cannot be too far apart in the direction "perpendicular to the combination direction" . In the direction perpendicular to the combination direction (axis direction), when the misalignment distance between two adjacent cartons or cartons in different rows is less than this value, they can be combined into one group. The default is 10.
- Example: when searching for the maximum number of cartons along the Picking Pose X-axis, if two adjacent cartons are offset in the Picking Pose Y-axis direction by 8 mm (≤10), they can be combined; if they are offset by 15 mm (>10), they are no longer aligned and cannot be picked together.
Maximum angular deviation (°): cartons to be combined should face almost the same direction. In the combination direction (axis direction), when the rotational deviation angle of the cartons is less than this value, they can be combined into one group. The default is 10.
- Example: if a carton is rotated by 5° relative to the combination direction, as long as it does not exceed 10°, it can be combined; if it is rotated by 15° (>10), the orientation differs too much, the robot will be skewed when picking, and it cannot be combined.

3.3.3.2 Robot configuration

（1）On the robot configuration page, add new placeholders in Vision computation communication message - Robot to PickWiz commands - Vision detection send command: maximum cartons per row and maximum number of combination rows, as shown below;

（2）In Vision computation communication message - PickWiz to robot commands - Pick-related information - Returned information when picking Target Objects, add Object Dimensions length, Object Dimensions width, and Target Object orientation.

After the robot configuration is completed, click the Run button.

3.3.3.3 View multi-pick runtime results

（1）In the 3D Matching window, hover the mouse over an instance to view the combined picking information of a single instance after carton combination, including 2D recognition results, Picking Pose, and instance combination information.

In the visualization window, click the Settings button in the upper right corner to set how the combined instance information is displayed.

Right-click an instance to view the combined picking information and Target Object information of the single instance.

（2）In the 2D recognition window, you can use the relevant combination buttons in the menu bar to view the combined ID, combined Mask, and combined bounding box.

How to Use Shadow Mode

Visual Parameter Adjustment Guide for Ordered Loading/Unloading and Positioning Assembly of Planar Workpieces (Image Matching)