Target Object Operation Guide
The Target Object is the object that the Robot needs to pick. Different types of Target Objects require different configuration content. Please select the corresponding configuration process according to the Target Object type.
1. Sack
1.1 Add a Target Object
Click + Add Target Object in the Target Object section of Run Configuration to enter the Target Object interface.
1.2 Configure the Target Object
（1）Target Object Information
| Target Object ID | The Target Object ID is usually used by the Robot to switch Target Objects during product changeover, and can be modified |
| Target Object Name | The default Target Object name is the Target Object type Sack & Target Object ID, and can be modified |
| Camera Configuration | Optional Camera configuration, not required |
| Vision Model | The Deep Learning model is used to segment instances from the scene. The default vision model for sacks is the generic sack model. If the generic sack model does not segment instances well, Mask Mode or Shadow Mode can be applied to optimize the vision model. |
（2）Model Training
| Mask Mode | Optional. If the generic sack model does not segment instances well, Mask Mode can be applied to optimize the segmentation effect of the vision model. Mask Mode requires new textures to be entered and synthetic data to be used for model training. For details, please refer to How to Use Mask Mode |
| Shadow Mode | Optional. If the generic sack model does not segment instances well, Shadow Mode can be applied to optimize the segmentation effect of the vision model. Shadow Mode requires real runtime data to train the model and optimize the segmentation effect of the vision model. For details, please refer to How to Use Target Object Shadow Mode |
1.3 Select the Target Object
After configuring the sack, click Return to return to the main interface, and select the corresponding sack in task Information - Target Object.
1.4 Example
2. Carton
2.1 Add a Target Object
Click + Add Target Object in the Target Object section of Run Configuration to enter the Target Object interface.
2.2 Configure the Target Object
（1）Target Object Information
| Target Object ID | The Target Object ID is usually used by the Robot to switch Target Objects during product changeover, and can be modified |
| Target Object Name | The default Target Object name is Carton & Target Object ID, and can be modified |
| Camera Configuration | Optional Camera configuration, not required |
| Vision Model | The Deep Learning model is used to segment instances from the scene. The default vision model for cartons is the generic carton model. If the generic carton model does not segment instances well, Mask Mode or Shadow Mode can be applied to optimize the vision model. |
（2）Target Object Configuration
| Length, Width, and Height | Configure the length, width, and height of the carton to prevent segmentation errors in the vision model. This is optional |
| Texture | Import the texture image of the carton and control the picking direction through the texture. This function has been disabled and does not need to be filled in |
| Pick Point | Set the Pick Point position for the Robot to pick the carton. Options are “Center of Carton”, “Upper-left Corner of Carton”, “Lower-left Corner of Carton”, and “Upper-right Corner of Carton” |
| Select Carton Body Vertex | If selected, the vertex of the carton itself is used as the coordinate origin, and the left side of the long edge is the upper-left; if not selected, the carton vertex in the pixel coordinate system is used as the coordinate origin, and the corresponding corner point is selected. |
Examples of carton Pick Point settings are shown below.
（3）Model Training
| Mask Mode | Optional. If the generic carton model does not segment instances well, Mask Mode can be applied to optimize the segmentation effect of the vision model. Mask Mode requires new textures to be entered and synthetic data to be used for model training. For details, please refer to How to Use Mask Mode |
| Shadow Mode | Optional. If the generic carton model does not segment instances well, Shadow Mode can be applied to optimize the segmentation effect of the vision model. Shadow Mode requires real runtime data to train the model and optimize the segmentation effect of the vision model. For details, please refer to How to Use Target Object Shadow Mode |
2.3 Select the Target Object
After configuring the carton, click Return to return to the main interface, and select the corresponding carton in task Information - Target Object.
2.4 Example
3. Generic Target Object
3.1 Add a Target Object
Click + Add Target Object in the Target Object section of Run Configuration to enter the Target Object interface.
3.2 Configure the Target Object
（1）Target Object Information
| Target Object ID | The Target Object ID is usually used by the Robot to switch Target Objects during product changeover, and can be modified |
|---|---|
| Target Object Name | The default Target Object name for a generic Target Object is “Cad&Target Object ID”, and can be modified |
| Target Object Type | The default Target Object type for a generic Target Object is standard Target Object and cannot be modified |
| Point Cloud File | Required. Upload the Point Cloud Template of the generic Target Object for coarse matching with the actual Point Cloud collected by the Camera. You can refer to Point Cloud Template Creation Guide to create the Point Cloud Template. |
| Fine Matching Point Cloud Template | Optional. Upload the full Point Cloud Template of the generic Target Object for fine matching with the actual Point Cloud collected by the Camera to optimize the pose of the Target Object. You can refer to Point Cloud Template Creation Guide to create the fine matching Point Cloud Template. |
| Custom Keypoints | Required. Generic Target Objects should enable custom keypoints and upload the keypoints of the generic Target Object for matching with the keypoints predicted by the vision model on the actual Point Cloud. The keypoint file name is generally keypoint_128.ply. You can refer to Point Cloud Template Creation Guide to create the keypoint file. |
| Camera Configuration | Optional Camera configuration, not required |
Generic Target Objects should enable Custom Keypoints and upload the keypoint file of the generic Target Object. The keypoint file name is generally keypoint_128.ply
（2）Model Information
| Vision Model | Required. A Deep Learning model used to segment instances from the scene. The vision model of a generic Target Object uses the CAD-based synthetic data training（one-click training）method. The vision models of different generic Target Objects are also different, so the vision model of a generic Target Object needs one-click training. For the specific process, please refer to One-click Training for Vision Models/ One-click Training for Imaging Models |
| Mesh File | Required. Upload the CAD model of the generic Target Object. The mesh file should be a standardized CAD model to support subsequent operations such as one-click training for the vision model, generating a Point Cloud Template, and aligning Point Cloud to mesh. Please refer to Point Cloud Template Creation Guide to standardize the mesh file. |
| Incoming Material Form | Only for ordered loading and unloading scenarios. It is used to simulate and enter the incoming material form of the Target Object in the scene, so as to support the use of one-click training models in ordered scenarios. Tight fit: helps quickly set the number of Target Objects in rows/columns, and is suitable for scenarios where the incoming Target Objects are orderly, the poses are consistent, and the spacing between Target Objects is small. Custom incoming material form: allows full configuration of the detailed parameters of the incoming material form and is suitable for all incoming material types. |
| Target Object Attributes | Optional. Used for one-click training models. Options include "Long-strip", "Symmetric", "Highly Reflective", and "lowsolidity" |
| Operating Environment | Optional. Enter the operating environment for one-click training models. During data generation in one-click training, the background that is randomly transformed by default will be replaced with the entered operating environment data, thereby improving the recognition effect. |
| Target Object Texture | Optional. Enter the Target Object texture for one-click training models. During model training in one-click training, the uploaded texture will be used for data augmentation, thereby improving the recognition effect. |
| Maximum Recognition Count of the Model | Used for one-click training to control the maximum number of instances that the vision model can recognize during Inference. Under the same conditions, the larger the recognition count, the more recognition results the model can return, but the longer the model Inference time. In ordered scenarios, it is generally recommended to fill in a value slightly greater than the number of materials in one layer; in random scenarios, it needs to be set according to the actual scene. |
In generic Target Object scenarios, when importing Point Cloud files/keypoint files, the alignment of the coordinate center with the mesh file is automatically checked.
（3）Pick Points
When adding a Target Object configuration, there are no Pick Points by default and they need to be added manually. After adding new Pick Points, adjust the pose of the Pick Points based on the absolute coordinate system or the Pick Point coordinate system.
| Absolute Coordinate System | Pick Point of the Target Object CAD Model |
|---|---|
| Pick Point Coordinate System（Offset） | Adjust the Pick Point with the center of the Target Object as the origin |
3.3 Select the Target Object
After configuring the generic Target Object, click Return to return to the main interface, and select the corresponding generic Target Object in task Information - Target Object.
3.4 Example
4. Surface Target Object
4.1 Add a Target Object
Click + Add Target Object in the Target Object section of Run Configuration to enter the Target Object interface.
4.2 Configure the Target Object
（1）Target Object Information
| Target Object ID | The Target Object ID is usually used by the Robot to switch Target Objects during product changeover, and can be modified |
|---|---|
| Target Object Name | The default Target Object name for a surface Target Object is “CAD without keypoints&Target Object ID”, and can be modified |
| Target Object Type | The default Target Object type for a surface Target Object is surface Target Object and cannot be modified |
| Point Cloud File | Required. Upload the Point Cloud Template of the surface Target Object for coarse matching with the actual Point Cloud collected by the Camera. You can refer to Point Cloud Template Creation Guide to create the Point Cloud Template. |
| Fine Matching Point Cloud Template | Optional. Upload the full Point Cloud Template of the surface Target Object for fine matching with the actual Point Cloud collected by the Camera to optimize the pose of the Target Object. You can refer to Point Cloud Template Creation Guide to create the fine matching Point Cloud Template. |
| Custom Keypoints | Optional. Surface Target Objects do not have keypoints, so Custom Keypoints do not need to be enabled |
| Camera Configuration | Optional Camera configuration, not required |
（2）Model Information
| Vision Model | Required. A Deep Learning model used to segment instances from the scene. The vision model of a surface Target Object uses the CAD-based synthetic data training（one-click training）method. The vision models of different surface Target Objects are also different, so the vision model of a surface Target Object needs one-click training. For the specific process, please refer to One-click Training for Vision Models/ One-click Training for Imaging Models. For surface Target Object loading and unloading（materials isolated from each other）scenarios, the vision model does not need to be uploaded |
| Mesh File | Required. Upload the CAD model of the surface Target Object. The mesh file should be a standardized CAD model to support subsequent operations such as one-click training for the vision model, generating a Point Cloud Template, and aligning Point Cloud to mesh. Please refer to Point Cloud Template Creation Guide`` to standardize the mesh file. |
| Incoming Material Form | Only for ordered loading and unloading scenarios. It is used to simulate and enter the incoming material form of the Target Object in the scene, so as to support the use of one-click training models in ordered scenarios. Tight fit: helps quickly set the number of Target Objects in rows/columns, and is suitable for scenarios where the incoming Target Objects are orderly, the poses are consistent, and the spacing between Target Objects is small. Custom incoming material form: allows full configuration of the detailed parameters of the incoming material form and is suitable for all incoming material types. |
| Target Object Attributes | Optional. Used for one-click training models. Options include "Long-strip", "Symmetric", "Highly Reflective", and "lowsolidity" |
| Operating Environment | Optional. Enter the operating environment for one-click training models. During data generation in one-click training, the background that is randomly transformed by default will be replaced with the entered operating environment data, thereby improving the recognition effect. |
| Target Object Texture | Optional. Enter the Target Object texture for one-click training models. During model training in one-click training, the uploaded texture will be used for data augmentation, thereby improving the recognition effect. |
| Maximum Recognition Count of the Model | Used for one-click training to control the maximum number of instances that the vision model can recognize during Inference. Under the same conditions, the larger the recognition count, the more recognition results the model can return, but the longer the model Inference time. In ordered scenarios, it is generally recommended to fill in a value slightly greater than the number of materials in one layer; in random scenarios, it needs to be set according to the actual scene. |
（3）Pick Points
When adding a Target Object configuration, there are no Pick Points by default and they need to be added manually. After adding new Pick Points, adjust the pose of the Pick Points based on the absolute coordinate system or the Pick Point coordinate system.
| Absolute Coordinate System | Pick Point of the Target Object CAD Model |
|---|---|
| Pick Point Coordinate System（Offset） | Adjust the Pick Point with the center of the Target Object as the origin |
4.3 Select the Target Object
4.4 Example
5. Circular Surface
5.1 Add a Target Object
Click + Add Target Object in the Target Object section of Run Configuration to enter the Target Object interface.
5.2 Configure the Target Object
（1）Target Object Information
| Target Object ID | The Target Object ID is usually used by the Robot to switch Target Objects during product changeover, and can be modified |
|---|---|
| Target Object Name | The default Target Object name for a Circular Surface Target Object is “Circular Surface&Target Object ID”, and can be modified |
| Target Object Type | The default Target Object type for a Circular Surface Target Object is Circular Surface and cannot be modified |
| Vision Model | The Deep Learning model is used to segment instances from the scene. The default vision model for Circular Surface is the generic Circular Surface model. If the generic Circular Surface model does not segment instances well, Shadow Mode can be applied to optimize the vision model. |
| Point Cloud File | Optional. When on-site precision requirements are high, the Point Cloud edge condition is poor, or the circle fitting error is large, you can select Vision Parameters - Object Pose Correction and upload a Point Cloud Template for pose correction of the Circular Surface. You can select the file in historical data \Builder\pose\output\inverse-pcd as the Point Cloud Template. |
| Camera Configuration | Optional Camera configuration, not required |
（2）Target Object Configuration
| Circle Radius（mm） | Optional. When the Pick Point of the Circular Surface deviates from the circle center, configure the radius of the Circular Surface in advance as prior data for Circular Surface fitting. |
|---|
（3）Pick Points
| Pick Point | The Pick Point of the Circular Surface is at the circle center by default |
|---|
（4）Model Training
| Shadow Mode | Optional. If the generic Circular Surface model does not segment instances well, Shadow Mode can be applied to optimize the segmentation effect of the vision model. Shadow Mode requires real runtime data to train the model and optimize the segmentation effect of the vision model. For details, please refer to How to Use Target Object Shadow Mode |
|---|
5.3 Select the Target Object
5.4 Example
6. Cylinder
6.1 Add a Target Object
6.2 Configure the Target Object
（1）Target Object Information
| Target Object ID | The Target Object ID is usually used by the Robot to switch Target Objects during product changeover, and can be modified |
|---|---|
| Target Object Name | The default Target Object name for a Cylinder Target Object is “Cylinder&Target Object ID”, and can be modified |
| Target Object Type | The default Target Object type for a Cylinder Target Object is Cylinder and cannot be modified |
| Vision Model | The Deep Learning model is used to segment instances from the scene. There are two vision models for cylinders: the generic short-cylinder model and the generic long-cylinder model. If the generic cylinder model does not segment instances well, Shadow Mode can be applied to optimize the vision model. |
| Point Cloud File | Optional. When on-site precision requirements are high, the Point Cloud edge condition is poor, or the cylinder fitting error is large, you can select Vision Parameters - Object Pose Correction and upload a Point Cloud Template for pose correction of the cylinder. You can select the file in historical data \Builder\pose\output\inverse-pcd as the Point Cloud Template. |
| Camera Configuration | Optional Camera configuration, not required |
- Vision Model
Click the Edit button to modify the vision model
Right-click the Model Path text box to choose Clear File, and then select a new path; you can also choose Open Containing Folder to open the folder where the vision model is located.
Click the Import Model button to upload the vision model.
- Point Cloud File
Right-click the text box of Point Cloud File to choose Clear File, and then upload a new file; you can also choose Open Containing Folder to open the folder where the Point Cloud Template is located, as shown below
（2）Target Object Configuration
| Circle Radius（mm） | Optional. When the cylinder cannot be fitted or the cylinder fitting error is large, configure the cylinder circular-surface radius in advance as prior data for cylinder fitting. |
| Height（mm） | Optional. When the cylinder cannot be fitted or the cylinder fitting error is large, configure the cylinder height in advance as prior data for cylinder fitting. |
（3）Pick Points
| Pick Point | Estimate the cylinder center at the middle position, or start from the endpoint and estimate the cylinder center according to the prior dimensions |
| Rotational Pick Interval（°） | Within the picking range, generate one Pick Point at each interval. The default is 45°. |
- Pick Points
Reference Case Description
For the figure below, if the cylinder center is estimated at the middle position by default, the Pick Point may be generated at the wrong position.
Therefore, you need to choose to start from the endpoint and estimate the cylinder center according to the prior dimensions, which can reasonably avoid the above situation.
Note
During Robot communication, this configuration takes effect when the prior dimensions of the Target Object are passed in or when the prior dimension values set in the software are not 0; otherwise, when the value is 0, the system estimates the cylinder center at the middle position by default.
- Rotational Pick Interval（°）
Within the picking range of the cylinder, a Pick Point is generated at every certain degree. The default is 180°
（4）Model Training
| Shadow Mode | Optional. If the generic cylinder model does not segment instances well, Shadow Mode can be applied to optimize the segmentation effect of the vision model. Shadow Mode requires real runtime data to train the model and optimize the segmentation effect of the vision model. For details, please refer to How to Use Target Object Shadow Mode |
6.3 Select the Target Object
6.4 Example
7. Quadrilateral
7.1 Add a Target Object
7.2 Configure the Target Object
（1）Target Object Information
| Target Object ID | The Target Object ID is usually used by the Robot to switch Target Objects during product changeover, and can be modified |
|---|---|
| Target Object Name | The default Target Object name is Quadrilateral&Target Object ID |
| Target Object Type | Default quadrilateral |
| Target Object Geometric Shape | For a Quadrilateral Target Object, choose rectangular or non-rectangular. Switch the Target Object shape to switch the vision algorithm so that it better adapts to Quadrilateral Target Objects of different shapes |
| Vision Model | Used to segment instances from the scene. Quadrilateral Target Objects use the generic quadrilateral model to segment instances. If the generic quadrilateral model does not segment instances well, Shadow Mode can be applied to optimize the vision model. |
| Point Cloud File | Optional. When on-site precision requirements are high, the Point Cloud edge condition is poor, or the quadrilateral fitting error is large, you can select Vision Parameters - Object Pose Correction and upload a Point Cloud Template for pose correction of the quadrilateral. You can select the file in historical data \Builder\pose\output\inverse-pcd as the Point Cloud Template. |
| Camera Configuration | Optional Camera configuration; the imaging may differ |
（2）Target Object Configuration
| Length（mm） | Optional. When the Pick Point of the quadrilateral deviates from the center, configure the length and width of the quadrilateral in advance as prior data for quadrilateral fitting, which helps achieve finer quadrilateral fitting. |
| Width（mm） | Optional. When the Pick Point of the quadrilateral deviates from the center, configure the length and width of the quadrilateral in advance as prior data for quadrilateral fitting, which helps achieve finer quadrilateral fitting. |
（3）Pick Points
| Pick Point | The Pick Point of a Quadrilateral Target Object is set to the center of the quadrilateral by default |
（4）Model Training
| Shadow Mode | Optional. If the generic quadrilateral model does not segment instances well, Shadow Mode can be applied to optimize the segmentation effect of the vision model. Shadow Mode requires real runtime data to train the model and optimize the segmentation effect of the vision model. For details, please refer to How to Use Target Object Shadow Mode |