Scenario Description and Pre-configuration Checks
About 1346 wordsAbout 4 min
2. Scenario Description and Pre-configuration Checks
2.1 Scenario Description
Using neatly arranged cylinders as an example, this case explains how to configure a “Loading and unloading of neatly arranged general type target objects” Task to recognize targets, calculate their poses, and guide a robot to pick them.
Setting up the Task requires a CAD model, Visual Model, Point Cloud template, and keypoint file that match the actual cylinders. The Visual Model identifies target instances. General pose estimation calculates an initial pose through keypoint Coarse Matching and can further refine it through Point Cloud Fine Matching.
This case focuses on checking that the template, keypoints, and CAD coordinate systems are consistent, as well as the position and direction of the Pick Point relative to the target. Follow the corresponding guides for common operations such as camera connection, robot communication, and Eye-hand calibration.
For the principles behind the solution, see Introduction to 3D Vision-Guided Solutions. For the pose calculation method, see the General Pose Estimation Configuration Guide.
Task Type
This case uses “Neatly Arranged Loading and Unloading > Loading and unloading of neatly arranged general type target objects.” “Cylinder” names the target used in this case; it does not mean that “Loading and unloading of Cylinder type target objects” should be selected. Determine the Task type based on the target’s geometry, differences between its front and back, and the actual incoming arrangement.
2.2 Expected Results
After configuration, the following results should be achieved:
- Each target cylinder corresponds to a separate instance. Adjacent targets are not merged, and the pallet and other background objects are not treated as picking targets.
- The calculated pose matches the actual cylinder’s position and orientation; the Point Cloud template fits the instance Point Cloud in areas with valid features.
- The Pick Point is at the position specified by the process, and its direction meets the End-effector’s approach requirements.
- The instance output order meets the process requirements, and invalid poses and results that do not meet the picking conditions can be filtered out.
- Verify the vision results by replaying historical data and checking the 2D/3D results and logs, then perform acceptance testing with the robot operating at low speed.
2.3 Pre-configuration Checks
Use the following checks to confirm the setup conditions and prepare the required materials. Templates, keypoints, or Visual Models that have not yet been created can be completed in the later “Point Cloud Template and Target Configuration” chapter; all configuration does not need to be finished before creating a new project.
2.3.1 Check the Targets and Incoming Arrangement
- Cylinders used in the same Task have consistent models, main dimensions, and shapes, and the CAD model matches the actual targets.
- The cylinders’ placement orientation, visible surfaces, row and column arrangement, spacing, and permitted positional deviation have been established.
- Whether cylinders may be tightly packed, occlude one another, or arrive in multiple layers has been established, and corresponding representative samples have been prepared.
- Permitted picking positions, the End-effector’s approach direction, and the desired picking order have been established.
The Pattern mask needs to reflect the actual scene so that targets can be configured and the model trained later. For related configuration, see the Target Configuration Guide.
2.3.2 Prepare Model and Template Materials
- Prepare a CAD model that matches the actual cylinders and confirm its dimensional units.
- Use the same normalized mesh as the reference for the coordinate relationships among the template, keypoints, and target.
- When using an existing Visual Model, check that it corresponds to the target and Pattern mask. If no suitable model exists, complete training through One-Click Connection.
- When using an existing Point Cloud template or keypoint file, check its source and coordinate system to avoid mixing files generated from different normalization results.
- When creating a template from camera data, prepare a Point Cloud that reliably shows the cylinder’s valid features, then align the Point Cloud with the mesh later.
For creation methods, see the Point Cloud Template Creation Guide. For the training process, see the One-Click Connection User Guide.
Coordinate Consistency
A Point Cloud captured by the camera and a normalized mesh may use different coordinate systems. Before importing the Point Cloud template, keypoints, and mesh into the target, check that their dimensions, origins, and axis directions are consistent. If the template’s coordinate relationship is incorrect, correct the alignment before setting the Pick Point.
2.3.3 Check the Camera and Communication
- The camera, industrial PC, switch, and robot are correctly wired and powered.
- The camera’s field of view covers the entire incoming target area, with a margin for permitted positional deviation.
- 2D images and 3D Point Clouds can be captured normally, and the cylinders’ key features remain distinguishable despite reflections, occlusion, or missing Point Cloud data.
- The camera and on-site light source are mounted securely; imaging conditions are comparable when creating templates, collecting training materials, and running the system in production.
- The communication method, connection roles, and related settings used by the robot or PLC and PickWiz are consistent.
For specific procedures, see the Hardware Installation Guide, Camera Connection and Parameter Tuning Guide, and Robot Configuration Guide.
2.3.4 Check the End-effector and Robot Readiness
- The End-effector can reliably pick the actual cylinders, and the picking position and approach direction meet the process requirements.
- The tool TCP has been established. When using a gripper model, check how its origin and direction relate to the robot TCP.
- Before operating the robot with the vision system, complete Eye-hand calibration, check the calibration results, and select the correct calibration configuration in the Task.
- The targets are within the robot’s reach, and the approach and retreat paths are free of obstacles posed by personnel, cables, or equipment.
- For the first coordinated run, use low speed, single-step operation, or a single cycle. Verify the approach pose before performing continuous picking.
For configuration and checks, see the End-effector Configuration Guide, Eye-hand Calibration Procedure, and Calibration Result Check and Analysis.
Robot Operation with the Vision System
If Eye-hand calibration has not been completed or the correct calibration result has not been selected in the Task, check only the recognition, Point Cloud, pose, and Pick Point generation results. Do not allow the robot to move directly using unverified vision output coordinates.
Return to Case Overview · Next Chapter: Create a Task and Configure the Basics