TCP Client is a communication method in which PickWiz acts as the client and the robot acts as the server. When PickWiz communicates with the robot, it actively controls the robot, sends commands to the robot, and the robot executes the commands and returns data after receiving them.

Before configuring robot communication, please first refer to Project Operation Guide to create a new Project, then click Robot Configuration Panel on the main interface to enter the Add Robot interface.

1. Add Robot

（1）Enter the Add Robot interface and click +Add Robot

（2）Select TCP Client as the communication method, click Next, and enter the Robot Configuration interface

2. Configure the Robot

On the Robot Configuration interface, you can configure robot information, basic communication settings, and custom communication messages.

2.1 Robot Information

Configuration Item	Description
Communication Method	The communication protocol established between PickWiz and the robot. This cannot be modified
Configuration Name	The name of the robot configuration. Chinese and English characters, numbers, and symbols are supported
Configuration ID	The unique identifier of the robot configuration. It can be modified and supports numbers only
Brand	The robot brand. If the robot side is connected to a PLC, select Other Device for the brand
Model	The robot model. If the robot side is connected to a PLC, select Other Device for the model
Model File	When Other Device is selected for brand and model, you need to upload the model of that device. The model file format is urdf
Axis Count	The number of robotic arm axes. Three-axis, four-axis, and six-axis are supported
Rotation Expression	Describes the robot pose in three-dimensional space. Currently, Euler Angles and rotation vector are supported, and the Euler angle order can be modified.

2.2 Basic Communication Configuration

Configuration Item		Description
Communication Address	Host IP	The host IP address. In TCP communication mode, the default is 0.0.0.0
	Port Number	The specified port on the host to which the robot connects. The default port is 8887 and usually does not need to be modified
Read/Write Format	Euler Angle Unit	The unit of Euler Angles. Degree and radian are supported, with degree as the default.
	Length Unit	The length unit used by the robot, including Robot Pose, workpiece dimensions, etc. The default is mm
	Pose Type	The representation of Robot Pose. The default is Tool Pose. Tool Pose refers to the pose of the robot end Tool; Joint Motion refers to the pose of each robot joint, and the combination of all joint poses determines the pose of the robotic arm or end Tool. When Other Device is selected for the robot brand and model, only Tool Pose can be selected as the default pose type
	Delimiter	The delimiter is used to separate data. For example, a pose has 6 values, namely xyzabc, and the delimiter is used to distinguish these 6 values, such as x,y,z,a,b,c (no delimiter is added at the end of the data). The delimiter can be any character except escape characters such as \n, \r, and \t.
	End-of-message Special Characters	Some robotic arms may require special characters at the end of each message they receive or send, such as \n\r. You can enter these special characters here. The software appends these special characters to the end of each message it sends, and when receiving each message from the robotic arm, it attempts to remove the end special characters (if any).
	Command Length Limit	Some robotic arms can only receive or send a specified number of bytes in one message, so you can set the number of bytes for one message here
	Reserved Decimal Places	How many decimal places to retain for floating-point data in robot commands. Numbers only are supported, and the default is 1

2.3 Custom Communication Messages

General Command Rules

• The encoding format of a message sent to the robotic arm is utf-8;
• Commands represent the communication protocol between the software and the robotic arm. Different operations require their respective commands to be defined. A command consists of a command token and custom characters;
• Command tokens are provided in the form of shell variables: $ {command token}; command tokens are used to indicate placeholders. When sending a message, one command token is replaced with specific data. For example, the command token ${p} is replaced with 0,0,0,0,0,0 (the delimiter in the middle can be customized);
• Custom characters can be any characters (except escape characters such as \n, \r, and \t), and they can appear at any position in a command;
• A command may contain no command token at all and consist entirely of custom characters, or it may contain only a single command token, but it must still ensure that the commands for each operation are distinguishable (that is, the communication protocol must be unique). Otherwise, neither the software nor the robotic arm can determine what function to perform after receiving a message.
• How the software determines the meaning of a message: when a message is received, the software replaces the command tokens in all user-configured commands so that the entire command becomes a regular expression. It then matches the message against all regular expressions generated from the commands. If a match succeeds, it means the software knows what operation the message represents. If all matches fail, it means either the command configuration is incorrect or the data sent by the robotic arm is wrong.

3. TCP Command Description

3.1 Calibration Communication Messages

• PickWiz sends the request getRobotPose to the robot to obtain the robot coordinates

• The robot returns the robot coordinates to PickWiz

p: pose information of the robot end Tool

$(p): outputs the pose information of the robot end Tool, 6 data values

Example 1

PickWiz sends the request getRobotPose to the robot, and the robot returns the robot coordinates p,${p} to PickWiz.

Example 2

PickWiz sends the request (getRobotPose) to the robot, and the robot returns the robot coordinates p,${p} to PickWiz

Example 3

PickWiz sends the request getRobotPose to the robot, and the robot returns the robot coordinates p,${p}\n to PickWiz. \n is a custom character.

When the custom character is not included in the robot coordinates returned to PickWiz, the software cannot match the command. Only when the custom character is included in the returned robot coordinates can the software match the command;

3.2 Vision Computation Communication Messages

（1）Commands Sent from the Robot to PickWiz

• Placeholders

Placeholders are symbols or markers used in programming. During actual communication, placeholders are replaced with specific content.

Placeholder	Description
p	Robot Tool Pose. For 6 data values, when Pose Type is Tool Pose, the robot returns the current pose of the robot end Tool to PickWiz
j	The robot reference joint pose. For 6 data values, when Pose Type is Joint Motion, the robot needs to send Reference Joint Pose to calculate the Pick Point joint pose closest to the reference joint pose
ps	The incoming workpieces may have multiple sizes, so the robot needs to automatically adjust the grasping strategy based on different sizes. In different scenarios, the number of workpiece dimension values passed from the robot side and the corresponding workpiece information differ. For the circular surface scenario, 1 value is passed, representing the radius; for the cylinder scenario, 2 values are passed, representing radius\height; for the carton scenario, 3 values are passed, representing length\width\height; for the quadrilateral scenario, 2 values are passed, representing width\length.
p_tol	The incoming workpieces may have multiple sizes, so the robot needs to automatically adjust the grasping strategy based on different sizes. In different scenarios, the number of workpiece dimension tolerance values passed from the robot side and the corresponding workpiece information differ. For the circular surface scenario, 1 value is passed, representing the radius tolerance; for the cylinder scenario, 2 values are passed, representing radius\height tolerances; for the quadrilateral scenario, 2 values are passed, representing width\length tolerances.
pid	Workpiece ID, used by the robot to automatically switch the workpiece to pick
wid	task ID, used by the robot to automatically switch tasks
calib_id	Calibration ID, used by the robot to automatically switch eye-hand calibration configuration files
ee_id	Tool ID, used by the robot to automatically switch the end Tool during task execution
roi_id	The ID of the ROI, used by the robot to automatically switch the ROI
vp_id	Vision Parameter ID, used by the robot to automatically switch vision parameters
so_id	Scene Object ID, used by the robot to automatically switch the Scene Object
co	Vision computation rule 0: retake a photo, acquire an image, and perform vision computation; 1: use the previous image data for computation; 2: only take a photo and acquire the image 3: take a photo for random pose automatic sampling 4: real-time correction of vision drift 5: use historical data for vision computation 6: IPC shutdown

• Send Command Information (commands sent from the robot to PickWiz)

Configure the content of the command sent from the robot to PickWiz

d: command identifier

$(p): pose information of the robot end Tool, 6 data values

$(wid): task ID, 1 value, used by the robot to switch tasks

Example 1

Set the send command information to d,${ps} so that the robot sends only pose information to PickWiz to trigger the process

Example 2

Set the send command information to d,${ps}. In the single-sack depalletizing scenario, when the robot sends workpiece dimensions to PickWiz, an error is reported indicating that it applies only to cartons, circles, cylinders, and quadrilaterals. In the single-carton depalletizing scenario, the robot sends the carton length, width, and height to PickWiz to trigger the inspection process

Example 3

Set the send command information to d,${p},${wid} so that the robot sends pose information and the task ID to PickWiz, automatically switching the inspection flows of tasks 1, 2, 4, and 5

• Clear Cache

Clear Cache: In the one shot, multiple picks scenario, clear the current remaining instance cache so that a new photo will be taken the next time the inspection process is triggered

Picking Logic:

• One shot, one pick: the vision system takes one photo and performs one computation, sends the result to the robot after computation, discards any excess results, the robot performs the related picking action, and then the process repeats;
• One shot, multiple picks: the vision system takes one photo and performs one computation, sends the result to the robot after computation, saves the excess vision results, the robot performs the related picking action, and the next time the robot sends a photo computation command, the system first checks whether the saved vision results meet the requirements. If they do, the system does not execute the photo-taking + computation logic and instead returns the result directly to the robot, then repeats;

Examples

When One shot, multiple picks is not selected, PickWiz must perform photo-taking and computation every time the robot triggers the inspection process

When One shot, multiple picks is selected in Vision Computation Configuration, PickWiz can implement one photo-taking/computation cycle with multiple picks. As shown below, 5 instances are recognized. The first pick is triggered by PickWiz, and the next 4 picks are triggered by the robot. During those 4 picks, the robot sends pose information to PickWiz to trigger the inspection process, and PickWiz does not perform photo-taking and computation. Instead, it directly returns the Picking Pose.

The 1st pick is triggered by the client, and the 2nd and 3rd picks are triggered by the robot. The robot sends pose information to PickWiz to trigger the inspection process, and PickWiz does not perform photo-taking and computation but directly returns the Picking Pose. Then enter the co command to clear the cache, and PickWiz starts photo-taking and computation again for the 4th pick

（2）Commands Sent from PickWiz to the Robot

• Placeholders

Placeholders are symbols or markers used in programming. During actual communication, placeholders are replaced with specific content.

Placeholder	Meaning
s	Inspection result signal, vision inspection signal, 1 value. The default success value is 100, and other exceptions are 0. Custom configuration is supported
entity	Returned instance type, 1 value, where 1 indicates a pallet and 0 indicates a workpiece
ln	Remaining instance count, 1 value. In the one shot, multiple picks picking logic, this returns the current remaining cached instance count
rn	Returned instance count, 1 value. Returns the number of instances identified by instance segmentation 2D recognition. This value is less than or equal to the Sent Instance Count. If there are multiple instances, the Grasp-related Information Length increases with the returned instance count
tn	Total detected instances, 1 value. The number of detected instances before instance filtering
vn	Valid instance count, 1 value. The number of detected instances after instance filtering
info	Grasp-related information
cid	Workpiece type
sr	Calibration sampling result, which is used by the automatic sampling calibration command. Different calibration sampling results correspond to different signal values. 0: the sample count reaches the minimum quantity 1: successfully added a new sample 11: failed to add a new sample 12: automatic sampling ended and the sample list has been executed 13: failed to generate a sampling pose 14: calibration sampling page was not opened 15: system error
cp	Next sampling pose, used by the automatic sampling calibration command. It indicates the moving position for the robot to collect the next calibration sample
<SPLIT>	Special separator. When using it, you do not need to add ${}; directly add this identifier to the command, and the data will be forcibly split at this position. This function is mutually exclusive with the function of splitting data by character count, and this function has a higher priority.

• Vision Inspection Send Command（commands sent from PickWiz to the robot）

Configure the content of the command sent from PickWiz to the robot

• Automatic Calibration Sampling Send Command

When the eye-hand calibration method is random pose automatic sampling, this command must be configured. Otherwise, the robot cannot communicate with PickWiz

• Grasp-related Information

The lengths of the grasp-related information described below are for a single Pick Point. If there are multiple Pick Points, the length is the number of values x the number of Pick Points

Universal information for all scenarios

Field	Description
Picked Workpiece ID	1 value; indicates the current workpiece sequence number
Pick Point Pose	6 values; indicates the pose data of the Pick Point
Pick Point Index	1 value; indicates the current Pick Point sequence number
Additional Information	1 value; indicates the custom additional information of the current workpiece
Forward Point	Only applicable when the pose type is Joint Motion. The length is 1 + the number of Waypoints; the format is (number of Waypoints, joint Waypoint poses from the pre-pick point to the Pick Point)
Retreat Point	Only applicable when the pose type is Joint Motion. The length is 1 + the number of Waypoints; the format is (number of Waypoints, joint Waypoint poses from the Pick Point to the retreat pick point)

• Vision Status Code

The vision status codes are used for troubleshooting. The recommended status codes are shown in the table below:

${s} must be configured in Vision Inspection Send Command

• If the vision status code is not custom-configured, the default value is 100 for success and 0 for all other exceptions

Content	Default Value	Related Node
Success	100
Bottom plate detected	301	Foreground Processing(Foreground Builder)
No container detected	4000	Container Detection(Container Builder)
No instance detected	5000	Instance Segmentation(Instance Builder)
No pose detected	6000	Pose Estimation(Pose Builder)
No Pick Point detected	7000	Pick Point Generation(Grasp Builder)
No Path detected	8000	Robot Trajectory Generator(Trajectory Builder)
No instance dimensions detected	9000	Workpiece Dimensions Generator(Size Builder)
Empty ROI	1101	Invalid Point Filter(ValidPoints Criteria)
No valid instance	12000	Instance Filtering(Instance Criteria)
No valid workpiece pose	13000	Pose Filtering(Pose Criteria)
No valid Pick Point	14000	Pick Point Filtering(Grasp Criteria)
All detection results are unpickable due to collisions	15000	Collision Detection Filtering(Collision Criteria)
No instance with valid dimensions	17000	Workpiece Dimensions Filtering(Size Criteria)
Exception	20000	General exception error. Please check the logs for confirmation
Insufficient video memory	20001	General exception error. Please check the logs for confirmation
Invalid input (please check algorithm parameters)	20002	General exception error. Please check the logs for confirmation
Full inspection	20003	Related to point cloud processing operator(full_detect_judgment)
Foreign matter detection exception	20004	Related to instance filtering operator(foreign_matter_judgement)

• Capture Finished

Configure the command CaptureFinished in Capture Finished. After the camera finishes taking the photo, PickWiz sends the CaptureFinished command to the robot

When a KINGFISHER camera is used, the Capture Finished signal is changed to Imaging Complete. Imaging Complete means that the camera has completed image acquisition and processed the image data.

• When a KINGFISHER camera is used, the robot can configure the Image Acquisition Complete signal

Image Acquisition Complete means that the camera has obtained image data after taking a photo, but no point cloud has been generated and no image processing has been performed. It is commonly used when one robot sends a command signal before imaging is complete to trigger the system/script to perform the next step, without waiting for imaging completion (because imaging completion takes longer than image acquisition completion), which can reduce the overall process time to some extent.

4. Edit, Copy, and Delete the Robot

（1）Edit Robot Configuration

On the Add Robot interface, left-click a robot configuration to edit the selected robot configuration.

（2）Copy Robot Configuration

On the Add Robot interface, right-click a robot configuration to copy the selected robot configuration.

（3）Delete Robot Configuration

On the Add Robot interface, right-click a robot configuration to delete the selected robot configuration.

5. Select a Robot Configuration

After completing the TCP communication configuration, you can select a robot configuration for the current task scene on the task main interface. Select the corresponding robot configuration in the robot configuration selection bar, and PickWiz will attempt to establish communication with the robot through the TCP communication protocol.

If the communication connection is successful, the robot communication connection status in the lower-left corner of the main interface will display "Connected".

If the communication connection fails, the robot communication connection status will display "Disconnected". Please check whether the network cable between the robot and the industrial PC is securely connected and whether the program loaded and running on the robot system is correct, then reconfigure the robot.

Previous pageRobot Configuration and Communication Guide

Next pageTCP Server Communication Configuration