TCP is a communication method in which PickWiz acts as the TCP server and the Robot acts as the TCP client. When PickWiz communicates with the Robot, it does not actively control the Robot side. Instead, the Robot side sends TCP commands to PickWiz, and PickWiz returns response data after receiving the commands. Therefore, when using TCP communication, users need to write programs on the Robot side to communicate with PickWiz.

Before configuring Robot communication, please first refer to Project Operation Guide to create a new Project, then click Robot Configuration Panel on the main interface to enter the Add a Robot interface.

1. Add Robot

（1）Enter the Add a Robot interface and click +Add Robot

（2）Select TCP Server for the communication method, click Next, and enter the Robot Configuration interface

2. Configure the Robot

On the Robot Configuration interface, you can configure Robot information, basic communication settings, and custom communication messages.

2.1 Robot Information Configuration

Configuration Item	Description
Communication Method	The communication protocol established between PickWiz and the Robot. This cannot be modified.
Configuration Name	The name of the Robot configuration. Chinese, English, numbers, and symbols are supported.
Configuration ID	The unique identifier ID of the Robot configuration. It can be modified and supports numbers only.
Brand	The Robot brand. If the Robot side is connected to a PLC, select Other Device for the brand.
Model	The Robot model. If the Robot side is connected to a PLC, select Other Device for the model.
Visualization Model	When Other Device is selected for both brand and model, you need to upload the device model. The model file format is urdf.
Axis Count	The number of Robot arm axes. Three-axis, four-axis, and six-axis are supported.
Rotation Expression	Describes the Robot pose in 3D space. Currently Euler Angles and rotation vector are supported, and the Euler angle order can be modified.

2.2 Basic Communication Configuration

Configuration Item		Description
Communication Address	Host IP	The host IP address. Under TCP communication, the default is 0.0.0.0.
	Port Number	The specified port used by the Robot to connect to the host. The default port is 8887 and usually does not need to be modified.
Read/Write Format	Euler Angle Unit	The unit of Euler Angles. Degree and radian are supported, and the default is degree.
	Length Unit	The length unit used by the Robot, including Robot Pose and Object Dimensions. The default is mm.
	Pose Type	The representation method of the Robot Pose. The default is Tool Pose. Tool Pose refers to the pose of the Robot end Tool; joint angles refer to the poses of each Robot joint, and the combination of all joint poses can determine the pose of the Robot arm or end Tool. When Other Device is selected for the Robot brand and model, only Tool Pose can be selected by default.
	Delimiter	The delimiter is used to separate data. For example, a pose has 6 data values, namely xyzabc. The delimiter is used to distinguish these 6 data values, such as x,y,z,a,b,c (no delimiter is added at the end of the data). Any character except escape characters such as \n, \r, and \t can be used as the delimiter.
	Ending Special Characters	Some Robot arms may require special characters at the end of each sent or received message, such as \n\r. You can enter these special characters here. The software appends them to the end of each sent message and tries to remove them from the end of each received message from the Robot arm (if any).
	Command Length Limit	Some Robot arms can send or receive only a specified number of bytes in one message, so you can set the byte count of a single message here.
	Reserved Decimal Places	How many decimal places to keep for floating-point data in Robot commands. Numbers only. The default is 1.

2.3 Custom Communication Message Configuration

General Command Rules

3. TCP Command Description

3.1 Calibration Communication Messages

• Command sent from the Robot to PickWiz

  • p: The pose information of the Robot end Tool

  • $(p): Outputs the pose information of the Robot end Tool, with 6 values

• Command sent from PickWiz to the Robot

  • getRobotPose: Gets the Robot coordinates

Example

1. PickWiz sends the request getRobotPose to the Robot, and the Robot returns the Robot coordinates p,${p} to PickWiz.

   

   

2. PickWiz sends the request (getRobotPose) to the Robot, and the Robot returns the Robot coordinates p,${p} to PickWiz.

   

   

3. PickWiz sends the request getRobotPose to the Robot, and the Robot returns the Robot coordinates p,${p}\n to PickWiz.

   When the Robot does not include the custom character in the returned Robot coordinates, PickWiz cannot match the command. Only when the Robot includes the custom character in the returned Robot coordinates can the software match the command.

   

   

   

   Or, enter \n in Ending Special Characters, and the software will automatically determine and parse it.

3.2 Vision Computation Communication Messages

3.2.1 Commands Sent from the Robot to PickWiz

Placeholder Reference

A placeholder is a symbol or marker used in programming. During actual communication, the placeholder is replaced with specific content or data.

Placeholder	Description
p	Robot Tool Pose. When the Pose Type is Tool Pose, the Robot returns the current pose of the Robot end Tool to PickWiz. The data length is 6.
j	Robot reference joint pose. When the Pose Type is Joint Angles, the Robot needs to send the Reference Joint Pose, which is used to calculate the joint pose of the Pick Point closest to that reference joint pose.
ps	Object Dimensions. When incoming materials include Target Objects of different sizes, the Robot needs to automatically adjust the grasping strategy according to different sizes. In different scenarios, the number of values passed for Object Dimensions and the corresponding object information are different. In circular-surface scenarios, 1 value is passed, representing the radius; in cylinder scenarios, 2 values are passed, representing radius\height; in carton scenarios, 3 values are passed, representing length\width\height; in quadrilateral scenarios, 2 values are passed, representing width\length.
p_tol	Object dimension tolerance. When incoming materials include Target Objects of different sizes, the Robot needs to automatically adjust the grasping strategy according to different sizes. In different scenarios, the number of values passed for dimension tolerance and the corresponding object information are different. In circular-surface scenarios, 1 value is passed, representing the radius tolerance; in cylinder scenarios, 2 values are passed, representing radius\height tolerance; in quadrilateral scenarios, 2 values are passed, representing width\length tolerance.
pid	Target Object ID, used by the Robot to automatically switch the Target Object to pick
wid	task ID, used by the Robot to automatically switch tasks
calib_id	Calibration ID, used by the Robot to automatically switch eye-hand calibration configuration files
ee_id	Tool ID, used by the Robot to automatically switch the end Tool when executing tasks
roi_id	ROI ID, used by the Robot to automatically switch ROI
vp_id	Vision Parameter ID, used by the Robot to automatically switch vision parameters
so_id	Scene Object ID, used by the Robot to automatically switch Scene Object
co	Photo-taking calculation type, used to switch different photo-taking methods depending on on-site use. 0: take photos again and perform vision computing 1: use cached images (images acquired last time) for vision computing 2: only take photos 3: take photos for Automatic random pose sampling 4: real-time vision drift correction 5: use historical data for vision computing 6: IPC shutdown

Vision Result Send Command

Configure the content of the command sent from the Robot to PickWiz. The default is d,${p},${wid}.

• d: Command identifier, used to avoid confusion between repeated commands. For example, in calibration communication, the sent command identifier is p.

• $(p): The pose information of the Robot end Tool, with 6 values

• $(wid): task ID, 1 value, used by the Robot to switch tasks

Example

1. Set the send command information to d,${p}. The Robot sends only pose information to PickWiz to trigger the process.

   

   

2. Set the send command information to d,${ps}. In a single-sack depalletizing scenario, if the Robot sends Object Dimensions to PickWiz, an error is reported indicating that it is applicable only to cartons, circles, cylinders, and quadrilaterals. In a single-carton depalletizing scenario, the Robot sends the length, width, and height of the carton to PickWiz to trigger the detection process.

   

   

3. Set the send command information to d,${p},${wid}. The Robot sends pose information and the task ID to PickWiz, automatically switching to the detection workflows of tasks 1, 2, 4, and 5.

   

   

   

   

   

Clear Cache

Only for the one shot, multiple picks case, this clears the remaining instance cache so that photos will be taken again the next time the detection process is triggered.

Prerequisite information:
You need to understand the usage logic of “Vision Computing Configuration - Pick Logic”:

• One Shot, One Pick: The vision system takes photos once and computes once, sends the results to the Robot after computation, discards any extra results, the Robot executes the related picking action, and then repeats.
• One Shot, Multiple Picks: The vision system takes photos once and computes once, sends the computed results to the Robot, saves the extra Vision Results, the Robot executes the related picking action, and the next time the Robot sends a photo-taking calculation command, the system first checks whether the saved Vision Results meet the requirements. If they do, it does not execute photo-taking + computation and directly returns the results to the Robot, then repeats.

Example

When One Shot, Multiple Picks is not selected, PickWiz must perform photo-taking and computation every time the Robot triggers the detection process.

When One Shot, Multiple Picks is selected in Vision Computing Configuration, PickWiz can realize one photo-taking computation for multiple picks. As shown below, 5 instances are recognized. The first pick is triggered by PickWiz, and the next 4 picks are triggered by the Robot. During those 4 picks, the Robot sends pose information to PickWiz to trigger the detection process, and PickWiz does not perform photo-taking or computation, but directly returns the Picking Pose.

The first pick is triggered by the client, and the second and third picks are triggered by the Robot. The Robot sends pose information to PickWiz to trigger the detection process, and PickWiz does not perform photo-taking or computation, but directly returns the Picking Pose. Then enter the co command to clear the cache, and PickWiz starts photo-taking and computation again for the fourth pick.

3.3 Commands Sent from PickWiz to the Robot

Placeholder Reference

A placeholder is a symbol or marker used in programming. During actual communication, the placeholder is replaced with specific content.

Placeholder	Meaning
s	vision status code, 1 value. The default success value is 100, and other exceptions are 0 by default. This can be customized.
entity	Returned instance type, 1 value. 1 means pallet and 0 means Target Object.
ln	Remaining instance count, 1 value. Under the one shot, multiple picks picking logic, this returns the current number of remaining cached instances.
rn	Returned instance count, 1 value. This returns the number of instances recognized by instance segmentation 2D. This value is less than or equal to Number of Instances to Send. If there are multiple instances, the Length of Picking-related Information increases as the returned instance count increases.
tn	Total detected instances, 1 value. The number of detected instances before instance filtering.
vn	Valid instance count, 1 value. The number of detected instances after instance filtering.
info	Picking-related information
cid	Target Object type
sr	Calibration sampling result. This is an automatic sampling calibration command, and different calibration sampling results correspond to different signal values. 0: the minimum number of samples is reached 1: a new sample is added successfully 11: failed to add a new sample 12: automatic sampling ends and the sample list is fully executed 13: failed to generate a sampling pose 14: calibration sampling page not entered 15: system error
cp	Next sampling pose. This is an automatic sampling calibration command and indicates the Robot movement position for collecting the next calibration sample.
<SPLIT>	Special delimiter. When using it, do not add ${}. Directly add this identifier into the command, and the data will be forcibly split at that position. Note: this function is mutually exclusive with the function that splits data by character count, and this function has higher priority.

Vision Result Send Command

Configure the content of the command sent from PickWiz to the Robot. The default is ${rn},${info}.

Automatic Calibration Sampling Send Command

When the eye-hand calibration method is Automatic random pose sampling, you need to set this command. Otherwise, the Robot cannot communicate with PickWiz.

Picking-related Information

The following length values for picking-related information represent the length for a single Pick Point. If there are multiple Pick Points, the total length is the number of values x the number of Pick Points.

• Information Common to All Scenarios

  Field	Description
  Picked Target Object ID	1 value; indicates the current Target Object index
  Pick Point Pose	6 values; indicates the pose data of the Pick Point
  Pick Point Index	1 value; indicates the current Pick Point index
  Additional Information	1 value; indicates the custom additional information of the current Target Object
  Approach Point	Applicable only when the pose type is Joint Angles. The length is 1 + the number of Waypoints; format: (number of Waypoints, joint Waypoint poses from the pre-pick point to the Pick Point)
  Retreat Point	Applicable only when the pose type is Joint Angles. The length is 1 + the number of Waypoints; format: (number of Waypoints, joint Waypoint poses from the Pick Point to the retreat Pick Point)

• Scenario-specific Information

  • Carton/Sack Depalletizing

    • Product Length: 1 value, the current product length (along the image x direction)

    • Product Width: 1 value, the current product width (along the image y direction)

    • Product Direction: 1 value, the product orientation (0: horizontal along x, 1: vertical along y)

    • Upper-left Pallet Vertex Position: 3 values (x, y, z), indicating the position information of the upper-left pallet vertex in the current Scene

    • Upper-right Pallet Vertex Position: 3 values (x, y, z), indicating the position information of the upper-right pallet vertex in the current Scene

    • Lower-left Pallet Vertex Position: 3 values (x, y, z), indicating the position information of the lower-left pallet vertex in the current Scene

    • Lower-right Pallet Vertex Position: 3 values (x, y, z), indicating the position information of the lower-right pallet vertex in the current Scene

  • Carton Depalletizing Information:

    • Product Height: 1 value, the current product height

  • Circular-surface Ordered/Random Picking:

    • Circular-surface Radius: 1 value, the current product circular-surface radius

  • Cylinder Ordered/Random Picking:

    • Circular-surface Radius: 1 value, the current product circular-surface radius

    • Cylinder Height: 1 value, the current product height

  • Random Picking:

    • Target Object Category: 1 value, the current Target Object category, used in scenarios that recognize multiple types of Target Objects.

  • Quadrilateral Ordered/Random Picking:

    • Quadrilateral Length: 1 value, the current product length

    • Quadrilateral Width: 1 value, the current product width

Vision Status Code

Vision status codes are used for troubleshooting. The recommended status codes are shown in the following table:

You need to configure ¥{s} in Vision Result Send Command

• If the vision status code is not customized, the default value is 100 for success and 0 for all other exceptions.

Content	Default Value	Related Node
Success	100
Bottom plate detected	301	Foreground Processing (Foreground Builder)
No bin detected	4000	Bin Detection (Container Builder)
No instance detected	5000	Instance Segmentation (Instance Builder)
No pose detected	6000	Pose Estimation (Pose Builder)
No Pick Point detected	7000	Pick Point Generation (Grasp Builder)
No Path detected	8000	Robot Trajectory Generator (Trajectory Builder)
No instance size detected	9000	Object Size Generator (Size Builder)
Empty ROI	1101	Valid Point Filter (ValidPoints Criteria)
No valid instance	12000	Instance Filtering (Instance Criteria)
No valid Object Pose	13000	Pose Filtering (Pose Criteria)
No valid Pick Point	14000	Pick Point Filtering (Grasp Criteria)
All detection results collide and cannot be picked	15000	Collision Detection Filtering (Collision Criteria)
No instance with valid size	17000	Object Size Filtering (Size Criteria)
Exception	20000	Common exception error. Please check the logs to confirm.
Insufficient video memory	20001	Common exception error. Please check the logs to confirm.
Invalid input (please check algorithm parameters)	20002	Common exception error. Please check the logs to confirm.
Full inspection	20003	Related to point cloud processing operator (full_detect_judgment)
Foreign matter detection exception	20004	Related to instance filtering operator (foreign_matter_judgement)

Imaging Completed

Imaging Completed means that the Camera has finished image acquisition and processed the image data.

Image Acquisition Completed

Image Acquisition Completed means that the Camera has obtained image data after taking a photo, but has not generated a Point Cloud or performed image processing. It is often used when a Robot sends a signal before imaging is completed to trigger the system/script to execute the next step. There is no need to wait for imaging to finish (because imaging completion takes longer than image acquisition completion), which can reduce the overall process time to some extent.

Example

In Image Acquisition Completed / Imaging Completed, configure the command CaptureFinished. After image acquisition or imaging is completed, PickWiz sends the CaptureFinished command to the Robot, which can trigger the next operation.

4. Edit, Copy, and Delete the Robot

（1）Edit the Robot configuration

On the Add a Robot interface, left-click a Robot configuration to edit the selected Robot configuration.

（2）Copy the Robot configuration

On the Add a Robot interface, right-click a Robot configuration to copy the selected Robot configuration.

（3）Delete the Robot configuration

On the Add a Robot interface, right-click a Robot configuration to delete the selected Robot configuration.

5. Select a Robot Configuration

After the TCP communication configuration is completed, you can select the Robot configuration for the current task Scene on the task main interface. Select the corresponding Robot configuration in the Robot configuration selection bar, and PickWiz will try to establish communication with the Robot through the TCP communication protocol.

If the communication connection is successful, the Robot communication connection status in the lower-left corner of the main interface will display "Connected".

If the communication connection fails, the Robot communication connection status will display "Disconnected". Please check whether the network cable between the Robot and the Vision system industrial PC (Industrial Personal Computer) is firmly connected and whether the program loaded and running on the Robot system is correct, then configure the Robot again.

When PickWiz acts as the server and multiple Robot clients connect, the logs will show a prompt indicating that the request is rejected when clients other than the already connected client attempt to connect, as shown below.

Previous pageTCP Client Communication Configuration

Next pageModBus Communication Configuration