The TCP client uses PickWiz as the client and Robot as the server. When PickWiz communicates with Robot, it actively controls Robot and sends instructions to Robot. After receiving the instructions, Robot executes the instructions and returns data.

Before configuring Robot communication, please refer to the [Project Operation Guide] (项目操作指南.md) to create a new project, and then click Robot Configuration Panel on the main interface to enter the New Robot interface.

1. Add new Robot

(1) Enter the Add Robot interface and click +Add Robot

(2) Select TCP client as the communication method, click Next to enter the Robot Configuration interface

2. Configure Robot

In the Robot Configuration interface, you can configure Robot information, basic communication configuration, and customized communication message configuration.

2.1 Robot information

Configuration Item	Description
Communication method	The communication protocol used to establish communication between PickWiz and Robot, which cannot be modified
Configuration name	The name of Robot configuration, supports Chinese and English, numbers, and characters
Configuration ID	The unique identification ID of the Robot configuration, which can be modified and only supports numbers
Brand	The brand of the Robot. If the Robot is connected to a PLC, select the brand. Other equipment
Model	The model of the Robot. If the Robot is connected to a PLC, select the model. Other equipment
Model	When selecting another device for brand and model, you need to upload the model of the device. The format of the model file is urdf
Number of axes	The number of axes of the robot arm owned by the Robot, optional three axes, four axes, or six axes
Rotation expression	Describes Robot's posture in three-dimensional space. Currently, there are two methods: Euler Angles and rotation vector. The order of Euler Angles can be modified.

2.2 basic communication configuration

in TCP communication mode . by default. Special characters at the end of The floating point data in the

configuration item		Description
communication address	host ip	host ip address, the default is 0.0.0.0
	port number	Robot connects to the designated port of the host. The default port number is 8887. Generally, there is no need to modify
read and write format	Euler Angles unit	Euler Angle's unit, optional angle, radian, default angle.
	length unit	Robot uses the length unit, including Robot Pose, Target size, etc. The default is mm
	pose type	Robot Pose representation method, default tool pose. The tool pose refers to the pose of the Robot End-effector; the joint angle refers to the pose of each joint of the Robot. The pose combination of each joint can determine the pose of the robot arm or End-effector. When selecting other equipment for the Robot brand and model, the pose type can only select the tool pose
	delimiter	The delimiter can be filled with any character except the escape characters \n, \r, and \t.
		Some robots may receive or send a message that requires some special characters at the end, such as \n\r. You can fill in these special characters here. The software will append these special characters at the end when sending each message. When receiving each message from the robot, it will try to remove the special characters at the end (if any).
	Instruction length limit	Some robot arms can only receive or send a specified number of bytes of data in one message, so you can set the number of bytes in a message here
	Retain the number of decimal places	Robot instruction retains several decimal places and only supports numbers. The default is 1

2.3 Custom communication message

General Instruction Rules

• The encoding format of a message sent to the robotic arm is: utf-8;
• Instructions represent the communication protocol between the software and the robotic arm. Different operations need to define their own instructions; an instruction consists of instruction symbols and custom characters;
• The command is provided in the form of a shell variable: ${command}; the command is used to identify this as a placeholder. When sending a message, a command will be replaced with specific data, such as the command ${p}, which will be replaced with: 0,0,0,0,0,0 (the middle separator can be customized);
• Custom characters can be any characters (except escape characters such as \n, \r, and \t), and can appear anywhere in an instruction;
• Instructions may not require a command character, they may all be custom characters, or they may have only a single command character, but it must be ensured that the instructions for each operation are differentiated (that is, the communication protocol should be unique), otherwise neither the software nor the robot arm will be able to locate what function should be performed after receiving a message.
• How the software locates the meaning of a message: When receiving a message, the software will replace the command symbols in all user-configured commands to turn the entire command into a regular expression, and then match the message with all the regular expressions obtained through the command. When the match is successful, it means that it knows what operation the message represents. If all matches fail, it means that there is either a problem with filling in the command, or the data sent by the robot arm is incorrect.

3. TCP command description

3.1 calibration communication message

• PickWiz sends a request to Robot: getRobotPose to obtain the coordinates of Robot

• Robot returns Robot coordinates to PickWiz

p: Position information of RobotEnd-effector

$(p): Output the pose information of RobotEnd-effector, 6-bit data

Example 1

PickWiz sends a request to Robot to getRobotPose, and Robot returns Robot coordinates p,${p} to PickWiz.

Example 2

PickWiz sends a request (getRobotPose) to Robot, and Robot returns Robot coordinates p,${p} to PickWiz.

Example 3

PickWiz sends a request to Robot for getRobotPose, and Robot returns Robot coordinates p, ${p}\n to PickWiz, where \n is a custom character.

When Robot returns Robot coordinates to PickWiz that do not contain custom characters, the software cannot match the instruction; when Robot returns Robot coordinates to PickWiz that contain custom characters, the software can match the instruction;

3.2 visual computing communication message

(1) Robot sent to PickWiz command

• Placeholder

Placeholders refer to symbols or marks used in programming. The placeholders will be replaced by specific content during actual communication.

Placeholder	Description
p	Robot tool pose, 6-digit data when the pose type is tool pose, Robot returns the current RobotEnd-effector pose to PickWiz
j	Robot reference joint pose, 6-bit data when the pose type is joint angle, Robot needs to send the reference joint pose, which is used to calculate the Pick Point joint pose closest to the reference joint pose
ps	Target size incoming materials are Targets of multiple sizes, and Robot needs to automatically adjust the crawling strategy according to different sizes. In different Scenes, the Target size digits passed in from the Robot side are different from the Target information corresponding to the digits. The circular Scene inputs 1 digit, representing the radius; the cylindrical Scene inputs 2 digits, representing radius\height; the carton Scene inputs 3 digits, representing length\width\height; the quadrilateral Scene inputs 2 digits, representing width\length.
p_tol	Target size tolerance The incoming materials are Targets of multiple sizes, and the Robot needs to automatically adjust the crawling strategy according to different sizes. In different Scenes, the number of digits of the Target size tolerance passed in by the Robot side is different from the Target information corresponding to the number of digits. The number of digits passed in to the circular Scene is 1, representing the tolerance of the radius; the digits passed in to the cylindrical Scene are 2 digits, representing the tolerance of radius\height; the number of digits passed in to the quadrilateral Scene is 2 bits, representing the tolerance of width\length.
pid	TargetID, used for Robot to automatically switch the crawled Target
wid	TaskID, used by Robot to automatically switch Task
calib_id	Calibration ID, used for Robot to automatically switch Eye-hand calibration configuration files
ee_id	End-effectorID, used to automatically switch End-effector when Robot performs tasks
roi_id	ID of ROI, used by Robot to automatically switch ROI
vp_id	Visual ParameterID, used by Robot to automatically switch visual Parameter
so_id	Scene ObjectID, used by Robot to automatically switch Scene Object
co	Visual calculation rules 0: Take pictures again and perform visual calculations; 03XQZ3: Taking pictures of "random attitude automatic sampling" 4: Real-time correction of visual drift 5: Using historical data for visual calculations

• Send command information (Robot sends command to PickWiz)

Configure the content of instructions sent by Robot to PickWiz

d: instruction identifier

$(p): RobotEnd-effector’s pose information, 6-bit data

$(wid): TaskID, 1 digit, used for Robot to switch Tasks

Example 1

The sending command information is set to d,${ps}. Robot only sends pose information to PickWiz to trigger the process.

Example 2

The sending command information is set to d,${ps}. In the single bag removal Scene, Robot sends the Target size to PickWiz, and an error will be reported and it is only applicable to cartons, circles, cylinders, and quadrilaterals. In the single carton removal Scene, Robot sends the length, width, and height of the carton to PickWiz, triggering the detection process.

Example 3

The sending command information is set to d,${p},${wid}. Robot sends pose information and TaskID to PickWiz, automatically switching the detection process of Task1, 2, 4 and 5.

• Clear cache

Clear Cache: In the case of take one shot and capture many, clear the current remaining instance cache and take pictures again when the detection process is triggered next time.

Grab logic:

• Take one and grab one: means to visually take a picture once, calculate it once, send the result to Robot after calculation, discard the redundant results, Robot performs the relevant grabbing action, and then repeats;
• Shoot once and grab many: It means to visually take a picture once, calculate once, send the calculation result to Robot, save the redundant visual results, and Robot performs relevant grabbing actions. The next time Robot sends a photo calculation instruction, it will first check whether the saved visual results meet the requirements. If so, the logic of taking photos + calculations will not be executed, and the results will be directly returned to Robot, and then repeat;

Example

If you don't check "Snap once, grab multiple times", every time Robot triggers the detection process, PickWiz will perform photo calculations.

In the visual computing configuration, check Shoot one, grab multiple, and PickWiz can realize photo calculation and capture multiple times at one time. As shown in the figure below, 5 instances were identified. The first capture was triggered by PickWiz, and the next 4 captures were triggered by Robot. During the last 4 captures, Robot sent pose information to PickWiz to trigger the detection process. PickWiz did not perform photo calculations, but directly returned to the capture pose.

The first capture is triggered by the client, and the second and third captures are triggered by Robot. Robot sends pose information to PickWiz to trigger the detection process. PickWiz does not perform photo calculations, but directly returns the capture pose. Then enter the co command to clear the cache, and grab PickWiz for the fourth time and start taking pictures again.

(2)PickWiz sent to Robot command

• Placeholder

Placeholders refer to symbols or marks used in programming. The placeholders will be replaced by specific content during actual communication.

Placeholder	Meaning
s	Detection result signal, visual detection signal, 1 bit, the default value is 100 for success, 0 for other exceptions, customizable configuration
entity	Returns the instance type, 1 bit, 1 is pallet, 0 is Target
ln	The number of remaining instances, 1 digit, under the fetching logic of shoot one, grab many, returns the current number of remaining cached instances
rn	Returns the number of instances, 1 digit, returns the number of instances identified by Instance Segmentation2D. The value is less than or equal to the number of sent instances. If there are multiple instances, the length of the fetched related information will increase with the number of returned instances
tn	The total number of instance detections, 1 digit, the number of detection instances that are not instance filtered
vn	Valid instance number, 1 digit, the number of detected instances after instance filtering
info	Grab relevant information
cid	Target type
sr	Calibration sampling result is an automatic sampling calibration instruction. Different calibration sampling results correspond to different signal values. 0: The sample reaches the minimum number 1: The new sample was added successfully
cp	The next sampling pose is the automatic sampling calibration instruction. The next time Robot collects the moving position of the calibration sample
<SPLIT>	Special delimiter, no need to add ${} when using, add this mark directly in the command, at this position, the data will be forcibly split. This function is mutually exclusive with the function of splitting data by the number of characters, and this function has a higher priority.

• Visual inspection sends instructions (PickWiz sends instructions to Robot)

Configure PickWiz to send instructions to Robot

• Automatic calibration sampling and sending instructions

When the Eye-hand calibration method is "random posture automatic sampling", you need to set this command, otherwise Robot cannot communicate with PickWiz

• Grab relevant information

The length of the crawling related information expressed below is the number of digits of the length of a single Pick Point. If it is multiple Pick Points, the length is the number of digits x the number of Pick Points.

General information for all Scenes

field	Description
grab TargetID	1 bit; indicates the current Target serial number
Pick Point pose	6 bits; represents the pose data of Pick Point
Pick Point index	1 bit; indicates the current Pick Point serial number
Additional information	1 bit; indicates the current Target’s custom additional information
forward point	is only applicable when the pose type is joint angle, the length is 1 + the number of path points; the format is (number of path points, joint path point posture from pre-grab point to Pick Point)
retreat point	is only applicable when the pose type is joint angle, the length is 1 + the number of path points; the format is (number of path points, joint path point posture from Pick Point to retreat Pick Point)

• Visual Status Code

Visual status codes are used for troubleshooting. The recommended status codes are as follows:

¥{s} needs to be configured in Visual Inspection Send Command

• If the visual status code is not customized, the default value is to return 100 on success and 0 for other exceptions.

Error content	Default value	Related nodes
Success	100
Detect base plate	301	Foreground Builder
Container frame not detected	4000	Container frame detection (Container Builder)
Instance not detected	5000	Instance Segmentation(Instance Builder)
No pose detected	6000	Pose Estimation (Pose Builder)
Pick Point not detected	7000	Pick Point Generation (Grasp Builder)
No path detected	8000	Robot Trajectory Builder (Trajectory Builder)
Instance size not detected	9000	Target Size Builder (Size Builder)
Empty ROI	1101	ValidPoints Criteria
No valid instance	12000	Instance Criteria
No valid Target pose	13000	Pose Criteria
No valid Pick Point	14000	Pick Point filter (Grasp Criteria)
All detection results are collision-uncatchable	15000	Collision detection filter (Collision Criteria)
No instance with valid size	17000	Target size filter (Size Criteria)
Exception	20000	General exception error, please check the log to confirm
Insufficient video memory	20001	General exception error, please check the log to confirm
Invalid input (please check the algorithm Parameter)	20002	General exception error, please check the log for confirmation
Full inspection	20003	Point Cloud processing operator related (full_detect_judgment)
Foreign object detection anomaly	20004	Instance filter operator related (foreign_matter_judgement)

• Photo taken

Configure the command CaptureFinished when the photo is taken. After the Camera takes the photo, PickWiz will send the CaptureFinished command to the Robot.

When using KINGFISHERCamera, the "Photography Completed" signal is changed to "Imaging Completed". "Imaging Completed" means that the Camera has completed taking pictures and processing the image data.

• When using KINGFISHERCamera, Robot can configure the "Picture acquisition completed" signal

"Image acquisition completed" means that the camera acquired the image data after taking the picture, but did not generate a Point Cloud or perform image processing. It is often used by a Robot to send out a command signal before the imaging is completed, triggering the system/script to perform the next operation without waiting for the imaging to complete (because the completion of imaging takes more time than the completion of image acquisition), which can reduce the time-consuming of the overall process to a certain extent.

4. Edit, copy, and delete Robot

(1) Edit Robot configuration

In the Add Robot interface, left-click the Robot configuration to edit the selected Robot configuration.

(2) Copy Robot configuration

Right-click the Robot configuration on the Add Robot interface to copy the selected Robot configuration.

(3) Delete Robot configuration

Right-click the Robot configuration on the Add Robot interface to delete the selected Robot configuration.

5. Select Robot configuration

After completing the TCP communication configuration, you can select the Robot configuration for the current TaskScene in the Taskmain interface and select the corresponding Robot configuration in the Robot configuration selection column. PickWiz will try to establish communication with the Robot through the TCP communication protocol.

If the communication connection is successful, the Robot communication connection status in the lower left corner of the main interface will display "Connected".

If the communication connection fails, the Robot communication connection status displays "Not Connected". Please check whether the network cable between the Robot and the industrial computer is firmly connected, whether the program loaded and run by the Robot system is correct, and then reconfigure the Robot.

Previous pageRobot Configuration and Communication Guide

Next pageTCP Server Communication Configuration