ROS2 Foxy Snap - Performance issue

[Thomas_SF]

Hi together,
I have developed a ROS2 snap with included UR driver and Datalayer interface for the ctrlX Core and installed it on the controller successfully.

When I trigger the command to start ROS2 and connect to the robot, everything works fine at the beginning. After the UR driver has been initialised and ROS is ready, I start the robot's External Control program with the configured ctrlX-IP and the robot connects to ROS successfully:

[ros2_control_node-1] [INFO] [UR_Client_Library]: Robot connected to reverse interface. Ready to receive control commands.

However, the connection breaks off again after a few seconds:

[ros2_control_node-1] [INFO] [UR_Client_Library]: Connection to reverse interface dropped.

In my experience, this problem occurs when ROS does not have enough computing power.

In addition, I noticed the following message by starting ROS2 on the control:

[ros2_control_node-1] [WARN] [UR_Client_Library]: No realtime capabilities found. Consider using a realtime system for better performance

As far as I know the ctrlX Core is realtime capable.

---

Question:
Is there a possibility to provide more computing power to my snap or to increase the priority of the process, or is the computing power of the controller not sufficient?
Is it possible to achieve realtime capabilities with a custom snap ?

---

Hardware:

• ctrlX-Core with Ubuntu 20.04
• UniversalRobot CB3

---

Thanks in advance

[CodeShepherd]

Our system is Ubuntu Core20 with PREEMPT patch to achieve realtime capabilities. I do not now if and how you can attach your ROS package to this. We a using a celix framework for the Rexroth specific RT parts with an integrated scheduler there customers can write their own bundles using our SDK for ctrlX AUTOMATION.

You can get information about the utilisation via the web UI:web UI information

web UI performance and processes

 

[Thomas_SF]

Thanks for the quick reply.

It doesn't look like my ROS2 snap is using too much processing power, after ROS is booted the CPU load of core 0 and 1 stabilises at around 55%.
It seems to me that the cycle time with which the snap is processed is too slow for the ROS communication.

Via the scheduler I found out that core 0&1 do not handle real time tasks. Core 2&3 seem to be defined for real-time tasks.

However, only callable functions can be assigned to the task and I'm not seeing my ROS Process or Snap there.

Would it be possible to define my ROS application as callable in order to achieve a higher cycle time and real-time functionality?

[CodeShepherd]

Own created apps always run free running in the system. So without any speed restrictions.

The scheduler like I mentioned is part of a CELIX framework written in C++. To be part of this you have to create a CELIX bundle and in this a callable can be created.

Could you check your RAM? Should be fine but just to be sure.

I already created some ROS(1) snaps that were running fine and could control a six axes articulated robot arm.

[CodeShepherd]

Any news here or can this topic be closed?

[Thomas_SF · Accepted solution]

Hi sorry for the late reply I have tried a few things.

It seems that all apps get by default a CPU affinity for the first two cores 0 and 1.
The ROS2-UR driver consists of several processes each reflecting a specific node. In this case the node "ros2_control_node" causes problems, because due to the CPU load of the two standard cores it comes to the fact that the connection breaks.
To solve this problem I assigned the node to another CPU, in my case core 3. This works with the Python "os-commands", using 

os.sched_setaffinity(0, {3})

, I can assign my current Python Skipr to the not heavily used CPU 3 and then start my launch file as a subprocess in it. The subprocess is then automatically assigned to the respective CPU of the Python script. The not so time critical nodes like a trajectory planner e.g. "move_group" can be executed on the busy CPUs.

This has significantly improved the stability of my application but not completely fixed it.

For a complete solution it would have to be assigned to a realtime priority, but as far as I found out there is no plug&play solution for this. (only possible via root access and SSH).