Node-RED sporadic DL_INVALID_ADDRESS

[AndreasL]

I have a customer that has a Node-RED flow that subscribes and reads datalayer nodes.

It works most of the time, but now and again they are getting DL_INVALID_ADDRESS when reading PLC variables.

They are getting a few errors in the logbook, but then it starts working again

info	2023-08-29T06:34:52.147288+02:00				29 Aug 06:34:52 - [error] [ctrlx-datalayer-subscribe:TransactionID] DL_INVALID_ADDRESS
info	2023-08-29T06:34:52.147288+02:00				29 Aug 06:34:52 - [error] [ctrlx-datalayer-subscribe:PlcState] DL_INVALID_ADDRESS
info	2023-08-29T06:34:42.128743+02:00				29 Aug 06:34:42 - [error] [ctrlx-datalayer-subscribe:TransactionID] DL_INVALID_ADDRESS
info	2023-08-29T06:34:42.124838+02:00				29 Aug 06:34:42 - [error] [ctrlx-datalayer-subscribe:PlcState] DL_INVALID_ADDRESS

We are not really sure how to track down what is happening here, anyone have any ideas?

I have the tar file from the core but I'm not allowed to share it publicly.

[AndreasL]

Forgot to attatch the logfile.

[Sgilk]

Hello Andreas,

What versions of ctrlX CORE firmware and Node-RED are being used? At what rate are these requests being made? Are these subscription nodes being triggered more than a single time? It seems like that might be the case based on the logs. Subscriptions only need to be created a single time and it's possible the duplicate subscriptions are triggering these messages. If you are programmatticaly setting the subscription node with msg.path, make sure it is valid.

Here is the code reference on the error:

* *CtrlxProblemError: DL_INVALID_ADDRESS*: Occurs, if the path given by `msg.path` or within the node configuration is not present (e.g. misspelled).

Further error messages which come directly from the Data Layer are also possible.
As an example, if the `msg.path` does not point to an existent Data Layer node, an error "*CtrlxProblemError: DL_INVALID_ADDRESS*" is emitted.

[AndreasL]

They are using node-red_1.18.0+3.0.2 and 1.20 in the core (not sure about the release).

They have two sheets with an subscrbtion on each, and the values are changed every 1minutes. I did not know that it is possible to retrigger a subscribtion in Node-RED?

I think this item in the log is added each time it works:

info	2023-08-29T08:14:31.876093+02:00				2023/08/29 08:14:31 GET /api/v1/status GetStatus 17.18µs
info	2023-08-29T08:14:31.832164+02:00				2023/08/29 08:14:31 GET /api/v1/status GetStatus 17.16µs
info	2023-08-29T08:13:31.874623+02:00				2023/08/29 08:13:31 GET /api/v1/status GetStatus 16.31µs
info	2023-08-29T08:13:31.832228+02:00				2023/08/29 08:13:31 GET /api/v1/status GetStatus 14.92µs

But the strange thing is that they dont get DL_INVALID_ADDRESS all the time, most of the reads works, but not all of them. And

[MauroRiboniMX]

@AndreasL is that possible that you have left some "trash" behind while programming? and maybe your' looking sometimes at something that doesn't exist? 

[AndreasL]

Then it wouldnt work sporadically, would it? I had an idea that they read/write to pointers, but its just normal variables in a GVL.

[Sgilk]

Yes, the subscriptions could be triggered multiple times. You could add a debug node just before the subscription that writes to the debug window to notify you each time it is triggered. A common cause would be an inject node with repeat interval set.

Are you actually seeing missed reads, or just the DL_INVALID_ADDRESS? I think this is related to the subscription process and not the actual read.

[MauroRiboniMX]

Give me the code 🙂

[Henrik_Bors]

Hello and thanks for your reply!

I am the customer with the DL_INVALID_ADDRESS issue.

I don't understand what you mean by "Yes, the subscriptions can be triggered multiple times. You can add a debug node right before the subscription that writes to the debug window to notify you each time it is triggered."

The subscription node is the first node in the flow. The subscription node has no inputs.

But I changed the cycle time from 1 minute to 1:30 which the PLCt updates the value of the variable that nodered has subscribed to. But we still get the same messages in the log book with the same interval (1 minute) as before. So it's probably like you're saying that Nodered triggers some subscription, but how do I change that setting?

[Sgilk]

This is going to be difficult to troubleshoot without the source flow. If you can pull out anything you are concerned is proprietary and send a simplified flow, I could take a look at it. My suggestion would be to start from a single subscription following the examples found below and add complexity from there. You could also add a "Catch" node to the flow to see if any further information is provided when an error occurs.

Simple Subscription Example 

[Henrik_Bors]

Its quite difficult for me to do extreme changes to the code because its live.

But i did some testing. 

Test1
Added the "Simple subcription Example" that you listed above. Everything seams fine with that. I dont get any errors i the logbook on that subscription node. I used the example and subscribed to CPU usage. Got updates on the CPU usage as designed.

Added test2
Inserted a Subscription node and a debug node, like the example above. But subscribed to a PLC variable.
No i got 3 errormessenges in the loggbook. 3subnodes and 3errors.

I attach full logbook from today.

It feels like the problem isnt between nodred and datalayer, more like datalayer doesnt get access to PLC vars?

Just a sidenote.
Accually I also have problems with OPC-UA subscriptions from Winstudio-app. Variables stops updating in winstudio. 3,5k vars...

Again, thanks for the help

[Sgilk]

Are the variables defined in a CODESYS symbol configuration file in the PLC?

Can you view these variables in the datalayer browser of the ctrlX WebUI?

Can you give me an example of the subscription path to your PLC variables?

Another possibility is https session expiration, although it seems your errors are only ~20 mins apart and I'm not seeing that exact message in the logs.

Subscription node emits an Server closed connection error every x hours

This happens, when the authorization token of the https session expires. A new connection with a new token will be established within a few seconds. If you do not want this to happen, then change the session timeout in the session policies. You can find this in the ctrlX CORE settings under "Settings" -> "Users & Permissions" -> "Session policies".

[Henrik_Bors]

Are the variables defined in a CODESYS symbol configuration file in the PLC?
Yes by pragma. {attribute 'symbol' := 'readwrite'}. The variables is VAR_OUTPUT from FB. In the Symbolfile i can see the fb.

Can you view these variables in the datalayer browser of the ctrlX WebUI?
Yes, all 3

Here is the copied path in DL: plc/app/Application/sym/GVL_MachineConfiguration/Manager/Jeeves/Report/TransitionID

Can you give me an example of the subscription path to your PLC variables?
"plc/app/Application/sym/GVL_MachineConfiguration/Manager/Jeeves/NodeRed_Read_State"
"plc/app/Application/sym/GVL_MachineConfiguration/Manager/Jeeves/Report/TransitionID"
"plc/app/Application/sym/GVL_MachineConfiguration/PalletIDCounter"
These three throws dl_timout

This one works like a charm "framework/metrics/system/cpu-utilisation-percent"

My session policy

[Sgilk]

In the logs I see the first error to come through is actually on a datalayer request node. How is this configured?

info;2023-09-12T13:07:15.487048+02:00;;;;12 Sep 13:07:15 - [error] [ctrlx-datalayer-request:ReadStatus] Error: socket hang up;;;snap.ctrlx-node-red.node-red.service;

[Henrik_Bors]

Here is a log after a Core reboot. So we accually know which error is the "First error".

[MauroRiboniMX · Accepted solution]

Hello,

After some trial the error was due to the fact that x3 was not enought for the application and the timeout was related to that. Solved passing to an X7. 

Mauro