HMI freezes after Update

[rexxon]

Hello everyone,

I wanted to raise some issues that I have been experiencing with the WebIQ software on the ctrlX after I updated from version 2.11.3 to 2.13. Specifically, I have the following problems:

1. The HMI is freezing and becoming unresponsive and inputting data on the HMI is no longer possible.
2. The timing of this issue cannot be determined as it occurs unpredictably.
3. Additionally, the WebIQ Runtime Manager cannot be accessed and does not load after the HMI has frozen.
4. Restarting the ctrlX seems to resolve the issue temporarily, but it keeps reoccurring at unpredictable intervals.

Furthermore, I have noticed in the Log (attached to this post) that there seems to be an issue with an incompatible data type (rPEQReadCurrent), which is a normal REAL data type that I have deleted and re-added to the project multiple times. Other variables of the same data type do not seem to be experiencing this issue. This issuesalso only seems to occur from time to time and not regularly? 

I would greatly appreciate any insights or suggestions on how to resolve these issues. Thank you in advance for your help!

Cheers, Sam

Ps. Is there any other way to start WebIQ Projects after an update, rather from the web-backend in the designer version 2.13?

[webiq-sk]

[2023-03-16 13:23:09.822015] [notification | ctrlX] Attempting to reconnect to 'opc.tcp://ctrlx-CORE:4840'.
[2023-03-16 13:23:09.834762] [ error | ctrlX] Connection error.
[2023-03-16 13:23:09.834762] [ error | ctrlX] service_type(): OpenSecureChannel
[2023-03-16 13:23:09.834762] [ error | ctrlX] error(): BadCommunicationError
[2023-03-16 13:23:09.834762] [ error | ctrlX] client_side_error(): no

This shows the following:
1. WebIQ cannot connect to the ctrlX OPC-UA server on port 4840 and is returned a BadCommunicationError (which is a standard OPC-UA error code).
2. Furthermore the error message shows that this is not a client-side error, i.e. is is caused by the OPC-UA server and not by WebIQ according to the OPC-UA stack - which error is logged in the OPC-UA server log at the exact same time?

I assume by "freezing" you mean that all WebIQ input widgets will be intentionally locked when there is an error with the item - like this one where the OPC-UA connection is not possible or breaks. That is a feature and not a bug, because it prevents you from changing values that would not be able to be transmitted anyway.

The underlying issue here is obviously the OPC-UA connection to the ctrlX OPC-UA server.

Questions:
1. What happens if you reload the URL of the HMI in the browser?
2. Which error exactly are you getting (->screenshot) when you are unable to access the Runtime Manager?

Regarding the other questions: please create separate posts rather then including multiple issues and questions in a single post in the future as this makes it easier for other users to find answers to their questions.

A1:
[2023-03-16 13:07:35.329682] [ error | data-recorder] Exception while attempting to process value.
[2023-03-16 13:07:35.329682] [ error | data-recorder] item(): rPEQReadCurrent
[2023-03-16 13:07:35.329682] [ error | data-recorder] what(): Attempting to record item value with incompatible type.
--> To analyze this we require the HMI and a screenshot of UA Expert showing all details of that item in the PLC

A2: Starting projects
You can start and stop projects also through the Runtime Manager since WebIQ 2.13

[rexxon]

Hey, many thanks for the quick response! You are right, by "freezing" I mean, that the HMI Items are kinda locked. No Iputs are accepted and Outputs are not updated. I will focus on the main question in the following:

"1. What happens if you reload the URL of the HMI in the browser?"

-> The URL of the HMI is not available and the browser keeps loading. This happens on the HMI PC as well as on another PC.

"2. Which error exactly are you getting (->screenshot) when you are unable to access the Runtime Manager?"

-> Also the webpage of the Runtime Manager just keeps loading. The rest of the ctrlX works just fine. I can even control the inputs through the PLC Engineering Tool.

I added the second questions regarding the REAL variable, because I thought, this might be connected to the freezing issue.

Do you have any idea, how to fix this issue?

Cheers, Sam

[webiq-sk]

Thank you for answering the questions. Here are a few more:

1. Are you using the PLC Engineering Tool from the same system where the browser is running on which shows the error?

2. What is the URL of the HMI and of the Runtime Manager you are using?

3. Which error is logged in the OPC-UA server log when the connection issue occurs? (asked before)

4. Does the issue also occur if you temporarily disable a firewall and antivirus software on the system you are accessing it from?

[rexxon]

Hey again,

"1. Are you using the PLC Engineering Tool from the same system where the browser is running on which shows the error?"

-> Usually not. The PLC Engineering Tool runs on a Windows-PC and the HMI runs on a Unix system. After the freezing, I tried to connect to the HMI with both of them and also a Android Smartphone, butneither of them worked. 

"2. What is the URL of the HMI and of the Runtime Manager you are using?"

HMI: [ctrlX-URL]:10123/projekt_name/ and Runtime Manager: [ctrlX-URL]:10123/.webui/

"3. Which error is logged in the OPC-UA server log when the connection issue occurs? (asked before)"

So far, I only get the following warning: "OPC UA Server warning User has not the required permission OPC UA StatusCode: 0x80210000". But after this warning is shown, the HMI still works fine. Today I rediscovered the issue after about 1,5 h after starting the ctrlX and HMI. During that time, I’ve updated the PLC program, successfully checked a few times if the HMI is still running but suddenly it stopped accepting inputs. It still updates the items and alarms on the HMI though and shows no new warnings in the ctrlX OPC UA Log. Furthermore, although the items are still updated, I can not reload the HMI URL, images are not loaded after switching tabs on the HMI, the Designer cannot connect to the server and the WEBUI also timeouts while loading.

"4. Does the issue also occur if you temporarily disable a firewall and antivirus software on the system you are accessing it from?"

Unfortunately, not.

[webiq-sk]

1. Can you please specify exactly whether you can connect with the PLC engineering tool to ctrlX at the same moment you cannot connect to WebIQ anymore from an Android phone?
Also, are you using a wired connection from the engineering PC to ctrlX, but a WiFi connection to your smartphone? Might a breaking WiFi connection simply be the reason?

2. Can you please specify the full URLs?

3. Do you see any error on the browser console (F12) when the issue with updating items occurs before you try to reload the HMI and it fails?

4. If I understand you correctly the issue does NOT occur when you disable all firewall and antivirus software on your system?

[rexxon]

"1. Can you please specify exactly whether you can connect with the PLC engineering tool to ctrlX at the same moment you cannot connect to WebIQ anymore from an Android phone?
Also, are you using a wired connection from the engineering PC to ctrlX, but a WiFi connection to your smartphone? Might a breaking WiFi connection simply be the reason?"

 I can connect to PLC Engineering Tool while I can’t access (or rather reloading) the HMI or the WebIQ Runtime Manager. Connecting to the ctrlX Web-Backend while I can’t access the HMI or the WebIQ Runtime Manager is also possible. I am using a wireless connection for the PLC Engineering Tool, wired connection on the HMI PC and of course wireless with the smartphone. All of these devices can’t (re)load the HMI or change any variables while its frozen.

"2. Can you please specify the full URLs?"

 HMI: http://192.168.177.100:10123/peq_test/ and Runtime Manager: http://192.168.177.100:10123/.webui/

"3. Do you see any error on the browser console (F12) when the issue with updating items occurs before you try to reload the HMI and it fails?"

Attached you can find the console log, after the HMI became unresponsive. I updated the HMI around 9:30 this morning and checked every hour without updating the HMI or PLC. It froze again and if you look up the timestamps in the console log, it might have happened around 13:15? Nothing gets added to the log, if I try to update an item after the HMI is frozen.

"4. If I understand you correctly the issue does NOT occur when you disable all firewall and antivirus software on your system?"

Please excuse me if I did not express myself correctly earlier. Deactivating the firewall does not change the fact that the variables cannot be changed after the HMI is frozen.

[webiq-sk]

Thank you for providing the information. You initially wrote this: "Restarting the ctrlX seems to resolve the issue temporarily, but it keeps reoccurring at unpredictable intervals."

I can't help you with ctrlX management, but I assume there is some kind of system and process manager which allows you to see which processes are running and if there is enough space on the disk. Can you please observe the system when the error occurs if you see any issues, RAM and CPU usage there?
-> #1 does the RAM, disk (free space and usage) and CPU load increase during the issue?

The issue with the "hanging browser" when trying to access the HMI when the issue occurs is not known to us, but it can be caused by proxy servers or other security software which intercepts, forwards and manipulates requests. Is there any proxy on your network? That might even be a transparent proxy.
-> #2 is there a proxy on your network?

Another question: does it happen with a single HMI only? Does the error also occur with simply HMIs like our demo HMI? That would be very interesting to know to narrow down on the issue.
-> #3 Does it happen with a very simple HMI that does not use OPC-UA as well?
-> #4 If it does not happen with a very simple HMI does it happen with a simple HMI that uses a few items from the same OPC-UA server?