Recipe edit, error when fetching item datas

[VincentDruart]

Hello,
I have a problem with recipe-edit widget, as soon as i linked it into the recipe controler, in WebIQ Deisgner, i have an error : "Error when fetching item datas".

I have the same one with live supervision, each time i go on the page. And the recipe edit doesn''t work. Tried many differents recipes. Even the demo has the same problem.

WebIQ Server version : 2.15.6 release /2.15.6 (58227027)
WebIQ Deisgner Version : 2.15.4 v2.15.4 (64cab7b2)

[webiq-sk]

This indicates that the items referenced in your recipe do (no longer) exist in WebIQ - did you delete any of them?

[VincentDruart]

I tried even without items, only meta data, it's the same. With new items it's the same error too.
I have the same error just by opening the web iq recipe demo.

[webiq-sk]

Please provide designer.log and connect.log for analysis

[VincentDruart]

Log files in attachment.

[webiq-sk]

Looks to me like something's interfering the network connectivity (bad switch, local firewall or antivirus software on PC, firewall or router on network) - might that be the case?

[2024-02-20 09:48:07.886954] [ error | http] write failed!
[2024-02-20 09:48:07.886954] [ error | http] error_code(): system:9:Bad file descriptor

[2024-02-20 08:47:10.371177] [ error | http] write failed!
[2024-02-20 08:47:10.371177] [ error | http] error_code(): system:32:Broken pipe
[2024-02-20 08:47:11.162314] [ error | http] write failed!
[2024-02-20 08:47:11.162314] [ error | http] error_code(): system:32:Broken pipe

[2024-02-19T06:41:28.443Z] [error] Error retrieving file: socket hang up Error: socket hang up
[2024-02-19T06:41:43.453Z] [error] Error retrieving file: socket hang up Error: socket hang up
[2024-02-19T06:41:58.529Z] [error] Error retrieving file: socket hang up Error: socket hang up

[2024-02-20T07:35:43.229Z] [error] Error retrieving file: socket hang up Error: socket hang up
[2024-02-20T07:35:43.612Z] [error] Error retrieving file: socket hang up Error: socket hang up
[2024-02-20T07:35:44.076Z] [error] Error retrieving file: socket hang up Error: socket hang up

[2024-02-20T07:38:41.663Z] [error] Error retrieving file: socket hang up Error: socket hang up
[2024-02-20T08:47:30.891Z] [error] Error retrieving file: socket hang up Error: socket hang up
[2024-02-20T08:47:31.447Z] [error] Error retrieving file: socket hang up Error: socket hang up
[2024-02-20T08:47:31.682Z] [error] Error retrieving file: socket hang up Error: socket hang up

There are also items used in widgets that do not exist in Process Data Manager:
[2024-02-20 08:30:05.417706] [ error | io_server] Failed to load item from database.
[2024-02-20 08:30:05.417706] [ error | io_server] item_alias(): "COCTRL1.Sensors_02A._02A_TT_01.SupervisionDatas.Values.TotalizerValue"
[2024-02-20 08:30:05.417706] [ error | io_server] error_code(): shmi:connect:io_server:4:item not found
[2024-02-20 08:30:05.417824] [ error | io_server] Failed to load item from database.
[2024-02-20 08:30:05.417824] [ error | io_server] item_alias(): "COCTRL1.Sensors_02A._02A_TT_01.SupervisionDatas.Values.FlowValue"
[2024-02-20 08:30:05.417824] [ error | io_server] error_code(): shmi:connect:io_server:4:item not found
[2024-02-20 08:30:05.417936] [ error | io_server] Failed to load item from database.
[2024-02-20 08:30:05.417936] [ error | io_server] item_alias(): "COCTRL1.Sensors_02A._02A_TT_01.SupervisionDatas.Values.TemperatureUnit"
[2024-02-20 08:30:05.417936] [ error | io_server] error_code(): shmi:connect:io_server:4:item not found
[2024-02-20 08:30:05.418073] [ error | io_server] Failed to load item from database.
[2024-02-20 08:30:05.418073] [ error | io_server] item_alias(): "COCTRL1.Sensors_02A._02A_TT_01.SupervisionDatas.Values.FlowUnit"
[2024-02-20 08:30:05.418073] [ error | io_server] error_code(): shmi:connect:io_server:4:item not found
[2024-02-20 08:30:05.418154] [ error | io_server] Failed to load item from database.
[2024-02-20 08:30:05.418154] [ error | io_server] item_alias(): "COCTRL1.Sensors_02A._02A_TT_01.SupervisionDatas.Values.TemperatureValue"
[2024-02-20 08:30:05.418154] [ error | io_server] error_code(): shmi:connect:io_server:4:item not found

And there are items that exist in Process Data Manager, but not on the OPC UA server:
[2024-02-20 07:41:58.256035] [ error | ] Failed to read item.
[2024-02-20 07:41:58.256035] [ error | ] item_alias(): COCTRL1.GV_Project.Project_Recipe.EnzymeQuantity
[2024-02-20 07:41:58.256035] [ error | ] plc_identifier(): plc/app/Application/sym/GV_Project/Project_Recipe/EnzymeQuantity;ns=2
[2024-02-20 07:41:58.256035] [ error | ] error_code(): opcua:-2144075776:BadNodeIdUnknown
[2024-02-20 07:41:58.256100] [ error | ] Failed to read item.
[2024-02-20 07:41:58.256100] [ error | ] item_alias(): COCTRL1.GV_Project.Project_Recipe.HotWaterQuantity
[2024-02-20 07:41:58.256100] [ error | ] plc_identifier(): plc/app/Application/sym/GV_Project/Project_Recipe/HotWaterQuantity;ns=2
[2024-02-20 07:41:58.256100] [ error | ] error_code(): opcua:-2144075776:BadNodeIdUnknown
[2024-02-20 07:41:58.256126] [ error | ] Failed to read item.
[2024-02-20 07:41:58.256126] [ error | ] item_alias(): COCTRL1.GV_Project.Project_Recipe.MusselQuantity
[2024-02-20 07:41:58.256126] [ error | ] plc_identifier(): plc/app/Application/sym/GV_Project/Project_Recipe/MusselQuantity;ns=2
[2024-02-20 07:41:58.256126] [ error | ] error_code(): opcua:-2144075776:BadNodeIdUnknown

However, if the issue already occurs on an unchanged recipe demo you downloaded from inside WebIQ Designer it looks like something is interfering with the network connection.

[VincentDruart]

I actually have a cisco L3 switch between my pc and the CtrlX yes. But there is no filter / firewall for the moment since i'm in lab mode.
items used in widgets that do not exist : Yes, for testing, but they are on Layout only
And there are items that exist in Process Data Manager, but not on the OPC UA server: i tried tu remove all my items for recipe, these 3 are parts of for tests but even without them it was'nt working.

I just downloaded a new recipe demo to be sure and it's the case, not modified and bugged.

Nothing to see on Cisco logs for these ips. Are you sure it can be that ? because even on designer mode and with the demo which don't have plc connected it should not be a network issue.

[webiq-sk]

If you create a support ticket on https://www.smart-hmi.com/user/support I can issue you a trial license for WebIQ Designer which you can use for testing the recipe demo locally, i.e. when not connected to ctrlX through the network, but only accessing the local WebIQ Server instance on your system.

That way you could find out whether the network is involved at all.

[VincentDruart]

Support asking Done.

I found this inside an other connect.log :

[2024-02-19 06:33:03.185283] BEGIN OF LOG
[2024-02-19 06:33:03.185522] [notification] Copyright Smart HMI GmbH 2013-2024 WebIQ Server v2.15.6 release/2.15.6/58227027 (Win32 x86_64 Release)
[2024-02-19 06:33:03.185694] [ information] Started: 2024-02-19 06:33:03
[2024-02-19 06:33:03.185694] [ information] Current clients: 0
[2024-02-19 06:33:03.710893] [ information | init] Skipping autostart projects because no system users were found.
[2024-02-19 06:33:03.711302] [notification | security] You don't have system users configured! WebIQ Server is in initialization mode!
[2024-02-19 06:33:03.711302] [notification | security] Please open "WebIQ Runtime Manager" to create a system user _now_.
[2024-02-19 06:33:03.711302] [notification | security] The Runtime Manager can usually be accessed under http://localhost:10123/
[2024-02-19 06:33:08.382367] [ error | license] Failed to connect to license agent.
[2024-02-19 06:33:08.382367] [ error | license] what(): Aucune connexion n’a pu être établie car l’ordinateur cible l’a expressément refusée [system:10061 at D:\deps\include\boost/asio/detail/win_iocp_socket_connect_op.hpp:105:5 in function 'do_complete']
[2024-02-19 14:53:32.436049] [ error | license] Connection to license agent lost.
[2024-02-19 14:53:32.436049] [ error | license] what(): Une connexion existante a dû être fermée par l’hôte distant [system:10054 at D:\deps\include\boost/asio/detail/win_iocp_socket_recv_op.hpp:90:5 in function 'do_complete']
[2024-02-19 14:53:32.438071] [notification | license] License claim has been invalidated. Unloading projects and disconnecting all clients...
[2024-02-19 14:53:33.886456] END OF LOG

Is this log talking about an issue on my computer with firewall maybe ? What are the ports WebIQ Designer using ? 10054 & 10061 are parts of them ?

[webiq-sk]

WebIQ Server only uses port 10123 by default, WebIQ Designer uses port 10124 for browser preview.

In the support ticket I asked you to temporarily disable all local firewall and antivirus software on your system - di you do this prior to this issue?

Please note that WebIQ is a networked system and there is no difference between a Runtime and a Designer system as WebIQ Designer also connects to WebIQ Server - albeit on the same local system - this means that some software can still interfere with the connectivity causing issues.

Therefore you have to disable all firewall and antivirus software on your local system temporarily to find out if the issue occurs then. If it doesn't the firewall/antivirus software is the reason and that's where you have to look for any fixes.

[VincentDruart]

Ok, so i receive the trial license.
The demo-recipe-2.14 work on my computer wihtout any error.
I will try without firewall or antivirus but it will take a little time because i need to do it with IT department.
I will post results here when done.

[HmiGuide]

Since WebIQ 2.14 the reverse proxy of ctrlX is supported, too. This means the following:

• https instead of http
• https default port 443 instead of 10123
• Different URL to load project (https://192.168.1.1/webiq/<project_name>/ instead of http://192.168.1.1:10123/<project_name>/)
  • Start Runtime Manager in ctrlX WebUI
  • Provide system user and password to login
  • "Start" and "Open" the project which automatically uses the ctrlX proxy URL

Maybe this option is working for you, because it uses the standard https port and not webIQ specific port 10123.

For more info on the reverse proxy and when you need it: see HowTo-view-content-from-multiple-web-servers-in-one-web-page 

[webiq-sk]

I do not think it is related to the port because technically the port is only a number, what goes through the port is regardless of the port.

[VincentDruart]

I'm back from the tests with IT guy.
Firewall and antivirus disabled on my computer. Cisco removed from the IT network, my computer is straitgh to the ENG port on CtrlX. No firewall on CtrlX.
Still have the problem.

[webiq-sk]

Please do the following:
1. Download and unpack Process Monitor: https://docs.microsoft.com/en-us/sysinternals/downloads/procmon
2. Quit and exit all other programs
3. Start Procmon64.exe
4. Click on this button to disable capture:

5. Click on Options -> Profiling Events -> Enable “Generate thread profiling events” and “every second”
6. Click on the trash can icon to delete the log
7. Click on the capture button (the one you disabled before, see screenshot above)
8. Start WebIQ Designer and reproduce the issue
9. Stop capture by clicking on the button again
10. File -> Save (use default options), and change the target path before clicking oh the OK button
11. ZIP that file and send it to me via PM

[webiq-sk]

Thank you for providing the log file.

I asked you previously to not connect to your ctrlX, but to use WebIQ locally on your system with the license I provided so WebIQ Designer would only connect to your local WebIQ Server instance. Instead, it looks like you still tested the connection directly to ctrlX which doesn't help here.

Can you please test it again on your local system with the default WebIQ Designer shortcut and not the one connecting directly to ctrlX? The goal here is to activate the license on your local system, download the recipe HMI on your local system, start the recipe HMI on your local system and see if the error occurs.

If it does not occur the issue can be found somewhere on the network.

Interestingly, the log you provided does not show any connections to 192.... IP addresses, but instead to internet IP addresses owned by Microsoft - is there a transparent proxy used in your network/on your system that routes everything to the internet somehow?

Also, did you ensure to temporarily disable both HP ProtectTools Security Manager Software and F-Secure PSB Computer Protection on your system?

In any case, we need to have a test on your local system without any connection to ctrlX first.

[VincentDruart]

As i reply before, the local test is working, i have no error so i don't understand why you want log about it.
20.1.xxx.xxx is the local network for the CtrlX process.
HP ProtectTools Security Manager Software and F-Secure PSB Computer Protection were disabled during test.

[webiq-sk]

Sorry, I somehow missed that it works on the local system.

Please note that the 20.* IP range must not be used for internal networks as it is reserved for internet use - there are only certain valid ranges available for private networks (~18 million): https://en.wikipedia.org/wiki/Private_network#Private_IPv4_addresses - on the WebIQ side this should not cause any issue however, because it's just a different destination. WebIQ doesn't care about the specific IP address.

Can you please reproduce the test and record with Wireshark and provide a dump file as PM?

[VincentDruart]

I know but it's a temp range ip for now, final range will be in 10.*
I'm in holiday now, i will test more in one week. I will put the final ip range before in case it's a problem and send you a wireshark dump.

[webiq-sk · Accepted solution]

After getting some Wireshark logs it became clear that the reason for the error is that the HMI has already been updated to WebIQ 2.15.7 and is used with WebIQ Server 2.15.6. WebIQ 2.15.7 added a new function for a fix (WEBIQ-4065) that is not available with WebIQ Server 2.15.6 leading to the error.

Solution: update WebIQ on ctrlX to WebIQ 2.15.7. You cannot downgrade HMIs, therefore this is the only solution - if you don't want to restore a version prior to updating it to 2.15.7.

[VincentDruart]

I didn't mention the 2.15.7 to 2.15.6 downgrade because i didn't even remember it and the recipe module wasn't in the project at this moment.
Now that the 2.15.7 is officially out for CtrlX i just updated it and the bug is fixed.

Thanks for help.

[HmiGuide]

So I wonder why it is possible to start the WebIQ 2.15.7 project with an old server 2.15.6. My expectation is, an error message appears, telling that the versions of project and server doesn't match. This should result in the fact, that the program can't be started. 

Is that a special case, or is the version check only done for major versions (e.g. 2.14 to 2.15)?

[webiq-sk]

I assume this happened because when you downgraded on your system to WebIQ 2.15.6 you did not delete the 2.15.7 Visuals package in your %APPDATA% directory. If you then edited the HMI again you used WebIQ Server 2.15.6 with WebIQ Visuals 2.15.7. As Runtime Manager only checks if the server version has changed which wasn't the case here obviously no warning could be shown as this was a non-supported setup then.