Failure to data-write from Node_Red to the SD-CARD

[Mr_Wonder_Why]

Hi , I am trying to log the data obtained from the CtrlX I/O, to the SD card. Ive tried plugging in and out, and do realize that the data didnt manage to be written in the SD card once plugged it back in. 

Side question: Is it necessary to connect the Lan cable to the XF10 port during the data logging process? 

I have attached my flows in node red in this post. The first part being the API request and response architecture which is used to automatically assign the SD card to be in "mount" status everytime when the SD card is plugged back in, so that the end-user did not have to access to ctrlX interface to manually click on the "mount" option.

The second part is the one thats responsible on data log from the ctrlX I/O to the csv file created in the SD card.

[Mr_Wonder_Why]

I have just tried data log while the XF 10 is plugged in , it works fine except that the data written to the CSV file is not up-to-date with regards to the current time i plugged off the SD card and insert in my window device(refer to the image attached) . It was already 11:11AM but the data in CSV only logs up till 11:06AM only ( this 5 minutes is not consistent, sometimes can reach up to 10/20 mins as well)

P/S: I would still like to know whether this data logging can be done only when XF10 is plugged in? 

[nickH]

Hi @Mr_Wonder_Why 

of course your Node Red flow can run and therefor the data logging can take place, also if you are not logged in to the device. Also it does not matter if you have something connected to the XF10 port or not. 

I see that you are subscribing some PLC-Variables. This means you are only notified when the data changes. Can it be the case that the data does not change over some minutes of time? 

Anther hint: Can you check the date&time settings in the ctrlX CORE? Is the time set correct?

Best regards, 

Nick

[Mr_Wonder_Why]

Hi @nickH ,

By referring to the image attached, The first ctrlx node is a constantly changing INT value due to signal noise  , For the second ctrlx node, I have included a trigger node that continuously sends the boolean signals. From the debug node, i observe that the values from these two nodes are continuously generated . 

I have also checked the time in my ctrlX core, its synchronized.

[nickH]

I would recommend you to insert debug nodes in your flow to see where you receive a new payload and where it does not. So you are able to localise the problem. For example after the subscribe nodes and after the join node and after the csv-node.

[CodeShepherd]

Are you removing the µSD card securely or just unplug it? As the writing on the card with the FAT32 file system on the flash storage will not be done by each and single write but cached and then written as a block. In standard this flush interval is set to 60 seconds in the settings.js file. So longer times seems to be strange.

Please tell us the ctrlX OS- Node-RED app version and the used palettes and its versions. 

[Mr_Wonder_Why]

Hi @CodeShepherd 

Im just unplugging it. Ive also added an API architecture in node red which always sets the SD card to be Mount whenever its plugged back in( verified via observation on the CtrlX interface) . However,i notice that the data written to the csv doesnt achieve a desirable result, as mentioned on the initial post,  "the 5 minutes off from current time is not consistent, sometimes can reach up to 10/20 mins as well)".

On the other hand, Ive also tried doing using a manual trigger(button). Via API, i will trigger the action to be either "Mount" or "Unmount" base on my button(true/false). For this ive also verified on the ctrlx interface in the storage tab and its working well! However, when i  test without plugging in the XF10, the data didnt even write to the csv file.

To conclude, I think that we will need to send a command to "mount" / "unmount" the SD card after inserting and before removing it, with this procedure, i can achieve my data desirably. However comes another issue, this only works when XF10 is plugged in,  data will not get updated to the SD card once XF10 is plugged off.

I appreciate if there is any alternative or a better approach on implementing this data logging architecture.Thanks.

For the node-red part, the version is V3.0.2 

For specific node such as ctrlX-automation node, I updated to V1.9.6 but still getting "Server Closed Connection" after some time(its pops up less frequent as compared to the version before). 

[CodeShepherd · Accepted solution]

In your screenshot you are addressing the storage API via the IP address, I guess of XF10 port (192.168.2.246). If you now unplug your network cable the port and its IP will get inactive. You have to use localhost or corresponding IP 127.0.0.1 to use internal ethernat communication.

This could generate the same issue with gathering and storing your data. Please check out the examples provided via our palettenode-red-contrib-ctrlx-automation.

[Mr_Wonder_Why]

Hi @CodeShepherd , aside of this there is another issue, which is the data loss for the latest few minutes(the Data loss duration is not consistent)      after plugging out the SD card. 

If i manually trigger "Remove Safely" feature in the CtrlX before plugging out the SD card, the data saved is up-to-date. 

However, is there any other solution such that the end-users can just plug and unplug the SD card without external triggers( "Safely Remove" feature)? yet retaining the value up-to-date.

[CodeShepherd · Accepted solution]

Last post is a duplicate of "Partial data loss while writing to a SD card" and will be further discussed there. In short it is technology dependent as data writing on flash drives are not done immediately to save storage lifetime.