How to use persistent data in c++

[BennyF]

In my c++ application, I have some data (array of uint8) that I need to store in the retain memory (NVRAM). This data is updated every program cycle (every 5..10 ms).

On power-off and restart of the ctrlX, the data should be read from the retain memory.

Currently, I'm not sure how to do this properly. I assume that I have to use the data layer for this. But do I just create a node for it as a data layer client or do I have to use RT access to the data layer? None of this worked yet in my test program.

Is there a code example somewhere?

[Sgilk]

Hi @BennyF ,

You can integrate your application with the solutions storage. See the SDK documentation on data persistence here.

You do not need to create any nodes or support RT datalayer access.

[CodeShepherd]

For the file persistence we even have a how to "SDK - Empower your app with Data Persistence".

If you want your data nodes to be persistent on the data layer you have to add them as data type shared retain. See documentation. If nodes did not exist when registering, you will get a pointer to a new address, if there are already data retained you get a pointer to their address. You can have a look to the ctrlX AUTOMATION SDK example "ctrlX Data Layer Shared Memory" that can be found on GitHub.

[BennyF]

I've read the articles you proposed. To the approach with the data persistence, I have doubts in my case. This seems to be more a storage for configuration files. Updating a file every 5 ms seems to me not to be a good idea as it might harm the hard disk on long term. This is why I want to use the NVRAM of the controller and not the file system.

As far as I understood, the way to use the NVRAM is to use the data layer with RT access. I've already tried the SDK example for shared memory and copied the relevant code to my project. This worked as long as I used the non-retain shared memory. As soon as I switched to shared retain memory, I got the error message "DL_CLIENT_NOT_CONNECTED".

In the meantime, I've found out that for some reason the shared retain memory doesn't work if you run the program in the app build environment, using the COREvirtual datalayer. If you use the datalayer broker of the app build environment, it seems to work. At least, it doesn't generate any error messages. But you don't see the data layer in your browser.

If you create a snap and let it run in the COREvirtual, it works as well. So especially the retain memory seems to have the restriction that it can only be used from the own controller.

In the end, it was a lot of trial and error, because of using the app build environment in combination with the COREvirtual. But finally, using just the COREvirtual, it worked.

[Sgilk]

Hi @BennyF ,

You are right about the datalayer broker and provider being required to run in the same environment. See this note from the documentation.

Because shared memory is used, the ctrlX Data Layer (and it's broker) must run on the same environment. Accordingly, the provider connection string must always be DL_IPC (ipc://).

 Anyway, it sounds like you've got it figured out?

[BennyF]

Hi @Sgilk ,

principally, I figured it out. But it doesn't always work. In my application, I:

• start the data layer
• create a provider (datalayer.factory()->createProvider(DL_IPC))
• start the provider (provider->start())
• register the node in the data layer (provider->registerNode(_baseAddress + "**", this))
• create/reconnect to the shared retain memory (datalayer.factory()->createMemorySync(owner, ...))

But sometimes, especially on restart of the COREvirtual, I get a DL_CLIENT_NOT_CONNECTED message on starting the provider. As consequence, I have no access to my retain memory.

Do I have to wait for something, before I start the provider?

[nickH]

Hi @BennyF 

It could be the case, that your app is started faster then the data layer. I would recommend to retry the connection to the data layer with your provider after some time. 

Best regards, 

Nick

[BennyF]

Hi @nickH ,

Yes, it seems that my app starts faster than the data layer. I tested a long delay (30 s) on start of the app. That worked. But in the end, I followed your proposal and implemented retries after 5 seconds, if starting fails.

Doing some testing, the retries are not necessary on restart of the app (as the datalayer keeps running). On restart of the CORE or COREvirtual, one retry is necessary. So, that part is fine.

But, the SharedRetain memory is only working in the COREvirtual. In the real CORE, it doesn't work. I can start the provider, but on createMemorySync, I always get the result DL_INVALID_HANDLE.

The line of code is:

result = dataLayer.factory()->createMemorySync(nvram, CtrlXDataLayerRoot + NonVolatileMemoryPath, provider, size, MemoryType_SharedRetain);

This line I use after starting the provider, no matter whether the app starts the first time and the shared retain memory has to be created or the app is just restarted and the memory should be there, already.

What can be the reason for the DL_INVALID_HANDLE fault in that case?

[nickH]

Hi BennyF, 

For the SharedRetain type there are some specialties you need to know (see also the manual: here).  

The remanence on the ctrlX CORE is achieved due to the hardware support. A retain memory of 128 kB is available. 8 kB of this memory are used for internal purposes. 120 kB remain for applications. If the control is used with a PLC, 64 kB of this memory is used for PLC retain data. The memory is emulated for a ctrlX CORE Virtual. The retain memory is only saved from time to time.

Could you check in the Data Layer at datalayer/retain/stats if there is enough space for your SharedRetain memory range available on the ctrlX CORE? 

Another hint: "DL_INVALID_HANDLE" indicates a invalid handle argument or NULL pointer argument. 

[BennyF]

Hi @nickH ,

I've read the manual already. That part is clear. Also, there is enough retain memory available, especially as there are no other apps like PLC running. On starting my app, I can even see my memory in "datalayer/retain/chunks". But, it still generates fault messages and doesn't work (in the real CORE).

I've done some more testing. I've installed the realtime sample on the CORE. This worked. Then, I've tested my app again, but changed "MemoryType_SharedRetain" to "MemoryType_Shared" for creating the memory. This worked as well! So, it must be something about the "Retain" part!

Maybe, it is a license topic? My license looks like this:

Or a snapcraft.yaml topic? I've copied the datalayer part from the realtime sample.

Is there maybe another plug necessary?

[nickH · Accepted solution]

Hi @BennyF 

I was able to reproduce your error and can confirm that this is a bug in the Data Layer. This bug occurs when using a real nvram from a seperate process (outside of the automationcore). We will fix this bug in the ctrlX Data Layer. I will let you know when I have news on the bugfix. 

In the meanwhile you can use a workaround. Its possible to also use the emulated nvram on the hardware ctrlX CORE. This setting can be changed in the Data Layer at "datalayer/server/settings". And will be active after a reboot of the control. 

Best regards, 

Nick

[CodeShepherd]

For the license topic, please beware that a custom app would need to claim this license if the web UI should correctly show that license is used.

[BennyF]

Hi @nickH ,

seems, we've finally found the issue. However, I think I cannot use the workaround, because I don't have this parameter. And if I try to add this parameter by just typing it, I get the following fault message:

I have Ubuntu Core 20 on my ctrlX CORE. Is this a setting of a later version (Core 22)? And will the bug fix be done only for Core 22?

Best regards,

Benny

[nickH]

Hi Benny, 

the emulated NV-RAM on the hardware is available since version 2 (based on Core22). Is it possible for you to switch to V2.6? 

Version 2.6 will also be the next LTS version. And will be upgraded from early adopter version to LTS version by end of July. 

Currently its not planed to have this bugfix for V1.20.

[BennyF]

Hi @nickH ,

in that case, it is clear that I have to upgrade to Version 2.6. As this is the prototype for a later series product, this is possible for me.

Maybe with the LTS release end of July, the bug is fixed.

Thanks for your support!

Best regards,

Benny