Telegraf - Provide timestamp as input to InfluxDB

[star_board]

I am using Telegraf to send a measurement ("Test1") to InfluxDB. Currently, I have two fields read from DataLayer nodes: "Value" & "Timestamp".

Configuration:

[[inputs.ctrlx_datalayer.subscription]]
measurement="Test1"

[inputs.ctrlx_datalayer.subscription.tags]
SensorNum="Sensor1"

[[inputs.ctrlx_datalayer.subscription.nodes]]
name ="Value"
address="plc/app/Application/sym/GVL/Sensor1/Value"

[[inputs.ctrlx_datalayer.subscription.nodes]]
name ="Timestamp"
address="plc/app/Application/sym/GVL/Sensor1/PLCTimeStamp"

Example of the DataLayer nodes:

"plc/app/Application/sym/GVL/Sensor1/Value" : 20
"plc/app/Application/sym/GVL/Sensor1/PLCTimeStamp" : 1720520212398

The PLC variable "PLCTimeStamp" is a timestamp in milliseconds. Instead of providing it as a field, I wanted to provide it as an InfluxDB timestamp. Is this possible using Telegraf? 

Thanks in advance 🙂 

[nickH]

Hi @star_board 

in Telegraf you can either use aggregators an processors to transform your data. See the Telegraf manual for more information: Transform data with aggregator and processor plugins

I have not tested for your specific usecase but I found three plugins which could maybe help you: 

• Starlark processor plugin: here you can implement your functionality with custom starlark (dialect of Python) scripts. 
• Converter processor plugin: is used to change the type of tag or field values.
• Timestamp processor plugin: Generate a metric timestamp based on values in a metric field.

Best regards, 

Nick

[star_board]

Is there anyway where I can save/see the ouput metrics of telegraf (besides influxdb) in order to test this different plugins?

Would something using an ouput plugin work on ctrlX, like File Ouput Plugin ?
[[outputs.file]]
   files = ["stdout", "/tmp/metrics.out"]
   data_format = "influx"

[Sgilk · Accepted solution]

Hi @star_board ,

Something like this should work for your use case, utilizing the Starlark processor @nickH mentioned.

[[outputs.influxdb_v2]]
  urls = ["https://127.0.0.1/influxdb"]
  token = "eaN73XZ4kWNzTwqkzWrhJpcH2ZJsZjkzuRwOPl-fe0cNMrMrg-2yRoJQvMYpnMVzBxLa8GN6EcXx42JEnEU2w="
  organization = "boschrexroth"
  bucket = "boschrexroth"
  insecure_skip_verify = true

## Configure the input plugin with the username and password to ctrlX Core  
[[inputs.ctrlx_datalayer]]
   server = "localhost"

   ## Authentication credentials
   username = "boschrexroth"
   password = "boschrexroth"

   ## Use TLS but skip chain & host verification
   insecure_skip_verify = true

   [[inputs.ctrlx_datalayer.subscription]]
      measurement = "metrics"
      nodes=[
      		{name="cpu_usage_percent", address="framework/metrics/system/cpu-utilisation-percent"},
        	{name ="timestamp",address="system/info/time"}
      ]

      ## The switch "output_json_string" enables output of the measurement as json. 
      ## That way it can be used in in a subsequent processor plugin, e.g. "Starlark Processor Plugin".
	output_json_string = false
      
[[processors.starlark]]
	source = '''
load("logging.star","log")
def apply(metric):
	log.error("----------------------")
	log.error(metric.tags['node'])
	if "timestamp" in metric.fields:
		log.error("Timestamp old: " + str(metric.time))
		metric.time = metric.fields['timestamp']
		log.error("Timestamp new: " + str(metric.time))
	return metric
'''

In the starlark processor, you would look for your PLCTimeStamp field and set the metric.time to that value. Here I used the time coming from the ctrlX system.

[Sgilk]

@star_board ,

Yes, the File Output Plugin is supported. You can view a full list of supported plugins on the Github.

[star_board]

If I use the File Output Plugin where can I save the output file? And how can I access it?

[Sgilk]

You can write to the active configuration. USB access is currently not supported, but I requested this for a future release.

Example:

# Send telegraf metrics to file(s)
[[outputs.file]]
  ## Files to write to, "stdout" is a specially handled file.
  files = ["/var/snap/rexroth-solutions/common/solutions/activeConfiguration/telegraf/test.out"]

  data_format = "influx"

You can then access the file in the active configuration Web UI or via WebDAV.

[star_board · Accepted solution]

Thanks for the help! I did manage to do it using your example with starlark processor! 😁

I had to slightly modify my approach and store the variables "Value" and "PLCTimeStamp" in a DataLayer array, rather than in separate nodes, to retrieve them in the same metric instance:

[[processors.starlark]]  
  source = '''  
def apply(metric):     
    metric.time = int(metric.fields["PLCTimestamp"])   
    return metric 
        '''

Additionally, I experimented with using the Converter processor. I will leave the code here since it might be usefull to another person:

[[processors.converter]]
  [processors.converter.fields]
    timestamp = ["PLCTimestamp"]
    timestamp_format = "unix_ms"