dimanche 1 mars 2015

How can I remove a carriage return, add a time stamp and ignore some data from a live MQTT feed


I type mosquitto_sub -d -t +/# from the Ubuntu terminal to access the MQTT stream.


Real output from the live MQTT stream is this:



Sending PINGREQ
Received PINGRESP
Sending PINGREQ
Received PINGRESP
Received PUBLISH (d0, q0, r0, m0, 'm/gf/TMX6BP/075/d/SVlts', ... (28 bytes))
86,1224830,27610 27869 17565
Received PUBLISH (d0, q0, r0, m0, 'm/gf/TMX6BP/075/d/status', ... (39 bytes))
86,1243000,164573,-33.836465,151.051189
Sending PINGREQ
Received PINGRESP
Received PUBLISH (d0, q0, r0, m0, 'm/NWRL/TMX/098/d/SVlts', ... (26 bytes))
806,3040421,7549 7750 3904
Received PUBLISH (d0, q0, r0, m0, 'm/NWRL/TMX/098/d/status', ... (39 bytes))
806,3069000,59666,-33.836465,151.051189
Sending PINGREQ
Received PINGRESP
Sending PINGREQ
Received PINGRESP
Sending PINGREQ
Received PINGRESP
Sending PINGREQ
Received PINGRESP
Received PUBLISH (d0, q0, r0, m0, 'm/NWRL/TMX/098/d/SVlts', ... (26 bytes))
810,5440995,6143 7807 4076
Sending PINGREQ
Received PINGRESP
Received PUBLISH (d0, q0, r0, m0, 'm/NWRL/TMX/098/d/status', ... (39 bytes))
810,5489000,59897,-33.836465,151.051189
Sending PINGREQ
Received PINGRESP


There is no way of predicting when the next PUBLISH will be seen as they are only seen in the stream when the vehicle has transmission/reception from the GSM/3G towers


To filter I add mosquitto_sub -d -t +/# 2> >(grep PUBLISH) this will only allow lines with PUBLISH in it, hence the ouput is:



Received PUBLISH (d0, q0, r0, m0, 'm/gf/TMX6BP/075/d/status', ... (38 bytes))
86,637999,164563,-33.836465,151.051189
Received PUBLISH (d0, q0, r0, m0, 'm/NWRL/TMX/098/d/SVlts', ... (26 bytes))
806,3040421,7549 7750 3904
Received PUBLISH (d0, q0, r0, m0, 'm/NWRL/TMX/098/d/status', ... (39 bytes))
806,3069000,59666,-33.836465,151.051189
Received PUBLISH (d0, q0, r0, m0, 'm/gf/TMX6BP/075/d/SVlts', ... (28 bytes))
86,1224830,27610 27869 17565
Received PUBLISH (d0, q0, r0, m0, 'm/gf/TMX6BP/075/d/status', ... (39 bytes))
86,1243000,164573,-33.836465,151.051189
Received PUBLISH (d0, q0, r0, m0, 'm/NWRL/TMX/098/d/SVlts', ... (26 bytes))
806,3640483,7463 7721 3933
Received PUBLISH (d0, q0, r0, m0, 'm/NWRL/TMX/098/d/status', ... (39 bytes))
806,3674000,59676,-33.836465,151.051189
Received PUBLISH (d0, q0, r0, m0, 'm/NWRL/TMX/098/d/SVlts', ... (26 bytes))
806,4240543,7291 7750 3933
Received PUBLISH (d0, q0, r0, m0, 'm/NWRL/TMX/098/d/status', ... (39 bytes))
806,4279000,59687,-33.836465,151.051189
Received PUBLISH (d0, q0, r0, m0, 'm/gf/MXE/065/d/SVlts', ... (25 bytes))
455,24715,28041 28041 967


How would I be able to eliminate a few of the fields and also add a time stamp everytime I receive something; I 've tried using sed but had no luck. I entered $ mosquitto_sub -d -t +/# 2< <(grep PUBLISH) 2< <(sed "s/^/ date/") , $ mosquitto_sub -d -t +/# 2< <(grep PUBLISH) 2< <(sed "s/^/$date`/")


Q: How can I change my input to the terminal so that the output from the above live feed would be:



'm/gf/TMX6BP/075/d/status', 86,637999,164563,-33.836465,151.051189,[timestamp]
'm/NWRL/TMX/098/d/SVlts', 806,3040421,7549 7750 3904,[timestamp]
'm/NWRL/TMX/098/d/status', 806,3069000,59666,-33.836465,151.051189,[timestamp]
'm/gf/TMX6BP/075/d/SVlts', 86,1224830,27610 27869 17565,[timestamp]
'm/gf/TMX6BP/075/d/status', 86,1243000,164573,-33.836465,151.051189,[timestamp]
'm/NWRL/TMX/098/d/SVlts', 806,3640483,7463 7721 3933,[timestamp]
'm/NWRL/TMX/098/d/status', 806,3674000,59676,-33.836465,151.051189,[timestamp]
'm/NWRL/TMX/098/d/SVlts', 806,4240543,7291 7750 3933,[timestamp]
'm/NWRL/TMX/098/d/status', 806,4279000,59687,-33.836465,151.051189,[timestamp]
'm/gf/MXE/065/d/SVlts', 455,24715,28041 28041 967,[timestamp]


Possible solutions (future referencing):


Using the mosquitto_sub -d -t +/# 2> >(sed -n "s|.*\('.*',\).*|\1|p") | sed "N;s/\n/ /;s/$/ $(date)/" The output is:



0 810,5440995,6143 7807 4076 Wed Feb 25 23:23:51 UTC 2015 810,5489000,59897,-33.836465,151.051189 810,6041055,7606 7693 4076 Wed Feb 25 23:23:51 UTC 2015

Using the mosquitto_sub -d -t +/# 2> >(grep PUBLISH) | sed "N;s/\n/ /;s/$/ $(date)/" command from the terminal the output is



817,3069000,60045,-33.836465,151.051189 609,24570,27553 27553 955 Thu Feb 26 00:06:26 UTC 2015

Using the mosquitto_sub -d -t +/# 2>&1 | sed -n "/PUBLISH/{s|.*\('.*',\).*|\1|;N;s/\n/ /;s/$/ $(date)/;p}" The output is



'm/gf/MX3/122/d/status', 610,33000,28162,-33.836465,151.051189 Thu Feb 26 01:18:17 UTC 2015

Aucun commentaire:

Enregistrer un commentaire