Walker, I have been watching you for some time now. I am a business school teacher across 10+ universities, teaching anything from fintech and supply chain management to management accounting, and I have won many teaching awards in the process. I can say this without reservation. You are one of the most authentic and effective teachers that I have ever seen. I always recommend junior faculty to watch your videos to learn how to find their own authenticity. I appreciate your service to the community!
I am using this clip to pre educate executives before we sit down to dive into their business and have a greater discussion on strategy...thank you Walker for providing these tools
What about pulling historical data? Do I pull it from the source or does the Unified Namespace keep history and if so where? How does buffering work, when say my MES layer was down and need to get the downtime states of the past 3 (say) hours for OEE? Looking forward to listen to you at the MESA Africa conference from South Africa next month!
Great Video, thanks for sharing this knowledge. OPC UA also provides pub-sub way for data communication. Deadband filters can easily be applied there. But sounds like MQTT got a lot more to offer than just data aggregation in UNS. Will love to know more about it
I'm following walker for almost 3.5 years. The unified namespace concepts and architecture are well suited for automotive industry and to some extent in pharma. When it comes to pharma or process industry most digitalization activities is depend on batch records. I believe the UNS architecture can solve problems to some extent. Can you provide more insights on how the dataops can be handled in a better way at pharma. I think there are some scenarios in process industries where we have to choose the right thing whether it should be UNS based or digital thread based. I would love to hear some detailed info from walker on this topic in white board series.
Hey Walker, Great video. So you are using MQTT to bring all the information into your UNS and you are then structuring your data following the ISA 95 Part 2. Let's say you were using MQTT Distributor as your broker - could be any broker I know - then when you are publishing e.g. data from the ERP system. Then you would need to subscribe to that data, merge it into the data structure following ISA 95 Part 2 and then publish that whole structure again right? Since with MQTT Sparkplug B you only have Group ID, Edge Node ID and Device ID, you can't directly translate that to ISA 95 Part 2, which is why you would need to do that intermediate step in order to structure all data within the UNS. Is this correct or am I missing something?
Enlightening that 93% of industrial data changes at one minute intervals or longer. A poll/response OPCUA system is over sampling most of the time. In the case of fast changing signals, OPCUA is under sampling. It only makes sense that the future of industrial measurements is a publish/subscribe system such as MQTT where data is captured only when it changes. Thanks for the great explanation of the UNS!
Depends, what level of communications and what are you doing process control, discrete manufacturing or motion control. A big deal made is protocols like MQTT are COS (change of state only) to save bandwidth. Two issues arise 1) COS doesn’t accommodate a heartbeat (how do you know the wire wasn’t run-over by a tow motor, or the cell tower was struck by lightning?) ,2) when a process upset occurs, typically traffic increases exponentially so much that a network size for normal COS only with retries Quickly crashes. How do you deal with those situations?
Hi walker how uns hand shake will all stacks data from plc scada erp etc? What software could do that? I mean Direct each layer to uns. And what software could take data info from uns for report dashboard bi? Thanks in advance.
Great video! However the dismissal of OPC UA is not accurate or helpful as it ignores OPC UA Pubsub and the strong support that it has in industry. Does anyone know where to find an unbiased technical comparison of OPC Pubsub vs SparkplugB?
Great pictorial explanation! How are data then structured or packaged at each node/ecosystem in order to be transferred to the single source of truth/name space?
Great question. Many ways depending on the data source. At the device level, data may need to be converted and sent through an MQTT gateway (transport format) using SparkplugB (payload format) and then parsed to populate the UNS---maybe just a timestamp, value and unit. The UNS structure both normalizes and contextualizes it for use by other software. Data from other systems, say the ERP, may involve using a business connector that takes the BAPI and converts it into the semantic hierarchy used by the UNS.
I do see a challenge when we have guarantee that orders are created from erp to mes using only mqtt. How do you make sure no data is lost when you don’t know of the application is listening? Do you let these platforms speak spb? Because I’ve seen no erp speak spb. If they don’t speak spb, do you build a microservice to run api calls to fetch this data?
I’m not sure I believe your stat 93% industrial data updated > 1/sec? (especially IO data) it Depends, what level of communications and what are you doing process control, discrete manufacturing or motion control. A big deal made is protocols like MQTT are COS (change of state only) to save bandwidth. Two issues arise 1) COS doesn’t accommodate a heartbeat (how do you know the wire wasn’t run-over by a tow motor, or the cell tower was struck by lightning?) ,2) when a process upset occurs, typically traffic increases exponentially so much that a network size for normal COS only with retries Quickly crashes. How do you deal with those situations?
I stopped watching after 11 minutes, can not give you more time to answer the question of you title. You still did not answer it, it is very frustrating to watch you beating around the bush so much. Is it a design concept? Is it software running somewhere? I have 25 yrs experience in IT specialized in data, software, design concepts (object oriented vs functional etc). Some years ago it was fashion for big companies to look into data brokerage middleware for communication between point solutions, those projects struggled to become succesfull, mostly because of the tough choices to make what you leave up to the point solution and what to the broker. Some benefits/points you mention are not exact, claiming that the UNS has the sst is not true. The truth lies in all connected systems, the UNS only holds ( with least latency as possible) a hot copy of a subset of all the information. I will look at other videos to find more information on what UNS really is.
@@4.0Solutions other videos gave me more information: UNS is realtime datalake where the data objects are ordered in a single namespace hierarchy. Updates/selects are done through a publish subscribe mechanism (like mqqt). Imo the biggest drawback is that companies think they should put everything from their pointsolutions into the UNS, but that is not feasible. Upfront choices should be made which information is needed realtime of the whole chain. Also changing/swapping point solutions is difficult as companies will start defining their uns based on their current solutions and the dataobjects used there. If a new or other solution stores different data for the same functional process, the challenge becomes mapping that to the current uns. The central realtime datalake is benificial for data-driven control, but it has also security challenges. The uns need to be able to communicate with all the network levels of the company including cloud. Normally factories don’t want to connect their factory floor to internet, for obvious reasons. Physically seperated networks where put in place for a reason. Could be mitigated a bit by only allowing publish for the factory sensors. You don’t want a hacker to gain control over the factory…
Nice! Things are very changed since the 3.0 epoch of automation. These were the days that i use to work as automation engineer but now days all are changed. I can recall programs that i use to work with, lets say process control programs from Opto22 and scada software Genesis I dont know if it is still in the automation market in the time beeing. I think that the new Elon Musk robots will be connected to a unified name space and because the Tesla is a Data company as you right said in other videos the robots will use that unified name space to read and write data in order to produce products. Thanks to you I restart to, lets say "play" again with automation, particular with home automation using Raspberry Pi as Mqtt broker and ESP32 and Arduino as clients. I want your advise in that That is, I use to connect lets say a ESP32 microcontroller to an local MQTT broker installed in a local Raspberry Pi and that local MQTT broker connected in and MQTT broker in the web in order to have access to the ESP32 microcontroller from everywhere in the globe. I use a mobile with MQTT client installed, in order to control and view data The problem was that, if somehow the MQTT broker in the web collapes the I loose connection to the ESP32. And then a Idea emerge in my head!!! I will use an VPN program, I will install a VPN in my local Raspberry Pi in order every time I connect my mobile to the VPN my mobile will be part of the local network so i will not need any more the MQTT broker in the web to control the ESP32 microcontroller when I am away from home, I will use only the local MQTT broker. Ofcourse * I use DDNS, * I have opened a Port in my Router Which is your opinion about this approach? What is your advise in that? You keep doing nice job and videos around the industrial automation world Call me "Bill from Greece" if you ever mention my name in future videos!!!! If you ever come to Greece be my Guest!!!!
Walker, I have been watching you for some time now. I am a business school teacher across 10+ universities, teaching anything from fintech and supply chain management to management accounting, and I have won many teaching awards in the process. I can say this without reservation. You are one of the most authentic and effective teachers that I have ever seen. I always recommend junior faculty to watch your videos to learn how to find their own authenticity. I appreciate your service to the community!
Dr. Connor - ty 🙏 I really appreciate that and I am honored to have that feedback.
Blessings,
Walker
I am using this clip to pre educate executives before we sit down to dive into their business and have a greater discussion on strategy...thank you Walker for providing these tools
We’re glad to help❤️
I'm doing a masters in industry 4.0 and your videos are pure gold. Thanks for share your knowledge!
We appreciate the support Iván!
That sounds cool do you have a link of the program?
Cool, which uni? I am also thinking about it.
@@staynjohnson4221 UNIR, it's a college in Spain
Very clear and to the point. The best UNS video so far.
Glad it was helpful!
Great job Walker!!
Killer video! 2k views already! 🔥 definitely worth watching several times.
What about pulling historical data? Do I pull it from the source or does the Unified Namespace keep history and if so where? How does buffering work, when say my MES layer was down and need to get the downtime states of the past 3 (say) hours for OEE?
Looking forward to listen to you at the MESA Africa conference from South Africa next month!
Great Video, thanks for sharing this knowledge.
OPC UA also provides pub-sub way for data communication. Deadband filters can easily be applied there.
But sounds like MQTT got a lot more to offer than just data aggregation in UNS. Will love to know more about it
I'm following walker for almost 3.5 years. The unified namespace concepts and architecture are well suited for automotive industry and to some extent in pharma. When it comes to pharma or process industry most digitalization activities is depend on batch records. I believe the UNS architecture can solve problems to some extent. Can you provide more insights on how the dataops can be handled in a better way at pharma. I think there are some scenarios in process industries where we have to choose the right thing whether it should be UNS based or digital thread based. I would love to hear some detailed info from walker on this topic in white board series.
Good question, I'm looking for a solution using this concept and architecture that can be used in the pharma sector.
thanks you for your explaination. Clean n clear!! love it
Hey Walker,
Great video.
So you are using MQTT to bring all the information into your UNS and you are then structuring your data following the ISA 95 Part 2. Let's say you were using MQTT Distributor as your broker - could be any broker I know - then when you are publishing e.g. data from the ERP system. Then you would need to subscribe to that data, merge it into the data structure following ISA 95 Part 2 and then publish that whole structure again right?
Since with MQTT Sparkplug B you only have Group ID, Edge Node ID and Device ID, you can't directly translate that to ISA 95 Part 2, which is why you would need to do that intermediate step in order to structure all data within the UNS.
Is this correct or am I missing something?
Enlightening that 93% of industrial data changes at one minute intervals or longer. A poll/response OPCUA system is over sampling most of the time. In the case of fast changing signals, OPCUA is under sampling. It only makes sense that the future of industrial measurements is a publish/subscribe system such as MQTT where data is captured only when it changes. Thanks for the great explanation of the UNS!
Depends, what level of communications and what are you doing process control, discrete manufacturing or motion control. A big deal made is protocols like MQTT are COS (change of state only) to save bandwidth. Two issues arise 1) COS doesn’t accommodate a heartbeat (how do you know the wire wasn’t run-over by a tow motor, or the cell tower was struck by lightning?) ,2) when a process upset occurs, typically traffic increases exponentially so much that a network size for normal COS only with retries Quickly crashes. How do you deal with those situations?
Thank you for the clarity.
You are very welcome, we’re glad this was helpful
This has got to be the best video on this junk.
Well he did create the architecture 😉
Funny, how Walker after all this time still looks at the camera with a face that has a bit of a questioning look - "Am I making sense here?"
Hi walker how uns hand shake will all stacks data from plc scada erp etc? What software could do that? I mean Direct each layer to uns. And what software could take data info from uns for report dashboard bi? Thanks in advance.
Great video! However the dismissal of OPC UA is not accurate or helpful as it ignores OPC UA Pubsub and the strong support that it has in industry. Does anyone know where to find an unbiased technical comparison of OPC Pubsub vs SparkplugB?
Next questions?
Great pictorial explanation!
How are data then structured or packaged at each node/ecosystem in order to be transferred to the single source of truth/name space?
Great question. Many ways depending on the data source. At the device level, data may need to be converted and sent through an MQTT gateway (transport format) using SparkplugB (payload format) and then parsed to populate the UNS---maybe just a timestamp, value and unit. The UNS structure both normalizes and contextualizes it for use by other software. Data from other systems, say the ERP, may involve using a business connector that takes the BAPI and converts it into the semantic hierarchy used by the UNS.
Your vídeos are Amazing...I Just have more one question, about historical data. How any application can read from UNS?
I do see a challenge when we have guarantee that orders are created from erp to mes using only mqtt. How do you make sure no data is lost when you don’t know of the application is listening? Do you let these platforms speak spb? Because I’ve seen no erp speak spb. If they don’t speak spb, do you build a microservice to run api calls to fetch this data?
I have the same question.
@4.0 Solutions Your comment about OPC UA does that change with the newer OPC UA PubSub model? That seems to address many of your problems with OPC.
How do we implement security in UNS or what strategies we need to follow to secure the UNS enviornment?
using user based access control lists.
Does UNS and the cloud portion of the automation stack work for ITAR companies (from a security/regulatory point of view)?
Walker knows this inside out 😅
Yes he does!
Walker actually coined the term Unified Namespace and developed the techniques he talks about at his integration company Intellic Integration.
This could not be more clear
Thanks. Great to hear!
I’m not sure I believe your stat 93% industrial data updated > 1/sec? (especially IO data) it Depends, what level of communications and what are you doing process control, discrete manufacturing or motion control. A big deal made is protocols like MQTT are COS (change of state only) to save bandwidth. Two issues arise 1) COS doesn’t accommodate a heartbeat (how do you know the wire wasn’t run-over by a tow motor, or the cell tower was struck by lightning?) ,2) when a process upset occurs, typically traffic increases exponentially so much that a network size for normal COS only with retries Quickly crashes. How do you deal with those situations?
Who would one seek to set up automation for a new manufacturing plant for a new business to opperate with industry 4.0 practices from the beginning?
You can reach out to contact@40solutions.com and we’d be happy to assist you.
Thanks
No problem, we’re glad this was helpful.
MQTT over NATS.
I stopped watching after 11 minutes, can not give you more time to answer the question of you title. You still did not answer it, it is very frustrating to watch you beating around the bush so much. Is it a design concept? Is it software running somewhere? I have 25 yrs experience in IT specialized in data, software, design concepts (object oriented vs functional etc). Some years ago it was fashion for big companies to look into data brokerage middleware for communication between point solutions, those projects struggled to become succesfull, mostly because of the tough choices to make what you leave up to the point solution and what to the broker. Some benefits/points you mention are not exact, claiming that the UNS has the sst is not true. The truth lies in all connected systems, the UNS only holds ( with least latency as possible) a hot copy of a subset of all the information. I will look at other videos to find more information on what UNS really is.
How is your UNS discovery journey going?
@@4.0Solutions other videos gave me more information: UNS is realtime datalake where the data objects are ordered in a single namespace hierarchy. Updates/selects are done through a publish subscribe mechanism (like mqqt). Imo the biggest drawback is that companies think they should put everything from their pointsolutions into the UNS, but that is not feasible. Upfront choices should be made which information is needed realtime of the whole chain. Also changing/swapping point solutions is difficult as companies will start defining their uns based on their current solutions and the dataobjects used there. If a new or other solution stores different data for the same functional process, the challenge becomes mapping that to the current uns. The central realtime datalake is benificial for data-driven control, but it has also security challenges. The uns need to be able to communicate with all the network levels of the company including cloud. Normally factories don’t want to connect their factory floor to internet, for obvious reasons. Physically seperated networks where put in place for a reason. Could be mitigated a bit by only allowing publish for the factory sensors. You don’t want a hacker to gain control over the factory…
Nice! Things are very changed since the 3.0 epoch of automation. These were the days that i use to work as automation engineer but now days all are changed. I can recall programs that i use to work with, lets say process control programs from Opto22 and scada software Genesis I dont know if it is still in the automation market in the time beeing. I think that the new Elon Musk robots will be connected to a unified name space and because the Tesla is a Data company as you right said in other videos the robots will use that unified name space to read and write data in order to produce products.
Thanks to you I restart to, lets say "play" again with automation, particular with home automation using Raspberry Pi as Mqtt broker and ESP32 and Arduino as clients.
I want your advise in that
That is,
I use to connect lets say a ESP32 microcontroller to an local MQTT broker installed in a local Raspberry Pi and that local MQTT broker connected in and MQTT broker in the web in order to have access to the ESP32 microcontroller from everywhere in the globe.
I use a mobile with MQTT client installed, in order to control and view data
The problem was that, if somehow the MQTT broker in the web collapes the I loose connection to the ESP32.
And then a Idea emerge in my head!!!
I will use an VPN program, I will install a VPN in my local Raspberry Pi in order every time I connect my mobile to the VPN my mobile will be part of the local network so i will not need any more the MQTT broker in the web to control the ESP32 microcontroller when I am away from home, I will use only the local MQTT broker.
Ofcourse
* I use DDNS,
* I have opened a Port in my Router
Which is your opinion about this approach?
What is your advise in that?
You keep doing nice job and videos around the industrial automation world
Call me "Bill from Greece" if you ever mention my name in future videos!!!!
If you ever come to Greece be my Guest!!!!