Dino Busalachi, Velta Technology PostedThursday, May 4, 2023 Q How do you describe to lay people what it is that you do? A We provide digital safety around connected physical systems, connected assets, the industrial control system world. We provide mitigating and compensating controls to protect and provide digital safety around that environment. Velta was formed specifically with that in mind. That’s predominantly what we focus on. You will hear a term called OT, operational technology. Operational technology describes pretty much everybody that has anything to do with running critical infrastructure facilities or manufacturing environments. That’s your plant management, that’s your process control engineering teams, that is your operators, your QA people, anybody that revolves and involves in making goods or producing services is OT. Everybody knows what IT is, which is the other side of the equation. IT has a role in regard to working with the OT environment predominantly, but at the end of the day, the OT folks are the ones that own those assets that are out there making goods and services for that organization. Q Talk about the history of Velta -- how did it begin and how has it evolved over the years? A It’s really a combination of 4 decades of me working in this environment leading up to Velta, which is coming into its 5th year. Prior to that, I was working for Rockwell Automation in their network security services group. Prior to that, I was working in the Rockwell channel with distributors and system integrators. Prior to that, I spent 20 years working for Anheuser-Busch in the engineering group and other manufacturers. Knowing that environment, knowing what is out in those manufacturing environments which covered decades, right? I mean in the OT world, industrial control system’s lifecycles are 10 and 20 and 30 years in some cases. It’s not in a short lifecycle like IT systems that are sometimes 3 to 5. And what we are focused on is how do we put security around something that didn’t have safety and security in mind when it was built? Velta was formed specifically with some of these new toolsets that were coming out that were geared for asset inventory, malware, vulnerabilities, and exposures built for the control system arena. We are recognizing that that gap was there that need to be filled. In this ecosystem of OT, suppliers, and vendors, we thought that this would be a really good fit in that space. That’s why we founded Velta, specifically to go after that. And there are a lot of good companies out there that have brought these tools forward, Cisco Systems being one. They’ve been in the industrial control system arena for quite some time providing network connectivity types of technology. They are switching platforms, rerouting platforms. It has been in the industrial environment for over 20 years. They saw the same issue because they are a big cybersecurity company and wanted to be relevant in the space. That’s one of our key partners, along with some of the others that specialized in OT and intrusion detection systems. Q Is there a specific industry or a product or discipline and why or why not? A There are two priorities that you find in manufacturing environment. Anybody with control systems, safety is always job one. Job two is unplanned, unscheduled downtime. You want to try to minimize that because it’s very costly when you’re not making goods. You got machines that aren’t producing goods. You got people that are there that you’re paying salaries. You got resources you’re consuming, utilities. You got trucks waiting. You got a whole bevy of activity where the average of an hour of unplanned, unscheduled downtime would cost $50,000 an hour. Automotive industries $20,000-$25,000 a minute if they are not producing goods. Availability is always key for them right up front, not confidentiality like what IT would do when it came to an event. Where in the OT world, if there’s something going on, 1, you got to be safe, 2, if you’re going to bring this equipment down, you got to do it in a safe manner. You don’t want to destroy it. You must decide, “Do we really need to go down?” And that’s one of the biggest challenges that the OT people are starting to learn as these organizations are attacked. If you don’t have visibility into your OT environment, you don’t know if you’re safe or not. Whereas IT can see what’s going on in their world. They’ve got tools that are telling them, “We got malware and we know where it’s going or where it’s at and what it might be.” But if you’re not doing the same thing on the OT side, then you have no idea what’s down there because IT is not watching that stuff. They don’t know what’s going on inside that PLC or necessarily an HMI that might be still running Windows XP on it or the engineering workstations or the drives that are out there, the robots. They have no idea what the status of any of that stuff is, safe or not. And if you’re not paying attention to it, then the natural move is to shut down to be safe, which is what we saw with Colonial Pipeline. They got attacked. It came in through their email systems. IT was getting hammered. They looked over to the OT guys and asked them, “How are you doing over there?” And they’re like, “I don’t know. We’re operating but we don’t know if we have malware. We can’t see it.” So, they had to shut down to be safe. We see that a lot. We are seeing more insurance companies coming in right now because they’ve gotten smarter over the last few years and it’s like, “Yeah, I get this report from you about what you’re doing to secure and protect your business systems, but you’re a manufacturer, what are you doing on the plant floor side? Demonstrate to me what you’re doing over there that’s the same you’re doing over here.” There is a big gap when that happens. Suddenly, they’re not getting their cybersecurity insurance because they’re not demonstrating that they are having reasonable duty of care in taking care of their control system environment. One of the questions we ask organizations when we engage in our table topic exercises is, “Are you applying the same amount of due diligence to protect and secure the control system environment as you are your enterprise? And if you are not, why aren’t you doing that?” It’s usually because of this relationship between IT and OT is not as well aligned and collaborative as many people think that it is. When you have a manufacturer that has 20, 30, 40, 50, or more plants in their fleet, regionally around the world, it’s very difficult for a few people sitting in a corporate arena to try to determine what is it really going to take at the local level to protect and secure systems out there that they have no idea what they are or what they’re doing or what strike they have. And what I mean by strike is the plant floor is littered with technology from all over the globe. IT standardizes on stuff. They standardize on a computing platform. They standardize on a networking platform. They standardize on an operating system and certain applications. In the OT world, it’s all over the map. You can find a 25 or 35 or 50 to 1 ratio difference between the number of OT/IoT assets in a manufacturing environment compared to how many IT assets you have. For every 1 IT asset, there are 20 or 30 or 50 IoT or OT assets out there in your manufacturing arena. Machines are getting smarter. All the new machines you buy coming in are getting smarter. They want connectivity. Your third parties, your OEMs, and your system integrators that have warranty and support responsibility are wanting access in there. COVID kicked the door wide open for remote access into these environments. For those people that thought that they were air gapped, those days are gone. Digital transformation has changed that, industry 4.0 has changed that, COVID changed that. And now, we have all this access in and out of this environment, and we’re not watching it or protecting it or have the wrong tools in place. A lot of the IT tools are incompatible for the industrial arena. They don’t speak the same language and their behaviors are different. When you get into the behaviors in the industrial world, injecting additional traffic into a control system environment to try to learn about it is disruptive, unplanned, unscheduled downtime. You can shut down the plant floor. They do shut down the plant floor. IT doesn’t necessarily know that they are doing it because for them, it’s just their normal behavior in trying to discover assets that are out there connected into the environment to try to determine: Do they need to be here? Do they belong here, or what are they, and what are they talking to, and what applications they are running on? What protocols are they using? They are doing their due diligence but that bleed over and trying to throw that on to the plant floor is very disruptive. That’s why these tools that came out that were geared for the industrial control system arena are passive in nature. They listen. They don’t talk. Where on the IT tools, they are screaming down the wire. If you have a PLC that’s sitting there doing something that’s very timely that can’t be disruptive, maybe it’s running a filler, filling 2,000 cans a minute for example, distracting it causes jams and causes backups and causes a mess and shuts down my filler because IT wanted to scan the network. Their practices are different. They need to understand that. And there are a lot of people who do not understand that right now because the tools they buy, the practices that they deploy, that’s just what – and the vendors that sell them these technologies, they don’t teach them any different on that, right? They just say, “Yeah, take my tool and go out there and scan and learn about everything you can in your environment regardless of the asset type that’s out there.” It takes a different mindset and knowledge on how to work in that space. It’s the perception that IT owns cybersecurity in the general sense. The executive level, the C-suite, they would assume that. Why would they not assume that? They’re the ones with the cybersecurity professionals. They are the ones with the budget. They are the ones with the tools. And OT is perfectly fine when you mention about cybersecurity to them, “That’s IT’s job.” They want to deflect. They don’t want that responsibility. But safety is everybody’s responsibility too. It’s either everybody’s responsibility or it’s nobody’s responsibility, depending on what kind of organization you’re working with. And what I would say is, when you get into those discussions, they need to have that. There needs to be the three-legged stool needs to be IT, the C-suite, and OT sitting down and having this conversation. Tabletop exercise is a really good way of getting those gaps up on the table because we’ve been in a few where -- I like showing pictures of the plant floor. I like opening the panels that house the PLCs and the drives and the network switches that are in there and maybe the HMIs that are on the door of the panel and showing those because that panel is a full-blown Windows computer running on that – that’s running packaging line for example. And when you show those to the executives and even IT, especially the CIO or the CISO and they see that stuff and they’re like, “Well, yeah, that’s not in my purview. That’s not my responsibility.” Well, if it’s not in your area of responsibility then whose is it? Because the people that own those assets think that you’re protecting them, No. 1, or No. 2, they don’t have the resources and money and time to go do the cybersecurity practices to mitigate and put in compensating controls around this stuff because they’re not using these tools either. The C-suite needs to see that and recognize that. And some of the insurance companies, like I said earlier, are starting to bring that to the table because now, risk is involved because they’re getting a letter from the insurance company saying, “Hey, we’re not going to cover you because you’re not fessing up on your OT side.” It’s starting to get on the table. We have regulations coming down. Critical infrastructure is getting pressure from the Department of Homeland Security. TSA, CISA, the EPA are starting to put regulations in these critical infrastructure groups asking them to do a better job of at least providing asset inventory and the state of those assets to them. And if you do get breached, you must respond within a period to let them know. You have local water municipalities that have no budget, no people, it’s a challenge. It really, really is a challenge out there. But there are grants. There are all kinds of money that’s available to them through the federal and state governments to help with training, to help with some of the tool selection, at least for them to get started. But they need to start taking a look at how they can get some help in these critical infrastructure groups that are providing water, wastewater and/or power services to their community. Q What are some other trends and challenges you’re seeing in industrial automation right now? A They all want to try to get data up and off that plant floor. How can I make more with less? How can I be more efficient? How do I fit into the community as far as being a good steward? We’ve seen some pretty nasty stuff going on here recently. We had a chocolate plant that blew up a couple of weeks ago. We’ve had trains falling off of tracks. Manufacturers are struggling. Dole recently got hit. Not everybody tells you. Everybody has been hit, I think. They just either don’t know it or they don’t want to disclose it. There are lots of clients out there that don’t want to say what has happened to them. The hundreds and millions of dollars in brand and loss that they suffered because of being attacked. And so, when I think about that piece of it, it’s like what are you doing to get better? Can you be better this year than you were last year? Are you going to be better next year than you were this year? What is your plan and are you developing that plan? Is it measurable? We created what we call a Connected Device Index. And what it does is it allows you to look at all your OT assets vulnerabilities and exposures that are listed either by the vendors themselves, Microsoft, NIST, CISA. And these are – bulletins are published so everybody knows what Patch Tuesday is with Microsoft. Every Tuesday, Microsoft pushes out their patches. And you can take them or not take them. The plant floor is notorious for not patching because it’s disruptive. Again, unplanned, unscheduled downtime. I take this patch; it could break my app, or I got to take my machine long enough to take that patch and I don’t want to do that. I got Windows XP machines. I can’t get patches anyway. So, when you think about it in that context, our CDV index allows you to see where you are. And then back in time, the vulnerabilities that became available and you can build on it so you can at least have a snapshot in real time of your current vulnerabilities. And then as you go through time, am I getting better? What am I doing to bring that down? Think of it as a FICO score. Just like your FICO score on your credit. You got a credit reading out there. Banks decide on what they are going to lend you based on that. Well, you can have an index on your OT assets to tell you the same thing. IT has it. IT can quickly pull up and look at all their IT assets that they have and what their patch levels are and when they were patched because that’s what IT does. They patch, patch, patch, patch, patch. That’s what they do. They take advantage as soon as those things come out, they push them out there. In the OT world, which doesn’t happen. That’s a big difference between those two groups. Then you got to develop those compensating, mitigating controls for them to determine how they can better protect themselves because if that’s not their behavior then what else can I do? And there are other things they can do but they have to explore those, but they have to know where they are first. You can’t just say, “Yeah, we don’t patch.” Well, how bad are you? I mean really, how bad are you? They don’t know.
Q How do you describe to lay people what it is that you do? A We provide digital safety around connected physical systems, connected assets, the industrial control system world. We provide mitigating and compensating controls to protect and provide digital safety around that environment. Velta was formed specifically with that in mind. That’s predominantly what we focus on. You will hear a term called OT, operational technology. Operational technology describes pretty much everybody that has anything to do with running critical infrastructure facilities or manufacturing environments. That’s your plant management, that’s your process control engineering teams, that is your operators, your QA people, anybody that revolves and involves in making goods or producing services is OT. Everybody knows what IT is, which is the other side of the equation. IT has a role in regard to working with the OT environment predominantly, but at the end of the day, the OT folks are the ones that own those assets that are out there making goods and services for that organization.
Q Talk about the history of Velta -- how did it begin and how has it evolved over the years? A It’s really a combination of 4 decades of me working in this environment leading up to Velta, which is coming into its 5th year. Prior to that, I was working for Rockwell Automation in their network security services group. Prior to that, I was working in the Rockwell channel with distributors and system integrators. Prior to that, I spent 20 years working for Anheuser-Busch in the engineering group and other manufacturers. Knowing that environment, knowing what is out in those manufacturing environments which covered decades, right? I mean in the OT world, industrial control system’s lifecycles are 10 and 20 and 30 years in some cases. It’s not in a short lifecycle like IT systems that are sometimes 3 to 5. And what we are focused on is how do we put security around something that didn’t have safety and security in mind when it was built? Velta was formed specifically with some of these new toolsets that were coming out that were geared for asset inventory, malware, vulnerabilities, and exposures built for the control system arena. We are recognizing that that gap was there that need to be filled. In this ecosystem of OT, suppliers, and vendors, we thought that this would be a really good fit in that space. That’s why we founded Velta, specifically to go after that. And there are a lot of good companies out there that have brought these tools forward, Cisco Systems being one. They’ve been in the industrial control system arena for quite some time providing network connectivity types of technology. They are switching platforms, rerouting platforms. It has been in the industrial environment for over 20 years. They saw the same issue because they are a big cybersecurity company and wanted to be relevant in the space. That’s one of our key partners, along with some of the others that specialized in OT and intrusion detection systems.
Q Is there a specific industry or a product or discipline and why or why not? A There are two priorities that you find in manufacturing environment. Anybody with control systems, safety is always job one. Job two is unplanned, unscheduled downtime. You want to try to minimize that because it’s very costly when you’re not making goods. You got machines that aren’t producing goods. You got people that are there that you’re paying salaries. You got resources you’re consuming, utilities. You got trucks waiting. You got a whole bevy of activity where the average of an hour of unplanned, unscheduled downtime would cost $50,000 an hour. Automotive industries $20,000-$25,000 a minute if they are not producing goods. Availability is always key for them right up front, not confidentiality like what IT would do when it came to an event. Where in the OT world, if there’s something going on, 1, you got to be safe, 2, if you’re going to bring this equipment down, you got to do it in a safe manner. You don’t want to destroy it. You must decide, “Do we really need to go down?” And that’s one of the biggest challenges that the OT people are starting to learn as these organizations are attacked. If you don’t have visibility into your OT environment, you don’t know if you’re safe or not. Whereas IT can see what’s going on in their world. They’ve got tools that are telling them, “We got malware and we know where it’s going or where it’s at and what it might be.” But if you’re not doing the same thing on the OT side, then you have no idea what’s down there because IT is not watching that stuff. They don’t know what’s going on inside that PLC or necessarily an HMI that might be still running Windows XP on it or the engineering workstations or the drives that are out there, the robots. They have no idea what the status of any of that stuff is, safe or not. And if you’re not paying attention to it, then the natural move is to shut down to be safe, which is what we saw with Colonial Pipeline. They got attacked. It came in through their email systems. IT was getting hammered. They looked over to the OT guys and asked them, “How are you doing over there?” And they’re like, “I don’t know. We’re operating but we don’t know if we have malware. We can’t see it.” So, they had to shut down to be safe. We see that a lot. We are seeing more insurance companies coming in right now because they’ve gotten smarter over the last few years and it’s like, “Yeah, I get this report from you about what you’re doing to secure and protect your business systems, but you’re a manufacturer, what are you doing on the plant floor side? Demonstrate to me what you’re doing over there that’s the same you’re doing over here.” There is a big gap when that happens. Suddenly, they’re not getting their cybersecurity insurance because they’re not demonstrating that they are having reasonable duty of care in taking care of their control system environment. One of the questions we ask organizations when we engage in our table topic exercises is, “Are you applying the same amount of due diligence to protect and secure the control system environment as you are your enterprise? And if you are not, why aren’t you doing that?” It’s usually because of this relationship between IT and OT is not as well aligned and collaborative as many people think that it is. When you have a manufacturer that has 20, 30, 40, 50, or more plants in their fleet, regionally around the world, it’s very difficult for a few people sitting in a corporate arena to try to determine what is it really going to take at the local level to protect and secure systems out there that they have no idea what they are or what they’re doing or what strike they have. And what I mean by strike is the plant floor is littered with technology from all over the globe. IT standardizes on stuff. They standardize on a computing platform. They standardize on a networking platform. They standardize on an operating system and certain applications. In the OT world, it’s all over the map. You can find a 25 or 35 or 50 to 1 ratio difference between the number of OT/IoT assets in a manufacturing environment compared to how many IT assets you have. For every 1 IT asset, there are 20 or 30 or 50 IoT or OT assets out there in your manufacturing arena. Machines are getting smarter. All the new machines you buy coming in are getting smarter. They want connectivity. Your third parties, your OEMs, and your system integrators that have warranty and support responsibility are wanting access in there. COVID kicked the door wide open for remote access into these environments. For those people that thought that they were air gapped, those days are gone. Digital transformation has changed that, industry 4.0 has changed that, COVID changed that. And now, we have all this access in and out of this environment, and we’re not watching it or protecting it or have the wrong tools in place. A lot of the IT tools are incompatible for the industrial arena. They don’t speak the same language and their behaviors are different. When you get into the behaviors in the industrial world, injecting additional traffic into a control system environment to try to learn about it is disruptive, unplanned, unscheduled downtime. You can shut down the plant floor. They do shut down the plant floor. IT doesn’t necessarily know that they are doing it because for them, it’s just their normal behavior in trying to discover assets that are out there connected into the environment to try to determine: Do they need to be here? Do they belong here, or what are they, and what are they talking to, and what applications they are running on? What protocols are they using? They are doing their due diligence but that bleed over and trying to throw that on to the plant floor is very disruptive. That’s why these tools that came out that were geared for the industrial control system arena are passive in nature. They listen. They don’t talk. Where on the IT tools, they are screaming down the wire. If you have a PLC that’s sitting there doing something that’s very timely that can’t be disruptive, maybe it’s running a filler, filling 2,000 cans a minute for example, distracting it causes jams and causes backups and causes a mess and shuts down my filler because IT wanted to scan the network. Their practices are different. They need to understand that. And there are a lot of people who do not understand that right now because the tools they buy, the practices that they deploy, that’s just what – and the vendors that sell them these technologies, they don’t teach them any different on that, right? They just say, “Yeah, take my tool and go out there and scan and learn about everything you can in your environment regardless of the asset type that’s out there.” It takes a different mindset and knowledge on how to work in that space. It’s the perception that IT owns cybersecurity in the general sense. The executive level, the C-suite, they would assume that. Why would they not assume that? They’re the ones with the cybersecurity professionals. They are the ones with the budget. They are the ones with the tools. And OT is perfectly fine when you mention about cybersecurity to them, “That’s IT’s job.” They want to deflect. They don’t want that responsibility. But safety is everybody’s responsibility too. It’s either everybody’s responsibility or it’s nobody’s responsibility, depending on what kind of organization you’re working with. And what I would say is, when you get into those discussions, they need to have that. There needs to be the three-legged stool needs to be IT, the C-suite, and OT sitting down and having this conversation. Tabletop exercise is a really good way of getting those gaps up on the table because we’ve been in a few where -- I like showing pictures of the plant floor. I like opening the panels that house the PLCs and the drives and the network switches that are in there and maybe the HMIs that are on the door of the panel and showing those because that panel is a full-blown Windows computer running on that – that’s running packaging line for example. And when you show those to the executives and even IT, especially the CIO or the CISO and they see that stuff and they’re like, “Well, yeah, that’s not in my purview. That’s not my responsibility.” Well, if it’s not in your area of responsibility then whose is it? Because the people that own those assets think that you’re protecting them, No. 1, or No. 2, they don’t have the resources and money and time to go do the cybersecurity practices to mitigate and put in compensating controls around this stuff because they’re not using these tools either. The C-suite needs to see that and recognize that. And some of the insurance companies, like I said earlier, are starting to bring that to the table because now, risk is involved because they’re getting a letter from the insurance company saying, “Hey, we’re not going to cover you because you’re not fessing up on your OT side.” It’s starting to get on the table. We have regulations coming down. Critical infrastructure is getting pressure from the Department of Homeland Security. TSA, CISA, the EPA are starting to put regulations in these critical infrastructure groups asking them to do a better job of at least providing asset inventory and the state of those assets to them. And if you do get breached, you must respond within a period to let them know. You have local water municipalities that have no budget, no people, it’s a challenge. It really, really is a challenge out there. But there are grants. There are all kinds of money that’s available to them through the federal and state governments to help with training, to help with some of the tool selection, at least for them to get started. But they need to start taking a look at how they can get some help in these critical infrastructure groups that are providing water, wastewater and/or power services to their community.
Q What are some other trends and challenges you’re seeing in industrial automation right now? A They all want to try to get data up and off that plant floor. How can I make more with less? How can I be more efficient? How do I fit into the community as far as being a good steward? We’ve seen some pretty nasty stuff going on here recently. We had a chocolate plant that blew up a couple of weeks ago. We’ve had trains falling off of tracks. Manufacturers are struggling. Dole recently got hit. Not everybody tells you. Everybody has been hit, I think. They just either don’t know it or they don’t want to disclose it. There are lots of clients out there that don’t want to say what has happened to them. The hundreds and millions of dollars in brand and loss that they suffered because of being attacked. And so, when I think about that piece of it, it’s like what are you doing to get better? Can you be better this year than you were last year? Are you going to be better next year than you were this year? What is your plan and are you developing that plan? Is it measurable? We created what we call a Connected Device Index. And what it does is it allows you to look at all your OT assets vulnerabilities and exposures that are listed either by the vendors themselves, Microsoft, NIST, CISA. And these are – bulletins are published so everybody knows what Patch Tuesday is with Microsoft. Every Tuesday, Microsoft pushes out their patches. And you can take them or not take them. The plant floor is notorious for not patching because it’s disruptive. Again, unplanned, unscheduled downtime. I take this patch; it could break my app, or I got to take my machine long enough to take that patch and I don’t want to do that. I got Windows XP machines. I can’t get patches anyway. So, when you think about it in that context, our CDV index allows you to see where you are. And then back in time, the vulnerabilities that became available and you can build on it so you can at least have a snapshot in real time of your current vulnerabilities. And then as you go through time, am I getting better? What am I doing to bring that down? Think of it as a FICO score. Just like your FICO score on your credit. You got a credit reading out there. Banks decide on what they are going to lend you based on that. Well, you can have an index on your OT assets to tell you the same thing. IT has it. IT can quickly pull up and look at all their IT assets that they have and what their patch levels are and when they were patched because that’s what IT does. They patch, patch, patch, patch, patch. That’s what they do. They take advantage as soon as those things come out, they push them out there. In the OT world, which doesn’t happen. That’s a big difference between those two groups. Then you got to develop those compensating, mitigating controls for them to determine how they can better protect themselves because if that’s not their behavior then what else can I do? And there are other things they can do but they have to explore those, but they have to know where they are first. You can’t just say, “Yeah, we don’t patch.” Well, how bad are you? I mean really, how bad are you? They don’t know.