Root Causes 54: 2019 Lookback - Infrastructure and IoT Security

Tim Callan

How you doing today, Jason?

Jason Soroko

Doing great, Tim, thank you. This is another one of our look backs at 2019.

Tim Callan

Right. We're doing a series of 2019 look backs. We started podcasting right at the beginning of the year and we're now right at the end of the year and so we thought, let's talk about what happened in 2019 and then imagine what we think is going to happen on that topic in 2020. We thought about the year, and we detected some themes. And one of the themes I think we detected was it was a big year for security in the area of IoT devices in general and infrastructure and utilities in particular.

Jason Soroko

Yeah, it sure was. There was a lot going on. I think that the white hats taught us stuff, the black hats taught us stuff.

Tim Callan

Right.

Jason Soroko

Legislation was involved, lights went on, lights went off. It was quite a year, Tim.

Tim Callan

Governments were involved. So yeah, and one of the big ones - - let's start with that last one lights went on, lights went off. One of the big ones, of course, is a big theme this year was the vulnerability of the world's energy grids and related utilities and infrastructure. We saw some notices, for instance, we saw, you know, the United States announced very publicly that it was able to turn off the lights in Russia and we had a whole podcast about that. We saw legislation in Texas about energy grids. We saw several suspicious outages and - - or not suspicious outages because they were, you know, unambiguously attributed to cyberattacks in one form or another. And we also saw certainly what appear to be cyberattacks on the infrastructure that that drives the energy grid, things like power plants and all these things happened in an area that certainly from a cyber perspective, leading up to this year, with few exceptions, has been calm.

Jason Soroko

Yeah, up until this year, we had the Crimean Peninsula lose their power, right and that probably was due to a cyberattack related to the nasty saber rattling and that's putting it lightly between Russia and Ukraine. To a lot of people that looked like a dry run, or I should say, an early run towards other types of outages that could come and sure enough, we saw it in 2019. I think what's interesting about it, Tim, is that some of the outages that happened, the attribution was difficult, which is perhaps why it didn't make more news. But what - - the things that you just rhymed off really should, even if you're not involved in the computer security industry, it should interest everyone.

Tim Callan

Yeah.

Jason Soroko

In that, you know, a new form of cold warfare is going to be, hey, I can turn your lights on and off from afar.

Tim Callan

Yeah, and that's one of the things about this topic is it touches so many aspects of our society. A lot of times when you and I are talking about computer security stuff, it feels like it's super, super weedzy. Like, if you're not in this business, you won't really care. But in this case, this touches politics, it touches to diplomacy, it touches economics, it touches all of these other facets of our interconnected global society that really depends on energy and utilities.

Jason Soroko

It wasn't that long ago, just a few years, we really wondered, you know, what's it going to take for governments to force things like energy distribution companies and other types of companies that work on low margins - -

Tim Callan

Right.

Jason Soroko

- - To move on this? And well, we have our answer. In Texas, it took a complete disassociation between the administration of the systems and the administrators. In other words, there was a - - it seems like there was some sort of cyber event, that's all we know, that’s all we have been told-

Tim Callan

They have been kind of mum about it. Yeah.

Jason Soroko

They've been very mum about it, for good reason. I mean—

Tim Callan

They've been very mum about it, for good reason. I mean—

Jason Soroko

Exactly. But the outcome of it was (a) Money. Wow. Right? Money. Which is like, forget it. Typically, these things get no money whatsoever. And secondly, like legislation, which is not passed easily, and it passed the state law in Texas fairly quick. That should show you that something happened there that was serious enough to do something that had never been accomplished in legislation or anywhere, ever. It freed up money and it freed up resources and it freed up legislation to push this forward because really bad things are happening.

Tim Callan

Right. And the money in point is an important point, right? Because ultimately that means that the people who are paying for these security advances were the taxpayers. So, the legislature determined that it was in the taxpayer’s best interest and the taxpayers needed to do this, because otherwise, the taxpayers would not have a reliable energy grid. And, you know, that's part of what gets interesting, I think, about this aspect, energy, infrastructure utilities, is we all have a stake in it, right? Everybody who flips on a light switch or everybody who drives across a bridge, right, cares about that infrastructure working correctly. And so, it gets out of the realm of just, you know, I'm a company and I buy things for my use and I'm going to make my own risk reward analysis. Instead, it moves into the sort of broader societal set of implications.

Jason Soroko

I hope at the very least Tim in 2020, what this causes is that we never have to hear too much again about either a regulator or a plant operator who just raises their hands and says, I'm firewalled; I'm air gapped; I'm not connected to the public internet; I'm all good, because that mythology just needs to go away. I'm not hopeful for it. I think that a lot of people are still going to live that myth, but I'm hoping in 2020 we start to see the ends of that.

Tim Callan

Yeah, maybe once upon a time, but not in the last decade, that hasn't been true. Certainly, perhaps earlier than that, but not anymore. So, legislation - - while we're talking legislation, there was some other legislation, right. So, we had the Texas energy grid bill is very interesting. We had a bunch of IoT security laws that either passed or are looking like they will pass, including California Senate Bill 327. Including some legislation in Oregon, which I don't have the name in front of me and then I think we also saw some national activity as well.

Jason Soroko

The UK has their own proposed and I'm sure there are other jurisdictions as well that I just don't have my finger on because it's becoming quite important. It's interesting to see how it started in California, Tim.

Tim Callan

Yeah.

Jason Soroko

I think we podcasted about this, but it's worth repeating. They started slow. It basically was a direct reaction to them, Mirai Botnet, and the problem of static credentials. Isn’t it an amazing that the problem of strong authentication moved a legislature? That's really something.

Tim Callan

Yeah.

Jason Soroko

And it's funny how we're a podcast, but we understand that we're a niche audience, right, that listens to us and yet, the topics that we're speaking about are now of national and global importance, is it - - I don't know, it gives me a little bit of a big head in some ways, Tim, because it shows the stuff that we're doing is kind of important.

Tim Callan

Yeah, it is kind of a who would have thunk it moment, if you think about it, where you know, imagine back you and I have both been in this world - - certainly I've been in this world longer than I want to talk about and, you know, I once upon a time, this really was a thing for bit heads and the concept if you went back in time to the ‘90s, and you asked me if, you know, I could imagine people discussing these things on the floor of the legislature, I would have laughed at you and said no freaking way. And sure, it took 20 years, but now we're here and they are so that's cool. And that's good.

Jason Soroko

You know, Tim, I'm thinking about various municipal level governments now who got hit badly with ransomware.

Tim Callan

Yeah, yeah, that was another big trend, wasn't it? Is this ransomware turned upon local governments trend that we saw this year?

Jason Soroko

Yeah, I think when people were first seeing ransomware, it kind of made sense that, you know, the average user would just, you know, lose access to their hard drive in their data and they pay a few Bitcoins and get it back, you know, if they were lucky. And otherwise, it was kind of a made you feel really bad event, but it started to really amplify into municipalities, but it also amplified into major logistical companies like Maersk. They saw some of the worst of it. I question whether or not that term ransomware - - because I know it works great for clickbait for those of you who are journalists who might be listening to this, good on ya, I mean, we're glad you chose a word that gets people - - but in reality, I think a lot of that what was considered ransomware and look like ransomware was really just good old-fashioned denial of service. It was to cause havoc; it was to it was to cause outages and never really meant to gain any monetary value. There was no fraud element to it. It was just good old-fashioned like sabotage.

Tim Callan

Right. But sure - - I think it all gets confused, right? It's hard to sometimes draw the lines of what was mere vandalism, and what was vandalism for economic gain and if you stop the vandalism, and if we want to take an old, an offline analogy, you know, if you're a small business owner and you have a shop downtown and you come to work one morning and your window is broken, that might be vandalism. But, you know, if your window is broken again the next morning that's probably somebody expects you to pay them off. And so, you know, how do you draw the line there? It all it all gets a little fuzzy and, you know, yeah, how much of this is ransomware and how much of this is other activity is not clear when you just read the headlines.

Jason Soroko

Yeah. You're right, Tim. However, in the case of things like Maersk, right, that the downtime was so severe that it really is, you know, it goes far beyond ransomware, and I think that's something that listeners to this podcast really need to understand is a lot of the worst of it is full on sabotage.

Tim Callan

Yeah, um, and, you know, this gets into some of the other topics that we've talked about in the past that we don't need to belabor now, but, you know, getting back to the governments and things, you know, as cyber activity becomes another form of diplomacy/Cold War, that obscures everything as well. You and I talked about it, it was kind of a light and funny topic, but you and I talked in an earlier podcast about the fact that it appears that the Russian government is spoofing GPS signals in the immediate proximity of Vladimir Putin. And so, we kind of laugh because it's funny, but it's also not funny, right? It shows how seriously that they're taking this stuff. And so, that's yet another example of where the cyber and the politics and everything are blending into each other much more so historically than they have been.

Jason Soroko

Yeah, it's I think that will be the trend into 2020, Tim, is this geopolitical - - these kinds of trends that we're seeing, which are not great - - I mean, it's so funny that we hear the, you know, the end of the Cold War, which was anything, but into whatever this is now, where we have nation states acting as kleptocracies. We have other nation states that are saber rattling for their own form of negotiation tactic. You know, we have China versus the United States, a trade war, which hopefully doesn't spin off into something worse. It's an interesting world that we're living in; however, everything we just brought up especially in the cyber physical space of IoT, this is it's big, big business. Really important. It affects people's lives. It's, you know, when you think back to the development of Windows, if you look at some of the classical errors that Bill Gates made - - and people feel free to disagree with me, but I'll just rhyme off a few things here. One of the things Bill Gates - - one of his early mistakes was not to embrace the internet, right, the internet was kind of a - - not a first factor consideration in the development of Windows. And in fact, I think Bill Gates called it a fad, although I - - I won't quote him as saying that. I think that that was attributed to him and sure enough, that was incorrect. But part of the problem was, you know, Windows in itself, the whole theory behind it was, it's okay to let it bloat because we have Moore's Law, and therefore, it's going to be lower quality software than if we were to obsess over it and make it like a work of art. The problem with that was, obviously Windows in itself, which is, you know, major infrastructure worldwide, really was not made with security in mind, and was not made with networking in mind, especially the public internet that we have today. And here we are, now, it's going to be 2020 next year, Tim—

Tim Callan

It will. Very soon.

Jason Soroko

And we are still dealing with infrastructure, especially in industrial areas, and other places that are so fundamental to the way we live, that still have these design principles of minimal to no security and we somehow are surprised when a plant blows up or the lights go out. It's something - - really, really something so I think 2020 will - - what I'm really hoping, Tim, is that this phenomenal amount of discipline that the black hats have, whether they're out for fraud, or whether they're out whether they're a nation state, the phenomenal amount of discipline these black hats have in not pulling the trigger on the worst of what is available to them in terms of a target. You know, I hope they just stick with fraud like they have been and, you know, have been growing very wealthy in terms of just the sheer amount of money that they can steal from people and do the odd outage that hits the news and do, you know, a breach, etc. I don't think any of those things are going to stop but I'm really, really hoping that we're not having to deal with power outages and plants blowing up, you know, in the worse way.

Tim Callan

And it is an intractable problem, though and this is part of what makes it tough. It's like, again, when you and I have talked about the quantum - - post quantum encryption conversation. That's an intractable problem because when you do the math, you go, well, there are secrets now - - encrypted secrets that need to be secret beyond, you know, beyond the bad date, right and it's already too late for certain things. I think this is a similar problem, right? You start to say, well, you know, what do we do? How do we go retrofit and upgrade our global infrastructure that has been built on the foundations of things that were, you know, made 100 years ago or more and make it all, you know, locked down tight, secure, while still meeting our other requirements, and always on, it works, etc., etc., and economical and etc.? And that's just super, super, super hard. And you know, the people who are putting up the security wall need to defend everything, and the bad guy only needs to find one hole.

Jason Soroko

Tim, I'm going to add to that with an area that - - this conversation seems to come up almost every time I'm talking to whether it's a customer, or anybody who's just interested in the topic and with respect to IoT specifically, you know, we really do have to break down the problem into two parts, which is the Brownfield and the Greenfield.

Tim Callan

Yeah.

Jason Soroko

Because the sheer amount of technology that even the company that we work for, Sectigo, can bring to bear on Greenfield IoT technologies is phenomenal. Not only can you be, you know, not only are you going to be able to pass with flying colors the kind of judgments that that legislation will bring to you and guidances from standards bodies like NIST, but you'll have a secure product. The problem, the problem is, there's only so much that we can do in the Brownfield problem.

Tim Callan

Yeah.

Jason Soroko

Which are devices that are already out in the field. And that term IoT, which seems like a new term, and it is new term does not address well the fact that there have been connected computing devices in industrial zones for 30 years plus and those devices really have little to no concept of security. Now there absolutely are passive forms of security that can be put into industrial plants. The problem is, you know, it doesn't take much of a search on a showdown engine to show you that there are still industrial controllers that are connected to the public internet with minimal forms of strength of authentication.

Tim Callan

Yeah, and the timeframes are so different, you know, when we, you know, if you capitalize laptops for your employees, you're probably capitalizing them on a three- or five-year basis, which means that somewhere between three and five years you're expecting 100% of those laptops to be cycled out to be in the garbage or donated or whatever you do with them, and that employee is using a new laptop. Now, if you walk into this world of devices in the field, you could have things around for 20 years, right? Think about a car. A car is a perfect example. A car is IoT device these days. And, you know, I still have a car. It's not my only car, but I have a car that I bought in 1993 that I still drive. And, you know, that's how long cars can last. And so, it puts things just in a very different time frame than what we historically have been used to when we just think about our classic computing devices, our servers, and our phones and our desktop machines.

Jason Soroko

There's probably an airplane being built right now in an Airbus or Boeing plant that we'll still be flying in the post quantum world.

Tim Callan

Yes. Yes. Exactly. Let's talk about cars, it's probably an important one. So, you know - - It's the end of 2019. What is the status of security for automobiles, for connected automobiles and where do you think it's going in the future?

Jason Soroko

The automotive industry was one of the pioneers in taking PKI into the automobile. So, in other words, secure elements on things such as electronic control units that manage everything from your engine systems to, you know, braking systems to you name it in an automobile. And I think where we've seen things go, which is natural, is certain sets of automotive manufacturers, OEMs have chosen to go in certain kinds of different directions. So, you have the German companies with some of their standards, you have some of the Japanese companies with theirs as well and, you know, I don't know whether it will all come together in the end, it probably won't. I think there's room for different implementations, but I think we’re definitely at the point where these things are being used. I just don't know if they're being used at a scale that is to the point where you could call an automobile, fully secure. And let me just - - here's what I mean by that. And those of you who work for the automotive industry, who'd love to counter us, hey, let's get you on the podcast. We want to talk with you, okay.

Tim Callan

Yeah.

Jason Soroko

But let me just use what's publicly available in terms of information. So, if you look at Keen labs, and what they did with Tesla, and even the most recent talk at Black Hat, for example, right, we had Secure Boot problems that were exploited on the electronic control unit, which meant, specifically that firmware on the ECU could be manipulated, right, which is a great way to persist if you're a bad guy in a car. Again, the infotainment system to CAN bus bridge continues to be bridged remotely. That's something we need to solve, right? So, things like embedded firewalls, incredibly important, right? So, in other words, the automotive manufacturers, even the most, you know, some of the ones that might be considered the highest tech, something like Tesla, they continue to have problems. Now, obviously, when things like Keen labs and the smart people over there, the white hats, report these problems, Tesla uses that and makes a better car. I think that whether it's the big auto manufacturers, the big three or four or five, or something like a specialty maker, like Tesla, I think they've all hired some of the best people around. I think that's a big trend, Tim, is I think they're all taking it seriously.

Tim Callan

Yeah.

Jason Soroko

I think also that the big tier one providers are also taking it seriously. I could go into some of the details of that but let's just leave it there for now and say, I think the automotive industry has always been a very forward-looking pioneer. I think that they took some early baby steps and I think they took some bigger leaps and now we're at the point where - - I think we're at the cusp of doing some much, much bigger things in future model cars. And I think that with autonomous vehicles, which I think if even a year or two or three ago, were question marks, like, where are we ever going to see this? Are the insurance companies going to allow this? I think the answer is clear. I think it's inevitable.

Tim Callan

Yeah, and I mean - - that's just if you want to talk about an industry that has big ambitions, let's talk about, you know, the automobile industry when, you know, with people going heavily after electric vehicles, after autonomous vehicles after, you know, various forms of connectivity and really trying to leap forward in terms of the product they're offering that they want to manufacture by the millions and put all over the globe, like, this is a very ambitious industry and all of this stuff is super computerized or it doesn't work. And if it's not secure - - if the IT security is not there, the consequences are bad. So, when you put all those pieces together, this is an area for sure where I'm expecting to see them continue to be on the forefront.

Jason Soroko

And they are Tim. So even if you consider things like electric cars and charging points, those have certificates in them. That’s PHI.

Tim Callan

Oh yeah. Sure. Right.

Jason Soroko

So once again, you know, the ubiquity of PKI, is I mean, we spend a lot of our lives in cars and it's all over. And I really do applaud the automotive industry for being one of the first in and they're pretty much the furthest ahead and I'd like—I would love to see it even further. But of course, that's just me. I've got my bias. But on the other hand, I do have to applaud them for what they've done.

Tim Callan

And it will be and I'm sure that this is a topic we'll be returning to again in 2020 because I'm sure we'll have interesting developments going on in IoT security when it comes to the automobile industry.

Jason Soroko

Yes, sir.

Tim Callan

So maybe that's a good spot to leave it for today. Certainly, it was an interesting year for IoT, and infrastructure and I think 2020 is going to be more interesting.

Jason Soroko

I have absolutely no doubt about that. I think that the progression that we began over the past few years, I mean, I've been watching it very closely since 2014, to watch it get to this point, I think we're going to really get to an inflection point next year, Tim. We will be watching it closely.

Tim Callan

We will be in and we’ll telling you about it as it happens so, make sure you subscribe and stick with us here at Root Causes. Thanks everybody.