Bryson Bort of Scythe.io talks about the Colonial Pipeline attack and the lessons learned from the crippling ransomware attack a year ago

Bryson Bort is a cybersecurity industry leader with experience in both the public and private sectors. He is the founder and CEO of Scythe.io, which performs high-level vulnerability assessments. He is also the co-founder of the ICS Village, a non-profit organization that builds critical infrastructure (CI) and presents it at various conferences. The non-profit also offers guidance and resources for IT practitioners.

Bort’s experiences made him the perfect fit to moderate a highly anticipated session at this year’s RSA Conference in San Francisco: Colonial Pipeline - What Happened, What Changed, featuring panelists from government, the media, and the fuel industry. ConversingLabs podcast host Paul Roberts checked in with Bort during this year’s conference to talk about the panel discussion, and the lessons learned from the Colonial Pipeline incident - including the need for closer scrutiny of critical infrastructure and the limits of voluntary, industry-driven standards for cybersecurity.

EPISODE TRANSCRIPT

PAUL ROBERTS
Okay. So welcome back to ConversingLabs. This is our RSA ConversingLabs Cafe, and we're here with Bryson Bort of SCYTHE. Bryson welcome.

BRYSON BORT
Hey. Good to see you again.

PAUL ROBERTS
It's great to see you again. How are you doing?

BRYSON BORT
Every day is a holiday.

PAUL ROBERTS
And you're out at the RSA conference. Talk a little bit. You both you are on a panel, which we're going to talk about shortly, but you're also doing some work there just with the conference itself. So talk about that.

BRYSON BORT
Yeah. So, well, every day is a holiday. Four days at RSA has already sold, like, four weeks. I'm out here in multiple capacities. So I believe this is our 9th or 10th year returning as the ICS Village nonprofit, which I co founded with Tom Van Norman. We build critical infrastructure, and we take it around to different conferences to educate folks into the space as well as I think the fun part that's becoming really relevant is help It security practitioners get some hands on to start to expand their knowledge into some of the nuances and getting past the scariness of OT security. I'm also, of course, here with my full time job as the founder and CEO of SCYTHE and as a software vendor, my job is, have you heard of SCYTHE? Why haven't you heard of SCYTHE? When do you want to buy? So doing that on the side. 

PAUL ROBERTS
"Can we offer you a T shirt?"

BRYSON BORT
"Would you like a T shirt?"

PAUL ROBERTS
"Or a laptop sticker?"

BRYSON BORT
I regret to inform you, well, we're very popular with our unicorn SCYTHE laptop stickers. It's really funny because, of course... Well, Paul, we can make that happen.

PAUL ROBERTS
I know. I'll see you at BlackHat. I'll see you at DefCon. We can do it.

BRYSON BORT
I will be at BlackHat and at DefCon in the same capacity. The ICS Village runs the ICS Village at DefCon. And that one's my favorite because folks that come there are really technical, and so they want to get hands on with all the extensive exhibits that we have there. And here at RSA, it's more of a walk by and wave at it, but we can absolutely get you a sticker. And not only that, if you want even more, we have swag.scythe.io to buy your own unicorn theme gear, and all of those proceeds go to saving the chubby unicorns. So we partnered with a Rhinoceros, an endangered rhinoceros conservation charity. So we are, in fact, saving real life chubby unicorns.

PAUL ROBERTS
I love it. I did not know that. All right, I'm going to have to check that out. I'm going to check out the chubby unicorns. Yeah, I mean, the differences. RSA and some of the other big shows. BlackHat, DefCon. Well noted and observed. One of the things you're doing out there in your very busy week in San Francisco is moderating a panel that took a look back at the Colonial Pipeline hack. We're about a year out from that. It was May of 2021 or thereabouts, that all went down and kind of taking the measure of what happened, what maybe happened that we didn't know about or didn't get a lot of media attention, but that we can appreciate here from a year away. And also, I guess what the lessons learned are. So if you wouldn't mind tell us a little bit about the panel, who was on it and what did you guys discover?

BRYSON BORT
Yeah, so I don't know if this is because I'm former government or an army officer that I've self anointed myself as the person who is going to help any agency that actually is this rated PG or rated R?

PAUL ROBERTS
This is rated PG.

BRYSON BORT
Okay, I will use a different verb. Any agency that has given itself the opportunity to learn...

PAUL ROBERTS
This is paid out of the marketing budget. I think that was the filter I ran that through.

BRYSON BORT
Yeah, no problem. I just needed to know how spicy to make my point.

PAUL ROBERTS
Yeah.

BRYSON BORT
So any agency that has faced challenges and has seen the opportunity to do better and obviously I think that was TSA with the Colonial Pipeline. And the first point I want to make so the panel was I put it together with Tim Weston, who is the Cybersecurity Coordinator at TSA, Tim Starks, who is the Editor in Chief over at Cyberscoop News. And then of course, we have to have industry representation and that was Suzanne Lemieux from the American Petroleum Institute (API). One is, what the hell happened? We get that snippets of a press perspective as everybody opines from the sidelines. And Colonial Pipeline was truly a unique watershed moment. And I think this is something that's important for the audience to get as we spend a lot of time pretty much talking to ourselves in our echo chamber of cybersecurity and really forget that about 99.9% of the world has no idea what we do other than they've seen Hugh Jackman or John Travolta in a Hollywood hacker movie once. And that's important because that's where they're...

PAUL ROBERTS
Which there's very little resemblance to what we actually do.

BRYSON BORT
I don't know. I pretty much have hacked a bank just by sipping some wine and playing with cubes.

PAUL ROBERTS
"I'm in!"

BRYSON BORT
"I'm in." It's important that I say that. I have to say I'm in. But no, this matters because and that's why actually I make sure I really include the press in this, is because they're part of the problem and the solution themselves with that education and outreach, because it's as that as a more informed citizenry exists, not only does that make us slightly safer, but it also allows government to fund and to resource correctly to things that matter to it. Colonial Pipeline being the first time that we directly affected people in their own lives, was where ransomware became a word in the layperson jargon and the real depth of the real depth of the problem became understood. And so the other thing that came out of the talk was, TSA has been panned, and rightly so for security directive 02. And what SD02 was, one, it was classified as a sensitive level, which means it could not be widely distributed for comments. And part of the reason for that was that it was prescriptive. Now, whenever government passes a regulation that is prescriptive saying thou shalt do it this way, and thou shalt do it this way, well, that doesn't usually go well. But what came out of the panel, and in fact, I didn't really even know this myself, was security directive 02 had actually been in process before the Colonial Pipeline instant. So government was trying to do the right thing, and then it just kind of happened too fast, and then they pushed that out. And of course, I have to answer the question, what was security directive 01? And that was what passed right after the Pipeline where government said, hey, we actually need data insights into what's going on, because if we're going to be making more informed, involved choices, then we need to have that data insight toward that and then the counterbalance of the panel. And this is where the API was in the hot seat. As the general industry representation, of course, is clearly voluntary, cybersecurity guidelines do not work. Turns out no company is really going to follow those in as much as they understand in their own context from a leadership perspective where cybersecurity really matters to their business, because every dollar to cybersecurity, that's not profit or an investment in the business itself.

PAUL ROBERTS
Right. A couple of things that really came out of Colonial Pipeline, I think, for many of us, first of all was, what is this thing Colonial Pipeline, I don't think many of us were aware of this is often the case with critical infrastructure. The stuff that our civilization relies on is more or less invisible to us. It's buried under the ground, or we drive by it and don't even know what its purpose is. So I think first of all, that like, oh, all the fuel we use on the east coast actually comes via this Pipeline from the gulf. And second of all, I think was just this realization that these ransomware attacks we'd all been talking and writing about could really impact our day to day lives in a very tangible way. There were obviously runs on fuel stations, especially in the south and southeast, resulting from Colonial, and certainly some very dire warnings about what was going to happen if the Pipeline didn't get back up and running. Did you get the sense that, on the one hand, it shouldn't take a ransomware attack on a critical piece of infrastructure to remind everybody why it's important to protect it. So did you get a sense in the panel kind of why it had taken that to happen for there to be movement on the government side on the directives and guidance for Pipeline owners, operators.

BRYSON BORT
Paul, humans, since the history of humankind, have been absolutely terrible at appropriate risk assessment and management. We never think it's going to happen. We never think it's going to happen to us. We never think it's going to be as bad as it possibly could be.

PAUL ROBERTS
It's like a species problem.

BRYSON BORT
And this is who we are. So government merely reflects that in that regard. I joke because it's not completely true, but I kind of joke that no one was more surprised they were in charge of that critical risk sector than TSA when it happened. And the joke is that they were not effectively resourced for that problem because at the end of the day, as an executive agency, they can only do what they are appropriated resource wise to do. CISA is three years old. An actual domestic agency whose whole goal is cybersecurity is only three years old. First time 2015, 2016 was the first time the US. Government changed its strategic approach to even considering the problem of cybersecurity for the private aspect of the country to be their problem. Previous to that, it was right. We have intelligence community and we have the military side of this, and they're doing their thing. But if you're not one of those two things, including federal civil, that's your problem. And of course, that's the soft underbelly of the attack target, is the fact that the number one economy in the world where everything is based on information technology, represents a very juicy target.

PAUL ROBERTS
And of course, there have been efforts over the years to address this in a more holistic way through new laws, data privacy data, security laws. Those have been shot down by the opposition from the chamber of commerce and other business interests. So we know that there were these DHS directives that came specifically out of Colonial. Do we get the sense that Colonial is going to lead to anything more meaningful in terms of federal regulation of critical infrastructure security?

BRYSON BORT
Well, first, going back to security directive 02 is mandatory compliance. Yeah, no longer voluntary. You will do this now for Pipeline operators, for Pipeline operators. And so it's going to be updated. And they talked about this on the panel. It's going to be declassified. They're going to pull back some of the more incorrect mitigation pieces that they pushed. The relationship with the asset owners industry, they are focused on that and there is a dialogue going between and that was very apparent on stage. Certainly, again, API representing the industry, but as a trade association, effectively what they do and that level of the collaboration that clearly had prepended the panel for them to be able to talk the way that they were between even them on stage. And I look forward to when the recording will be available from RSA, because I think having been here now three days, it was the most attended panel or talk that I've seen in the three days, we were standing room only. So I know that there is an even larger demand for everyone that wasn't able to be here in person to see that, be able to do that. So that video will be coming, and it's the dialogue and a lot of the revelations I cannot emphasize enough. We're truly interesting from all three sides.

PAUL ROBERTS
So can we talk about the attack a little bit, or what we know about the attack via some really good reporting? We know what we suspect, or we know it was the dark side ransomware group kind of cyber criminal group, kind of ties to Russia and former Soviet republics. We know that. Or if I recall, the initial access was through a vulnerable VPN concentrator that Colonial just actually wasn't even aware of, had just overlooked in some of their own assessments. Did we learn anything new about the specifics of the attack, how it played out, or what happened again after that sort of initial access?

BRYSON BORT
No, frankly, I think we use the summary of what happens to set the foundation for I think the more interesting parts were what does this mean to critical infrastructure at large? What's next? What have we learned and improved from? I think the key foundation point to take away is a private company unilaterally made a decision that had national impact. The entire Southeast was out of oil and gas for five days. And one of the things I particularly picked on was the comment that was unexpected, where we had the run on gas stations from places that weren't even affected. And I think that shows a lack of how important this is and the psychological component, that a fear that affects a population at large. And that was, again, part of why I made sure that the press was a part of that. And then the key element, the operational technology environments of Pipeline operations were not directly affected by the attack at all. It was only the information technology environment. It was the enterprise environment. Now, think about that. We had critical infrastructure operations go down, and I hypothesize more out a sense of fear of what might happen versus technically what did happen.

PAUL ROBERTS
That's right. Which shows that from the attacker standpoint, you don't necessarily need to compromise the OT system. If you compromise and the IT system, they might shut down the operational technology just out of caution. You've created enough uncertainty about what level of access they do have that they might take extreme steps to kind of protect themselves or protect their customers, which is a really interesting point. Your expertise, obviously, is on the OT side, and Colonial is interesting because it's the most prominent attack on industrial control systems. But as you pointed out, it actually wasn't an attack on industrial control systems, just the company that operated the critical infrastructure. So I guess is that risk still on the OT side, still real? What do we know about that particular risk, maybe not ransomware, but just in terms of overall cyber threats.

BRYSON BORT
So first I'm going to point out it is not the largest or the most well known attack that has had the same effect. Aramco in 2012 with Shamoon

PAUL ROBERTS
Shamoon, yeah.

BRYSON BORT
Was 35,000 enterprise desktops becoming paperweights overnight. And that affected operational technology because literally the business could not function. The second part is I've given two talks in the last three weeks with the security manager from Oklahoma Gas and Electric, Ian Anderson. The last time I gave this talk was five days ago at SANS ICS in Orlando. So I know SANS is going to be publishing that video. And what we did is we broke down the attack perspective on critical infrastructure. And the first thing I want to point out is every company has a critical infrastructure or is a part of it. The easiest point to make is where do you think your electricity and water comes from? Let alone when you start thinking about building systems, HVAC, physical security and so on. Those are all industrial control systems that affect your environment. But again, going back to Colonial Pipeline, that's actually the most common approach that we see. It's not going to industrial control systems or operational technology directly, it is going to the easiest thing, which is I would say a hacker can't hack what they can't touch and IT by definition is internet facing, which is what makes it so easy to get access to that. IT is the primary ingress, the starting point they iterate through lateral movement to the first crossover, which is what I call the beach head and that is human machine interfaces or distributed control systems, higher level industrial control systems that sit in the operational technology environment. Typically that is a DMZ. But there is that touchpoint and here's why it works so well. Besides the fact that there's that touch point to be able to get to is the operating system that they run with is the same kind of operating system you already see in the IT environment. And HMI would run Windows7. So not only is it something that I can see, it's typically something that's not supported and it's incredibly vulnerable. And here's the best part because of course the third part of that are the traditional operational technologies, the programmable logic controls that are out there changing valves or adjusting things and affecting the physical world.

PAUL ROBERTS
Actuators and stuff like that.

BRYSON BORT
Yes, exactly. I don't have to speak any of those custom protocols or any of those things because that's already organic to the environment. That's literally what those beach head systems do. So we saw this with the Oldsmar water facility attack in Florida. Again VPN into Team Viewer. Team Viewer had access to an HMI and they just slid a bar and suddenly I poisoned the water supply.

PAUL ROBERTS
Right. 

BRYSON BORT
That's the setup because that's what those machines are there to do is make it easy...

PAUL ROBERTS
That was when the monitor actually was observing this as it was happening and was able to...

BRYSON BORT
Watch the mouse cursor go across the screen and slide it over. And he was like huh, I didn't do that because I'm right here.

PAUL ROBERTS
Right.

BRYSON BORT
And I wouldn't do that.

PAUL ROBERTS
Fortunately for everybody really?

BRYSON BORT
Well, no. So let's get out of the fear, uncertainty and doubt part of this. So all too often it's like oh my gosh, someone's going to kill us. Or in this case, somebody literally did change the water processing to a point that death would have resulted had that been introduced in the environment. But here's the thing so we know for the last twelve years there are publicly documented the government has released these attacks. Attack is strong, because attack implies impacts of damage. Where there have been campaigns against critical infrastructure by the Chinese, by the Russians, and by the Iranians. And here's the thing not one of them has led to the most obvious results of any operational technology tact versus IT, which is the fact that in IT data gets so encrypted in OT somebody gets hurt or killed. And that has not happened once. The reason that I believe that has not happened is because that's not what they're trying to do. They're trying to learn and test and in this case measure the response. So when you look at how a water processing plant works, yes, had that been allowed to happen, an amount of water would have been processed to a poisonous level. But anybody who knows how those work knows that there's a secondary physical backup where that goes into a storage for 24 to 72 hours, is physically tested then before it's released into the water system. So I believe the attacker knew that and was measuring the response.

PAUL ROBERTS
Interesting. So I guess one of the things that's a little, I guess not distressing, but it seems like when you read about these attacks, Colonial or what have you, or when you read alerts like the one CISA put out about the Chinese nation state groups and the work they're doing, often the types of things they're warning companies about and advising them to do really amount to sort of basic blocking and tackling, make sure there are no remotely exploitable CVEs in your environment, make sure you don't have exposed remote access, remote control, Citrix or some VPN that people can take advantage of, use strong authentication, that type of stuff. Is that it, is that 90% of the problem is just getting these critical infrastructure owner operators to just do the basic blocking and tackling and maybe tracking it and monitoring it or is there heavier lifting to be done?

BRYSON BORT
So, first difference between a traditional operational technology environment from a security perspective versus an IT is you mostly cannot patch. Whatever vulnerability is understood or found will live there. And industrial control systems how do I know something's in ICS? It's at least 20 years old, which means it has a long capital life cycle. It's expected to be in the environment for 20 to 30 years, which means whatever that vulnerability is, is what you're going to have to work around. That is a substantial difference. You cannot just patch it. 

PAUL ROBERTS
There's a damn out in the West Coast running Tesla equipment from the late 19th century.

BRYSON BORT
And by the way, when we mean Tesla, we mean Nikola Tesla.

PAUL ROBERTS
Nikola Tesla, that's right.

BRYSON BORT
No, this actually ties to what I do full time with SCYTHE. My recommendation for everyone is you need to test whatever it is. It has to be tested. You can't have things that you assume work a certain way, whether that is a configuration or whether that is the security control or whether that is some processor policy that may or may not actually be followed by personnel. And that's part of the tension that we also see between IT and OT is that cultural bridge that's still there. Security and OT, I joke, is about 20 years behind what IT was. And hell, look where IT security is.

PAUL ROBERTS
Still a lot to do, still a lot to do. Okay, so for folks who haven't seen who weren't at it or haven't yet seen the conversation, what's the top level takeaway? Let's say you're a large enterprise out or you're a critical infrastructure owner operator out there, maybe not in the Pipeline space, but some other space. What is the takeaway for you from this presentation? What should you be learning from Colonial?

BRYSON BORT
Just like target was the canary in the coal mine for IT security.

PAUL ROBERTS
Yeah, and third-party risk. 

BRYSON BORT
And Colonial... And third party risk and critical infrastructure is a huge problem. Colonial is that same canary in the coal mine for OT. There will be more, and it is coming right. In fact, this is what's amusing to me, or I don't know, there's a sense of something. And the last time I was at RSA was two and a half years ago, and Chris Krebs and I gave a talk on stage and predicted ransomware. And the sad part is we were right. I wish we weren't. And what makes this work is the economic utility, your organization, no matter how big or small you think it is, is worth something to somebody, because that's how ransomware works. And we have not solved the fundamental operator and economic model of that. It's not a technical challenge in as much as that's why they are motivated to do what they do. The technical challenges on your organization, to treat this as a priority and make a difference.

PAUL ROBERTS
Bryson, is there anything I didn't ask you that I should have?

BRYSON BORT
I don't know do you want a grab bag, what's my prediction of what bad thing is going to happen?

PAUL ROBERTS
Yeah, that's good. I'd love to hear your prediction. If you were accurate two years ago on the ransomware and critical infrastructure piece, look ahead another year or two, let me know and let's hear it.

BRYSON BORT
So I'm going to do a follow on of the ransomware prediction, which is that tied to the pervasive nature of Internet work computers in our lives and everything. Right. Even just the difference from three years. There is more computer online in parts of ourselves.

PAUL ROBERTS
Yeah.

BRYSON BORT
That sometime in the next five years, somebody is going to literally wake up, go downstairs, get their coffee, head out the door, open the car door, get in, turn on the car to go to work, and that infotainment screen is going to pop up and say, you've been ransomware'd, pay me this much to get your car back.

PAUL ROBERTS
Yeah, absolutely.

BRYSON BORT
So the B2B ransomware model will become a B2C ransomware model.

PAUL ROBERTS
Right. Internet or your refrigerator. Right. Or whatever. Right.

BRYSON BORT
I like the car story because...

PAUL ROBERTS
Car story is a great one.

BRYSON BORT
Ransomwaring the refrigerator. So it's just not going to be...

PAUL ROBERTS
Right. And one of the really interesting things about that, too, is that, first of all, the implications of that for that particular car owner not being able to get drive to work or drive their kids to school are huge. The financial implications. And also my sense is that automakers are not prepared for this. Right. That if these systems needed to be these cars needed to be re-imaged in the driveway, that Tesla or Ford couldn't support that at the scale they might need to. Right. If this were a widespread attack, not just one owner, but tens of thousands in a geographic area.

BRYSON BORT
I do want to give credit the car manufacturers, because this is one of the things we did at Graham was we were actually some of the first to hack cars. We just didn't go public about it. Obviously, that is a Wired article in 2015 reference. And I remember at that time yes, I remember and I appreciate that they created a market. In fact, I was interviewed at DefCon that year with one of our car hacking exhibits. I remember the press going, so where's this going to go? And I'm like, well, we're pretty much doing this for fun, because when we go and talk to car manufacturers, they tell us not $1 more for security, go pound SANS. After the Charlie and Chris article came out in 2016, we started getting large dollar contracts solving that problem. And so I have personally seen car manufacturers take this problem seriously. The reality, just like all of this, is that just becomes the test matrix for the attacker. So whatever you have achieved well, I'm not going to deploy something until I already know it's going to work. But I just wanted to give credit that I have personally seen the car manufacturing industry almost across the board take this seriously.

PAUL ROBERTS
So you think they'll be ready?

BRYSON BORT
Again, the attacker is going to find the hole that they are ready for. But I just wanted to say that they have been investing in making it a harder problem.

PAUL ROBERTS
Yeah. Preventing the attack is hard, if not impossible, right? To be perfect. But being prepared for the eventuality of attack, that's something that's much more doable. Anyway. Bryson Bort of SCYTHE, thank you so much for coming on and speaking to us on ConversingLabs. I really appreciate it.