Adam Gleeson:

Hello and welcome to our latest podcast. Today with me I have Peter Mackenzie, who is the Sophos Director for Incident Response. We’re going to be talking about responding to a ransomware incident. Hello Peter, and thank you very much for joining me today. Would you like to just introduce yourself quickly to the people at home?

Peter Mackenzie:

Yeah, so Peter McKenzie, I work at Sophos. I run our  incident response services as a global team, around 50 people dealing with a variety of different cyber attacks. It could be ransomware, it could be a network breach, or maybe even a nation state attack.

Adam Gleeson:

Thank You very much and hopefully I need no introduction. Adam Gleeson. I’m the Vendor Alliance Manager for CyberLab. So let’s start off with just setting the scene, Peter. So why are we here today? So ransomware is not the only type of cyber incident out there. I’ve talked about this on other podcasts about the different types of cyber incidents that you can have, but it’s by far the most impactful cyber incident that modern businesses face, isn’t it?

Peter Mackenzie:

Yeah, there are obviously a wide variety of attacks. You could have someone stealing data, you could have someone stealing your personal username password, you could have people defacing your website or taking down your external websites and various other things. But the one that always hits the headlines is ransomware because of the impact it can cause, it’s so disruptive, disrupt to businesses, so disruptive to people disrupt it can stop businesses operating and to some extremes that they may never recover.

Adam Gleeson:

An incident response specifically is something that, it’s one of these things that not everyone thinks about. And why in your opinion, do we need to have a plan around response?

Peter Mackenzie:

Well, attacks have changed. If you think of malware and viruses 10, 20 years ago there were a bit of code that got onto your computer, maybe you downloaded the wrong thing or went to the wrong website and they did a job and they caused some damage and that was basically it. And we have antivirus software that was introduced to detect those things and remove them modern day attacks like this while they use malware and they use tools like that. What you are really facing is a human attacker that’s gained access to your network. And just like if you were an admin on a network, you can do a whole variety of different things normally for good, but of course you can do it for bad as well and you can cause a lot of impact. So when you are fighting an attacker, it’s not just about removing the malware they’re using. You’ve got to identify what accounts they were using, how did they get into your network, have they set up any back doors so they can come back in a week later? You have to understand everything they’ve done so you can make sure you can patch those vulnerabilities, change the passwords of the compromised accounts. It’s not just about deleting a single file, they’ll just come back again.

Adam Gleeson:

But it’s about being prepared for this to happen because when these sorts of things happen, it’s really, really stressful, especially for the IT department who are the ones, they’re the ones that are in the site sort of thing of fix it and fix it now sort of thing. And if you’re not prepared, then obviously that’s not great.

Peter Mackenzie:

Yeah, I mean it’s hard to not sound all doom and gloom, but these things will happen especially if you don’t think they actually will if you’re not taking the right basic precautions to secure your network. I mean what we find with ransomware attacks, we often ask the victim, when did this attack happen? And the typical answer we’ll get is today when they saw the encrypted files and the ransom note and we have to then through our investigation highlight that the attack started two weeks ago, that’s when they got in. What happened today is the attacker announced it to, they left their receipt on your computer. Now think about what if they hadn’t done ransomware? What if they had just taken your customer data for example? Would you have even known they had been there? And the answer is no most of the time.

Adam Gleeson:

You just touched upon it there actually, that there are numerous attack vectors that can result in a ransomware attack. And again, as you alluded to there, that by the time it’s announced that they’ve already been in there causing damage, a lot of damage. And of course the scale of these things can vary as well can’t they? There’s a big difference to the level of disruption. I’ve spoken to customers myself where they were sort of say, oh, we had ransomware last year, we just imaged that machine. Whereas the types of things that you are dealing with day in day out are vastly larger than that.

Peter Mackenzie:

Yeah, I mean there’s some broad categories of ransomware and it’s almost a pet peeve of mine when people say, oh, I had ransomware, it wasn’t that bad. And it’s like, yeah, but that’s because dealing with one type of ransomware, so the ransomware that we sort of experienced a lot like 10 years ago is stuff that you get a phishing campaign or a spam campaign and there’s a malicious file attached or email. This rarely happens anymore, but you get a malicious file that is the ransomware and it’s an attachment to your email and you run it on your machine and it encrypts your holiday photos, your C-drive, and that’s kind of the damage. And you go tell your boss and they go, okay, we’ll give you a new computer. Try not to do that again, modern day ransomware attacks, you do still get those ones that are sort of isolated to individual users, individual machines because they’ve gone and downloaded some pirated software or they’ve used a keygen or they’ve gone to, again, a compromised website and it’s affected their computer.

Then you’ve got the ones that are run by humans by actual hands on keyboard as we call it. They’re on the network, they’re preparing their attack for days often, and they’re not just launching their ransomware, they’re first finding your backups and wiping them out. They’re stealing data normally one to two days prior to them actually launching the ransomware. They’re go and steal gigabytes or even terabytes of your company data, your customer data and then they will deploy the ransomware to your network. They may use GPO, they may use PowerShell lots of different ways, but the point is it’s not sort of spreading a virus, it’s deployed like an admin would deploy a new bit of software to your machines and they control it and that can cause huge amounts of more impact than just a user running it on their machine. Of course.

Adam Gleeson:

Different types of, so we’ve talked about the different, you were just sort of alluding to the different types of attack and we’ll talk about them a bit later and when we start looking at what an ins response plan actually looks like, but what does that dawning awareness of ransomware look like? What are the likely indications? So at what point do we go from it’s business as usual to, oh no, we’ve got a problem.

Peter Mackenzie:

It varies. Most of the time what we see is that a user will phone up their IT help desk and say, I can’t access this file, or this application has stopped working. And then the admin will remote into the machine where this things hosted and they’ll go, why have all these files got weird extensions and what’s this Read me file on the desktop. And that’s when they’ll realise that an attack has happened, but they won’t know the scale yet. And then they’ll start looking at other machines and they’ll go, okay, they’re on here as well. Or what they might do is they might go into vSphere and their virtual environments and they might go, why are all our virtual machines turned off? And that’s because the attackers got into their SXI host or something like that, turned all the virtual machines off and then encrypted the raw VMDK files and basically without getting too technical shut down the entire business and all of their virtual machines are now gone crippling a business normally

Adam Gleeson:

As an ex-VMware fanboy, the thought of an ESXI host or environment being encrypted,

Peter Mackenzie (08:51):

It’s a growing trend. We see it on almost all of-

Adam Gleeson:

– it would be terrifying. It would be terrifying. And again, I mentioned this occasionally, if you’ve got your backup server on a virtual environment, get it off and put it on a physical server.

Peter Mackenzie:

Yeah, no, we see SXI being targeted in almost all of the main, most damaging attacks we investigate, they’ve gone after the virtual machines.

Adam Gleeson:

You just more bang for your buck, I guess.

Peter Mackenzie:

Yeah.

Adam Gleeson:

So the unable to access file service or services becoming unavailable, you might even get a ransom note or a ransom email. I’ve heard of that.

Peter Mackenzie:

Yeah, your website might be down as well. That’s often one thing we look at when we are getting a new client to work with to understand how much panic they might be in. We go and see if their website’s still up because if their website’s down, that really puts people into a panic. But yeah, and you’ll have that ransom note which will have some instructions and generally a threat saying we’ve taken data and we are going to publish or your secrets and or your customer data and you have maybe a week to respond or two days to respond. And normally there’s a link to a dark web onion address that you access via the tour network and then you can go onto there and you can chat to them. You can have a conversation with them and they’ll typically tell you, here’s the evidence of the data we stole and this is how much ransom demand you’re going to need to pay.

Adam Gleeson:

Terrifying stuff really. So this has happened to us, what actions should we do? What sort of things do we start doing immediately?

Peter Mackenzie:

Assuming you’ve just found out that the attack is potentially still happening, still running right now. It’s not something that you want to investigate two weeks later, but the first thing to do is probably going to want to take your critical systems offline because if they haven’t been hit already, which they probably have, you want to protect them. So anything that is critical to your business or even just everything, get them offline. If you’ve got ways to isolate them, so various security products have options to isolate machines, that’s a good way of doing it. You can then keep investigating those machines while they’re isolated, but if you don’t know what to do and you don’t know what tools you’ve got, just turn things off-

Adam Gleeson:

– shut it down.

Peter Mackenzie:

Just shut things down.

Peter Mackenzie:

Cut the internet, put firewall rules in to block all traffic, turn machines off, get your backup servers offline and go and check if you’ve still got your backups. We’ve worked with clients that have assumed they had backups and it wasn’t until they went and checked and realised that the attacker had deleted them.

Adam Gleeson:

Yeah, I’ve had to have unfortunate conversations with customers that have been in that situation as well. It’s incredibly disheartening here. I think when we were talking, actually, you said that there was one chap actually when you asked him to go and check here.

Peter Mackenzie:

Yeah, we won’t get too graphic, but yeah, so it was a riot ransomware attacks or anyone that knows that one even knows a few years ago now, but it was a small organisation. They only had six servers. I think it was a $200,000. It was a US company and yeah, $200,000 ransom and I was on the phone with them and said, have you checked your backups? Have you got backups? And he was like, yes, yes, they’re fine. I’m like, have you actually checked? Because these guys, they almost always delete people’s backups before they launched the attack. So went and looked while we’re on the phone and then I started hearing him sort of hyperventilate and get a bit more visceral.

Adam Gleeson:

As the dawning awareness and his anxiety went right through the roof. It would be terrifying. It’d be a horrible situation to be in. So I kind of know a little bit, there’s something that I’ve learned doing this podcast series actually that some of the other steps, so you’re absolutely right. I would absolutely say stop the attack getting worse, shut things down. One thing that we did touch upon, I’ll mention it now and we’ll come back to it, is around the old way that we used to respond to a virus outbreak would be to start flattening stuff, delete format, hard drives, re-image, reinstall the operating system. What are the reasons that we shouldn’t do that nowadays?

Peter Mackenzie:

Well, I mean I was never a huge fan of that anyway. That’s always the bearing your head in the sand kind of approach. Like if you want to get technical, you could reinstall windows, you could do everything you wanted to do to feel safe on that machine and there’s still technically some bits of malware that could exist even after you go and reinstall windows and stuff. But ignoring that scenario, the problem with it nowadays is it’s not so much a problem of what’s on the machine, it’s how did they get access to the machine in the first place. If we say so most people here will be hopefully familiar with remote desktop protocol, RDP or some sort of firewall or VPN if there’s a vulnerability or some sort of open door that allows the attacker in, then they get in and they do whatever damage they want to do and you can go and wipe those machines. But unless you understand the root cause of how did they get in, even if you’ve gone and wiped and put all new machines on their modern operating systems all patched up, if you haven’t found that initial access method, that root cause, someone else can come in a day later and do all the same.

Adam Gleeson:

And they sell the access on the dark web as well, don’t they?

Peter Mackenzie:

Initial access brokers.

Adam Gleeson:

Oh, you can get into this organisation by doing this.

Peter Mackenzie:

Yeah, there’s a whole black market of initial access brokers. Their job is to flood the internet with scans and brute force attacks and try and get into any system they can and once they get in, they’ll set up an admin account, they may instal some sort of backdoor, and when I say backdoor, in previous years, that would be a bit of malware that would allow them to access the machine remotely. Nowadays, while they still do that as well, they will often just instal legitimate remote admin tools like anydesk or screenconnect or splashedtop all of the things that many of you will use and it’s perfectly all right to use, but the attackers use them too. They know they’ll blend in with what you’re doing. You see a server with screen connect installed and your IT team use screen connect. You’re not going to give it a second glance, but it was installed by the attacker potentially. So that’s what the initial access broker does. They get access, they get the credentials, they set up some sort of re-entry method and then they’ll go onto a market and they will say, I’ve got access to this size organisation in this industry, in this country. I want a thousand dollars, $10,000 or whatever it is.

Adam Gleeson:

It never ceases to sort of amaze me just how an entire industry, a criminal industry has now cropped up around this. And you mentioned before you have that sort of dialogue with the attacker via the links that they give you…

Peter Mackenzie:

Yeah, the attackers, I mean often they outsource their negotiations as well. So you’ve got initial access brokers to get into your network, they go sell that access. Then you may have the ransomware person that’s going to actually do the attack, but they’re not the ones that actually developed the ransomware. They’re using ransomware as a service as well. So someone goes and creates the ransomware, creates a nice web dashboard. You log in, you can configure your ransomware, you can set different demands, you can define how it works. Is it a Windows ransomware? Is it a Linux ransomware? You can configure all of that and then your job as the ransomware attacker is to go and get that onto machines and run it. Then if someone wants to negotiate your outsourced negotiator, we’ll go and have a conversation with them. You may even outsource a second group of people to start harassing the victim, phoning them up, emailing them, phoning their customers and their partners and making threats. It’s an incredibly large sort of business nowadays.

Adam Gleeson:

I mean from one perspective it’s pretty impressive, but doubly terrifying that they’re that organised now.

Peter Mackenzie:

It is incredibly organised, and so this is going back to 2022 now I believe it was, where there was a ransom group called Conti and they were well known for having Russian origins, lots of Russian people involved, not Russian government, just Russians. And when the Russia-Ukraine conflict started, they made a press statement saying, we support Russia. But it turned out a lot of what they call their affiliates, the people that are doing the attacks, a lot of them were Ukrainian and they said, well no, wait a sec.

Adam Gleeson:

Yes, I remember reading about this now!

Peter Mackenzie:

We’re not okay with this. So the story goes that one of these Ukrainian affiliates that is a bad person stole a load of data from the Conti infrastructure and they released all of the details of their files and thousands of chat messages, internal chat messages from the Conti group and it was incredibly revealing that they have teams, they have team leaders, they have monthly targets, they have recruiters. It’s a business, you had a monthly salary and stuff like that. It is incredibly organised!

Adam Gleeson:

Absolutely fascinating, but as usual, I digress and I’ve moved away from, so we were talking about what actions should be taken, so stop the attack getting worse just to buy yourself some time and shut these effective machines down. I’m not going to dwell upon the next couple of points I can just refer back to our previous podcast. So contacting legal representation is very important to do fairly quickly as well to understand if any data has been exfiltrated, you need to look at the data governance angle and see what the start looking at how are you going to manage what data may have been exfiltrated or shared without authorization. Then you need to start looking at your cyber insurance company and contacting that. Now, one of the interesting things that as a software vendor that you said is that, and it stands to reason that you should then contact your security software vendors.

Peter Mackenzie:

Yeah, go in order then. So why would you contact legal as you say, the data that may have been accessed or stolen? PII – personally identifiable information. We see a lot of victims that have large amounts of data stolen thousands, hundreds of thousands of files and we look through it and we tell them, this is your employee data. This is customer data and that has ramifications. They may have to report it to certain authorities, it depends which country you’re in, but generally if you’ve had that kind of data stolen, you have to report it to someone and you often have to do it in a certain time range or fines occur. And most businesses we talk with don’t realise the impact. Often when we say they have access to this file server and they took a terabyte of data, they go, well, these are all old files and things like that. We don’t care about those so much. What we find though, and this happens very regularly, one of the first things we look for are passports. Because in almost every business you’ll have executives and their executive assistants or whoever has got a copy of their passport and they’ve kept it somewhere so that when they travel or whatever, they’ve got it and they can have.

So we often find the passports of their executive team are in this data that has been taken and that obviously then impacts those people at the top immediately they have to do even more because their passport could be now being used fraudulently by whoever. So you need to get the legal team involved because you may have to take certain steps at certain times or face fines and it could be affecting you personally as well. Then as you say, cyber insurance – now not everyone will have cyber insurance. Some will have it as part of a wider business IT policy. Some may have dedicated cyber insurance, but you need to understand when you have an incident, how do you invoke that cyber insurance carrier because they will likely have rules about who you can use, what legal teams you might need to involve and things you might have to do in order to get a claim approved. So you should absolutely involve your cyber insurance as quickly as possible and then yes, for the security vendor or whoever it is that’s going to actually come in and help you because realistically, even if you’ve got your own in-house security team, your security operation centre, your SOC, they didn’t stop the attack, you’ve been attacked, something’s going on, your team didn’t stop it, and for that reason alone, you might want to get some outside unbiased opinion of what happened. So who is that going to be? Is it your endpoint security vendor? Do they actually offer that kind of service or do you need to go to someone the insurance dictates? Either way, these are the type of things you should identify before an incident. They’re not the things you want to work out on the day because essentially we get busy IR vendors and there are a lot of times where vendors have to say, we can’t help you today. We are simply too busy. You have to go to another vendor and they may be busy as well and all of that is costing you time.

Adam Gleeson:

That’s quite a scary prospect. I would just kind of assume that as we do as organisations that serve customer needs, that you do whatever you can to assist your customers. But of course, especially if over the Christmas period for example, it’s rife because they know offices are closed, that people are not there, people are not watching stuff. So I would imagine that that was a very busy period for you guys.

Peter Mackenzie:

Yes, December. is always busy for us because the attackers take advantage of admin admins being off and things like that. But yeah, I mean  incident response is not easy and it’s not quick. It takes hundreds of hours from often a large team, sometimes going on site, sometimes working with law enforcement. It’s a lot of intense work and that’s why we can’t promise availability sometimes, and this is true of the whole industry and we’ve had scenarios where a hospital has come to us and saying, we need help. We’ve asked three other vendors, none of them can take us. Can you help? Kind of thing. The attack was two days ago and this hospital has effectively shut down while they try and find someone that can help them. So do you have cyber insurance? Do you have an IR retainer? These are the things you should be thinking about

Adam Gleeson:

And these things that we’re talking about need to happen in quick succession as well. There’s lots of things that we’re talking about here, and this is why this highlights the need to have an incident response plan before or to at least think about what you’re going to do and understand the order that these things need to happen. It might not be the same for every single organisation. It might not be the same for one cyber insurance supplier might have a different set of requirements for the legal department or for the incident response supplier that they want you to use. These things may differ from company to company, so we need to think about this stuff beforehand. There were just a couple of other things that I wanted to touch upon. So we mentioned backups as well. So this is something else that it’s important to think about your backups and make sure that they’re protected because that’s your only ‘Get out of jail card’, ‘Get out of jail free’ card, which I’ve said this before, but I’ve seen customers who didn’t have good backups and it was just devastating exactly as we sort of talked about there.

Peter Mackenzie:

Yeah, I mean the general rule for backups is three different copies of the data using two different systems and one is offline. Correct?

Adam Gleeson (25:33):

Yeah. 3, 2, 1 rule. Good old Veeam. So you also might want to think about disaster, maybe not disaster recovery. It depends on the size of your organisation and your appetite, your budget appetite for doing that. But business continuity is also something that everyone should be thinking about. These incidents stop things dead in the tracks and they stop the ability, they remove your ability to transact business as usual. So being able to put something in place to allow you to keep going and make sure that you’re not just all of a sudden everyone downs tools and you can’t do it.

Peter Mackenzie:

I mean most of the examples we see, so for a typical business, some of the biggest problems they have other than obviously they may not be able to respond to customer requests, maybe their phone system is down or something like that, but we see their finance systems being down so it’s getting to the end of the month and they start realising we can’t pay our staff. That normally causes quite a bit of panic, but then there’s just so many other things that you don’t think about that one of our partners that you work with, they connect to your network and they’ve now heard that you’ve been attacked and they’re going, well, we’re not connecting to your network again until you can provide some evidence that you are now safe. “We are not working with you until you’ve done that.

Adam Gleeson:

No, why would you? It’s like, I don’t want that. I mean it’s like covid all over again. It’s like I don’t want to go anywhere near you because I might catch it. Okay, so let’s say that, we’ve had a ransomware outbreak within our environment. We’ve identified all the machines that we think have been affected. We dunno what to do with ’em. So we’ve just shut them all down. We’ve contacted our cyber insurance and legal maybe the other way around or what you, if we then at the point where we’re reaching out to an incident response supplier, what does that kind of engagement look like?

Peter Mackenzie:

So it depends what kind of prearrangement you might have with them. You may have a swim retainer where you’ve got a bucket of hours that you’ve already paid for or it may be that you’re just reaching out to someone you’ve never spoken to before and you have to pay for their service, pay for their hours. But once you’ve done that sort of financial and initial scoping conversation, so you will typically speak to someone that will ask about the incident, what type of incident it is, when did it start, what kind of impacts have, what are your sort of objectives? Is your objective just to get things back up and running? Do you want a forensic investigation? Do you have ransom negotiations? So they’ll try and get a bit of information about what you’re actually hoping to achieve and that will determine maybe how many hours are needed. Once all that’s happened, which can be minutes an hour to happen, you’ll typically sign some sort of documents. And that’s when my team would typically get involved. And we are looking, once we know you are our customer, we would be wanting to have a kickoff call with you. And that means your admins, your senior management. Normally within an hour or so, we are typically ready to go as soon as possible. We give you an hour to make sure you can get your people on the call typically. And then that kickoff call, again, it’s not just a meet and greet, it’s tell us what happened as far as you’re aware. Give us those objectives that you want us to help you with. And we will typically be either at the same time or just after that meeting, we’ll be deploying tools to your network collecting evidence. And for most incidents we will be firefighting to start with. So the first thing is to stop any more damage occurring. So we want to get protection, Sophos or whatever, whoever the vendor is. We want to get your endpoint protection product in place. You want to probably put isolation rules in your firewall, things like that. We want to put the fire out. That’s all in the first few hours, first few days. That triage stage that we normally refer to it as where it’s just about stopping further damage and removing access, identifying compromised accounts, identifying IP addresses that need blocking, things like that. How was the ransomware deployed? Finding that, putting an end to that. Then I don’t want to say it slows down, but the work takes longer. So it’s more the forensic side figure out what did they do on this machine? What did they do on that machine? Did this give them domain admin, did this allow them to zip up your files and where did they exfiltrate them from? And ultimately we’re trying to identify how do they get in the first place?

Adam Gleeson:
Excellent. So that’s kind of an enumeration phase where we try and understand what’s actually happened, and then you start to move into securing the environment. So, what sort of steps do we take then? Because as a customer, I’m going to be sat there chomping at the bit, wanting to see you getting rid of this. But of course, you have to be careful before you do that, because once that forensic information’s gone, that’s the information that’s going to tell you how they’ve got in and what they’ve actually done. So, we need to do the enumeration. Once we’ve done that, how do we then start to clean things?

Peter Mackenzie:
Well, it’s normally just getting some sort of security software installed and letting the security software do its job to start with—finding the malicious files and removing those. But then, it’s the IR team to actually investigate and say, “Right, has the attacker left anything on this machine? Do we need to remove anything manually that isn’t necessarily malicious but is part of the attack?” What most customers will start wanting to know is how they can stop this happening again, what steps they can take immediately to improve their situation. And most of the time, other than the fact that they’ve obviously had an incident they’re dealing with, the situation is already improved because they’re now aware of what can happen. They’re now looking at their systems, reviewing how many domain admins they actually have, whether they need all of them, and then they’re going, “Well, wait a sec. All these old legacy systems that we’ve kept around, knowing they’re vulnerable but we don’t actually need—this is our time to get rid of them.” They start getting rid of old systems, identifying vulnerabilities. We, or whoever’s working with them, will typically run some sort of vulnerability scan, at least on their perimeter, to go, “Yeah, look, you’ve got these firewalls or VPNs, and they’re vulnerable. Also, you’re not using multifactor. Everyone should be using multifactor. It doesn’t stop all attacks, but it’s a very impactful thing you can do for making…

Adam Gleeson:
More secure. It’s a pain in the neck. And this, I think I’m going to mention this—we’ll come onto it. I’m a big believer that many of these commoditised ransomware attacks that we see now aren’t being done by skilled operators. They’re people running off a crib sheet. They might have a bit of experience with it, but they’re not highly qualified hackers.

Peter Mackenzie:
Well, for any ransomware attackers watching, I’m not insulting you—please don’t attack me! But no, you are right. I mean, obviously, there’s a spectrum. There are some very skilled, very technical people out there that are often the ones running the ransomware-as-a-service services and developing those tools. Most of the affiliates that are actually doing the attacks, they will have specialties in certain areas, but they will also follow the sort of habits that they use each time. Like you said, they’re following a playbook. In fact, we’ve actually seen—we published, I think, five years ago or so now—there’s a couple of different groups out there where we managed to get hold of literally their scripts: “Press three to launch ransomware attack, press four to deploy ransomware.” And they are literally following a menu of steps, or a set of instructions. Most attacks, as you say, aren’t from nation-states. They’re not highly sophisticated. They’re not the absolute best in the world at doing this. They’re taking advantage of the low-hanging fruit. I don’t want to say stopping ransomware is easy, because it certainly isn’t, but a large amount of attacks can be quite easily prevented if you’ve got the right tools, the right people with the right experience noticing, “Hey, we just had an alert on this server that says someone’s trying to get my domain admin password.” A lot of people just ignore it, unfortunately, or don’t understand it. So, the attack continues because the tool being used and being detected, that’s just one of the things the attack is doing. And while that’s blocked, they’re already trying something else. So, if you’ve got someone who can understand those warning signs and take action, you can stop those attacks from getting any worse relatively easily.

Adam Gleeson:
And going back to the point I wanted to make, and it kind of touches on what you were saying about how they have team leaders and stuff like that, but they almost operate like sales teams, in that these people have targets to meet. They have to have exploited X amount of money, pounds, each week. And if you can make it harder for them to do that, they might just think, “You’re too much of a tough nut to crack.” Even though I’ve gotten in here, there’s not really an easy way for me to do this, or my crib sheet isn’t working in this instance, I’ll just go and look elsewhere.

Peter Mackenzie:
We had to change our wording we used with clients a while back because we referred to things as a “targeted attack.” What we meant by that, on the IR and technical side, was that there was a human involved. They knew who they were attacking. What many victims thought that meant was that they’d been specifically targeted—maybe because they drill for oil or whatever—and they thought, “We’re being targeted for that.” And it’s not that. Most attackers, as I say, they’re buying credentials to a business. They don’t know much about it other than they can get into it. Often, we’ve seen attackers, they’ve gotten into a network, loaded up a browser, and gone to look at that company’s website to work out who they’re actually attacking. They don’t really know or care. Their job is to go and do something.

Adam Gleeson:
It’s just a computer.

Peter Mackenzie:
If something gets in their way and there’s another victim ready to go, they’ll go, “Okay, I can’t be bothered. I’ll go attack these guys instead.” Don’t be the low-hanging fruit—that’s generally the best advice.

Adam Gleeson (36:13):
Which, again, sorry, I’ve taken this off on a bit of a track, but hopefully people found it interesting. But going back to your point about MFA—implementing that makes it exponentially harder for them to actually then ransack your systems. They’ve got to overcome that obstacle in order to progress. You mentioned about disabling accounts and credentials that aren’t necessary because they represent easy ways in. What other kinds of things do we need to do to eradicate malicious software?

Peter Mackenzie:
Your security software—Sophos, or whatever it is you’re using—should be able to do the bulk of it. But for example, most of them won’t remove ransom notes. Ransom notes are typically a text file, and they’re not technically malicious. But if all your users are about to come back to work and there are ransom notes across their desktops and stuff—A) it’s alarming, and B) if they don’t understand it, the last thing you want is them posting it on Twitter or someone asking, “Hey, what’s this?”

Adam Gleeson:
Or if they’ve got links in them, you don’t then want them going and kicking it all off again.

Peter Mackenzie:
You don’t want them joining the negotiation checks. When that happens, things tend to spiral out of control pretty quickly, in fact. But yeah, so I think if we think about—not the ransom notes so much, but an actual malicious file. So, there’s an executable file, maybe it is the ransomware, or maybe it’s just some other tool the attacker was using. If your security vendor isn’t blocking it, what do you do in that situation? What does your incident response plan say, if you’ve got one, about how to deal with that? Do you have tools in your environment to say, “I want this file gone from all my machines”? Because if you don’t have those kinds of tools, you may have to go and get a copy of that file, you may have to send it to your security vendor. You may have to wait for them to analyse it, wait for them to release an update, and then for it to get detected. All of that could take anywhere from minutes to hours to days—during which time impact is happening on your network potentially. So, do you have the ability to remove files from your network? Do you have the ability to reset passwords? Some of the things we find when we speak to people are that we say, “We need you to block this IP address on your firewall,” and they go, “I don’t know how to get into my firewall. The admin for that’s on holiday.” We’ve had customers have to bring staff back from holiday just so they can log into their firewall. We’ve had ones where they say, “We use this company to manage our firewall and they don’t work weekends, so we’ll have to wait till Monday to block this malicious IP address the attacker’s communicating with.” So, there are so many things that we as a response firm want to be able to do to make you safer, but you don’t realise you don’t have the ability to do that until It’s too late.

Adam Gleeson:
This again goes back to the need to be thinking about this stuff ahead of time to try and get out ahead of this, so you don’t end up in that situation where it’s like, “Well, we’ve just invested a load of money in an incident response solution or service from a vendor. They’ve gone through and cleared all of this stuff off, but now we can’t actually shut the door that the people got in through for two days. Someone else might be coming back in to undo everything that’s just been spent. So, there’s a lot of things to think about, and you need to be understanding to make sure you’re effective at fighting back against these things.

Peter Mackenzie:
Don’t get me wrong. You can think about it at the time. You can wait until the incident has happened, but it will cost you more. It will cost you time, money in recovery, and the recovery will take longer. You’ll have to do more than if you had planned for what you would do in this scenario.

Adam Gleeson:
So let’s start to look at some of the recovery things. And I think one of the important things to note is that not all incident response suppliers will actually restore your systems for you necessarily.

Peter Mackenzie:
Yeah, no one’s going to know your network better than you. Your applications, the configurations, what your staff do, your random developer servers, your random database servers—it’s going to be very hard for an outside company to come in and rebuild your databases, for example. So recovery from a DFIR (Digital Forensics and Incident Response) firm is normally about getting your security solutions working, providing at least advice on how to configure your firewall safely. They may help you rebuild new domains. Often, customers will go, “Well, we just don’t trust this network at all. We’re going to start from scratch: new Active Directory, a whole new network, all new machines.” And they’ll help you with that. But normally, recovering your actual applications, your settings, and those kinds of things—that’s down to you or the IT partner that put them in place in the first place.

Adam Gleeson:
So, again, it goes back to having that plan—what are you going to do?

Peter Mackenzie:
Do they work nights?

Adam Gleeson:
Yeah, I suppose that’s very important. Alright, I think we’ve covered most of the areas I really wanted to. Now, during our preparation for these two sessions, there were a couple of amusing anecdotes and experiences you’d had. Would you mind sharing them with the audience?

Peter Mackenzie:
I mean, I’ve been doing this for a decade now—so I’ve got an endless list of things we’ve seen. I think the one thing that no longer shocks my team, but everyone, anytime I say it, the normal response is, “Really? That’s still a problem?” is people exposing RDP over the internet. The ransomware deployment protocol, as we normally refer to it, is people having RDP open to the internet. It’s normally, if you open RDP to the internet, it’s normally seconds before someone identifies it and starts trying to attack it. But we’ve had so many incidents, and I’m not trying to make light of things, but you’ve got situations where someone’s gone to the effort of creating a detailed incident response plan and put it on their server. They’ve been very proud of it, and we ask them if they’re following it. And they say, “We can’t tell you because it got encrypted. It was on one of the servers that got attacked.” That is unfortunately quite common. I remember a few years back, if you’re familiar with coin miners…

Peter Mackenzie:
So coin miners—they take up a lot of your CPU, slow down your machines, and try to mine for cryptocurrency. Generally, the damage they cause is slowing down machines and maybe increasing your electricity bill. And we had a company, I think they were in South Africa, contact us saying, “Our machines are running slowly and we’re getting all these infection detections about this thing.” We looked and said, “Well, yeah, it’s a coin miner, and it’s all over your network.” We looked at their policy settings, and their exclusions, and they had put an exclusion in for the entire C drive of the machine. They’d added a comment saying, “Scanning is slowing down machine.” So, they had effectively turned off all scanning of the C drive because they decided the security software was slowing it down, but then they were contacting us because of the coin miner infection that was slowing down their machines far more. So yeah, we see a variety of bad decisions. We see things that people didn’t think about. Like, “We put our backups on this virtual machine,” well, the attacker got into your host server and deleted your backup server. We actually saw where they had offline backups, and we saw the attacker access the machine that had access to the offline backups. It was a tape drive. They had unfortunately left the tape drive connected with the backups in and on. We saw the attacker download the instruction manual for that tape software so they could work out how to delete what was on the contents. Yeah, it’s a crazy world of things we’ve seen, and there are so many things people don’t think about when it comes to the impact of an attack. As I say, you hear your files have been taken, but you don’t realise that’s my passport in there. They were a frozen food supplier, so big warehouses, big freezers, lots and lots of frozen food, large amounts of stock. Basically, we were on a phone call with them—it was a kick-off call with them talking about the incident—and this was a large company. They had, I think it was about 50 people on the call, plus my team. And so, it was already a little chaotic. Trying to control 50 people in a panic is quite difficult. I remember during the call, you could see someone burst into the room and start saying, “The freezers have stopped! The freezers have stopped!” And the panic—everyone got up, and they were like, “Get fans! Open doors!” They realised within the next couple of hours they were about to lose millions of pounds worth of stock as it defrosted because their freezers had been encrypted, the machines that controlled them.

Adam Gleeson:
Again, it’s one of those things that’s amusing looking back, but I can imagine that was quite stressful for all involved. I don’t know whether you can talk about this or not, but I remember when we were talking, and it goes back to this making life for the attacker harder, and it was an inadvertent sort of…

Peter Mackenzie:
Yeah, I know which one you mean.

Adam Gleeson:
A security measure. The most unlikely security measure that I’ve ever heard of.

Peter Mackenzie:
We have, through a recent company we’ve been working with, identified a new method of protecting yourself from exfiltration. This was an organisation hit by BlackSuit ransomware, and the attackers made a claim—350 gigabytes of data had been stolen. We were investigating the machine and had identified where they were zipping up the data they were going to exfiltrate, and it was very sensitive data as well. We found a few different methods they were trying to use to exfiltrate it, and on each occasion, they gave up after a few minutes. We were trying to work out why, and then we suddenly realised that while talking to this customer, their internet is awful. Literally, they’re a group of islands in the Pacific, basically, and yeah, they just have very, very slow internet. They were uploading at about 300 kilobytes a second. So we were trying to pull data from these machines, and it was taking days. Yeah, we suddenly realised the attacker got bored. They couldn’t be bothered to do the exfiltration because the internet was so slow. Now, we can never be a hundred percent sure on these things, but everything we looked at came to the same conclusion: they started, it wasn’t working very well, and they stopped.

Adam Gleeson:
So there you go, folks. Break out your ADSL routers. It’s the latest and greatest security feature (sarcasm).

Adam Gleeson:
Now, obviously, you’re a director at Sophos, and I try to keep vendors out of these podcasts just for talking specifically about vendor products and stuff like that. However, Sophos MDR is a product that I really, really like, and if I were a customer, I would be moving heaven and earth to make the budget available, because it would give me peace of mind as an administrator, or as the owner of a business, that I’ve got people watching it 24/7. So, if we’ve talked around the process that would happen in these ransomware incidents, what would that look like if they were an MDR customer already? Now, with the caveat that we can never be a hundred percent secure because it’s just not possible to do that—a determined attacker is going to find a way in. So before we start to wrap this up, and because you’re obviously a director for Sophos, and the Sophos Managed Detection and Response product is great, if I were a business owner, or if I were an administrator within an organisation, I would be recommending to the owner that this is one of the best cyber security defences we can put in place. It’s not a hundred percent bulletproof, but it is 99.99999%. There’s no guarantee with cyber security, and what we’ve just talked about here is the stages of ransomware infection and what the whole thing looks like. What would that look like if this was an MDR customer?

Peter Mackenzie:
Yeah, so MDR—24/7 team of people looking at the alerts, looking at the detections, looking at the suspicious activity that is happening on a network, as well as doing what we call threat hunts. So, if there’s some new vulnerability out there, we’d check all of our customers and go, “Right, you guys are vulnerable. You need to go patch this.” So, we’re alerting them, making them safer in general. But the real crux of it is watching your network, and when something suspicious starts happening, having people who can react within minutes rather than days or weeks to say, “This machine needs isolating, this username needs disabling,” or whatever. So, what a ransomware attack looks like is someone gets access to a network, maybe one machine, and they’re going to want to get domain admin, they’re going to want to move around, they’re going to want to do maybe a network scan. They’re going to want to understand where your FS are, and they’re going to come up with a plan on how to deploy their ransomware. This takes time, takes tools, and many of these things will get detected by your security solutions as they go through that process. And if you’ve got the MDR service, or someone who can see these things happening, they can take action there. So, we often get asked, “How many ransomware attacks does the MDR team stop?” And we go, “We don’t really know because we stop it long before it becomes a ransomware attack.

Peter Mackenzie:
Yeah, exactly. We stop them when they get in. We take actions. We can’t read their minds. We don’t know what they were going to do. They may have been doing ransomware, they may have been doing something else, but the point is, we can take action before it gets anywhere close to that.

Adam Gleeson:
And this goes back to what we talked about right at the start of part one of this podcast, and that was that the question is asked, “When did this happen?” And they’ll typically say, “Yesterday or today,” because that’s when they first became aware of it. But as you pointed out, it’s actually been going on for weeks before that potentially. And with MDR, that just wouldn’t have happened because it would’ve been intercepted before then.

Peter Mackenzie:
What we’ve found is, if you search our Sophos Active Adversary report that we release multiple times a year, talking about the data we get from my team and what we’re seeing from these attacks, the bigger the organisation you are, the quicker the attack normally is. If you’re a large company, the attackers we presume think, “Well, you probably do have your own security team or your MDR service. You probably do have the tools that can identify us and kick us out. So we need to work faster.” That’s what we think the attackers think on large companies. But when it’s a small company, they take their time. They don’t need to be in a rush. They think, “Well, no one’s probably watching. Just because they’re smaller doesn’t mean they’re not going to pay a large ransom. I don’t need to worry about it so much, so I’m going to be on your network for two weeks rather than two days,” kind of thing.

Adam Gleeson:
Cool. So long and the short of it—if you can get it, it’s an invaluable tool that’s in your kit bag to give you defence. That’s all we’ve got time for. Thank you very much for your time and patience, Peter, with me talking too much. Again, I hope you’ve found this interesting. We’ve certainly enjoyed talking about this. As always, I learned something new, even though I’ve been doing this for a long time. So, thank you, Peter, for that.

Stay secure.