David Dixon

Artificial intelligence (AI) being used for malicious intent has surfaced as a significant concern within the digital space. Cyber criminals are using Large Language Models (LLMs), like ChatGPT, and deepfake technology to launch cyber-attacks and scams. In this blog, we focus on the darker facets of AI, shedding light on the exploitation of AI systems, its impact on the threat landscape, and what organisations can do now to better protect themselves and their most sensitive assets against this new wave of threats. 

Malicious ChatGPT Prompts for Sale on the Dark Web Marketplace 

• Recent reports reveal a disturbing trend where thousands of malicious prompts designed to jailbreak and exploit AI are up for sale on the dark web. These prompts deceive AI models, enabling threat actors to steal data, orchestrate sophisticated scams and other illegal activities with alarming efficiency. 
• According to recent research carried out by Kaspersky, thousands of these nefarious prompts and compromised premium ChatGPT accounts are now available for purchase, posing a significant threat to ChatGPT, its users and their data. (source: The Register) 

AI and deepfake technologies are becoming more readily available. OpenAI, for example, recently announced their new generative AI, Sora, that can create video from text. And, although this advancement in technology and its availability is exciting, it is also inevitable that there will be cyber criminals looking to use it maliciously. 

Around the globe we are already seeing examples of these technologies being exploited by advanced threat actors, including cyber criminals, nation states or nation sponsored hacker groups. 

$25 million theft executed through a sophisticated deepfake scam

A recent article by Ars Technica has shed light on a ground-breaking cyber crime incident considered to be the first successful heist of its kind: a $25 million theft executed through a sophisticated deepfake scam. The scam involved the creation of highly convincing AI generated deepfake videos, which were used to impersonate key individuals within a financial institution.  

By leveraging these deepfake videos, the scammer manipulated employees into authorising fraudulent transactions, resulting in the substantial loss. This unprecedented heist marks a significant escalation in the sophistication of cyber criminal tactics, underscoring the evolving threat landscape faced by organisations worldwide. As the prevalence of AI-driven scams will inevitably continue to rise, it becomes increasingly crucial for businesses to bolster their cyber security posture and remain vigilant against such deceptive schemes. 

Deepfake news segments 

Iran-backed hackers had recently disrupted TV streaming services in the United Arab Emirates (UAE) by injecting deepfake news segments into the broadcasts according to The Guardian. These deceptive deepfake videos, generated using AI technology, were designed to resemble legitimate news reports, spread misinformation, and sow discord among viewers. This incident underscores the growing threat posed by state-sponsored threat actors and the increasing weaponisation of deepfake technology for political purposes.  

As nations continue to grapple with the challenges of cyber warfare and disinformation campaigns, it becomes imperative for governments to collaborate and implement international legislation that both prohibits and protects against the use of such attack methods, as well as educate and inform organisations across all industries about AI threats and how best to protect themselves and their assets. Additionally, organisations need to enhance and adapt their cyber security capabilities to be able to identify and defend against orchestrated AI driven attacks, which is backed up by a recent assessment conducted by the NCSC. The assessment focuses on how AI will impact the efficacy of cyber operations and the implications for the cyber threat over the next two years. (source: NCSC) 

According to the above-mentioned assessment by the NCSC, AI is poised to significantly impact the cyber threat landscape in the near future. The report suggests that AI will almost certainly be utilised by cyber adversaries to enhance their capabilities, including the development of more advanced attack techniques and procedures (TTPs).  

As AI technologies evolve, cyber criminals are increasingly going to automate tasks, evade detection, and execute targeted attacks with greater precision. This assessment underscores the urgent need for organisations to adapt their cyber security strategies to effectively mitigate the evolving threats posed by AI-driven cyber-attacks. This includes enhancing detection and response capabilities, investing in AI-powered security solutions, enforcing zero trust policies, implementing a culture of sufficient cyber awareness and vigilance amongst staff, and staying informed about emerging AI-driven threat vectors. 

While ChatGPT and other LLMs may not yet be capable of being used to write sophisticated malware to be sold at scale on the dark web or be in possession of nefarious nation states, we may not be far away from AI being used to orchestrate attack chains or write malware that can evade detection. A separate recent report from the National Cyber Security Centre (NCSC) sheds light on how AI driven ransomware attacks could become a reality by 2025. (source: NCSC)

As AI technologies are rapidly evolving, the application of its use for both good and bad is evolving with it, leading to a rapid shift in the threat landscape. It is imperative for organisations to not just understand how to defend against AI driven threats, but to learn how to use AI technologies securely and in a manner that best protects their assets and does not expose them to new vulnerabilities or risk. 

Already we are seeing collaboration amongst the international community to tackle this very issue. A recent publication on how to engage with Artificial Intelligence has been developed by the Australian Signals Directorate’s Australian Cyber Security Centre (ASD’s ACSC) in collaboration with the NCSC, United States (US) Cyber security and Infrastructure Security Agency (CISA), Canadian Centre for Cyber Security (CCCS) and several other cyber security/government agencies from international partners. The publication highlights some key threats related to AI systems and summarises steps organisations should take when engaging with AI technologies (both in-house and 3rd parties) to mitigate risk. (source: ASD’s ACSC) 

While this new wave of advanced threats seems daunting and paints a bleak future for stakeholders responsible for managing risk, there are several steps organisations can do to protect against these threats. Many of these types of attacks still rely on the presence of human error and social engineering. Regularly training your people and creating a positive cyber awareness culture are key to reducing this type of threat.

Further to this, unsecured vulnerabilities are a common route of entry for cyber criminals and can be identified with regular vulnerability scanning and penetration testing to identify your security weak spots.

Organisations across all sectors, of all sizes should not neglect the fundamental steps that make up the foundations of any cyber security strategy. Few organisations have the right tools, people, and processes in-house to manage their security program around-the-clock while proactively defending against new and emerging threats. Adopting security defences like Sophos MDR can provide an elite team of threat hunters and response experts to take targeted actions on your behalf to neutralise even the most sophisticated threats.

For better or worse, AI is going to change how we live our lives greatly, and while its application for solving huge problems on a global scale is something to be embraced, we should also be aware of its capacity to cause great harm. Organisations need to adapt to the new world of AI driven technologies and attacks, whilst continuing to invest in the foundations of their cyber security posture.

As we progress through 2024, the cyber threat landscape continues to evolve rapidly, presenting new challenges for organisations and individuals alike. In this edition, we shift our focus from past incidents to the present and future threats that pose the greatest risks.

Understanding these threats is crucial for developing effective strategies to safeguard against them. Cybercrime is expected to become the world’s third largest economy by 2025, estimated to cost $10.5 trillion in damages globally, according to cybercrime magazine.

This month, we highlight the top five most dangerous cyber threats of 2024, exploring their nature, potential impacts, and the steps you can take to protect yourself and your organisation. Join us as we explore these pressing cyber threats and provide insights into enhancing your cyber resilience.

At the RSA Conference 2024, cyber security experts identified offensive AI as a significant threat multiplier, presenting it as one of the top five cyber threats for the year. Stephen Sims highlighted how malicious actors leverage AI and automation to rapidly identify vulnerabilities, automate the generation of exploits, and launch sophisticated attack campaigns.

This offensive AI capability accelerates the discovery and exploitation of vulnerabilities, posing a formidable challenge for defenders. Sims emphasised the urgent need for defensive strategies capable of countering the speed, automation, and intelligence wielded by attackers, underscoring the importance of innovative defensive measures to mitigate this evolving threat landscape in 2024.

Social engineering has long been a prominent cyber threat, relying on psychological manipulation to deceive victims. Attackers exploit human traits such as trust, fear, and curiosity to gain access to critical systems or sensitive information. Traditional social engineering methods include phishing, baiting, pretexting, and tailgating.

The increasing digital transformation and real-time information sharing have made individuals more susceptible to these attacks. In 2022 alone, there were 493 million ransomware attacks, and 19% of all data breaches were due to stolen or compromised login credentials.

AI has significantly amplified social engineering tactics, enabling attackers to develop more complex and convincing attacks tailored to targeted individuals. AI-driven social engineering can include:

• Hyper-personalised phishing
• AI-generated natural language content
• Emotional manipulation
• Detection evasion
• Automated reconnaissance

These advancements allow attackers to craft tailored, context-aware campaigns quickly and efficiently, making traditional defences less effective. Businesses must now contend with AI-generated deepfakes, persuasive phishing emails, and sophisticated data manipulation, requiring a proactive and adaptive approach to cybersecurity.

Implementing multi-factor authentication, employee training, phishing simulations, and AI-based defence mechanisms are essential to counter these advanced threats.

Despite significant global law enforcement efforts, ransomware activity has continued to surge in 2024. According to the Sophos 2024 Ransomware Report, global ransomware attacks in 2023 set a record high, surpassing the previous year by nearly 70%. In the first quarter of 2024 alone, 1,075 ransomware victims were posted on leak sites. Major ransomware groups like LockBit and ALPHV/BlackCat were responsible for 30% of the activity.

The report also highlights the financial impact of these attacks, with average ransom payments increasing by 500% in the last year. A staggering 63% of ransom demands were for $1 million or more, and 30% exceeded $5 million, indicating that ransomware operators are targeting larger payoffs.

Overall, these reports indicate that ransomware isn’t slowing down in 2024 and remains one of the largest threats to organisations. Few organisations have the right tools, people, and processes in-house to manage their security program around-the-clock while proactively defending against new and emerging threats.

Supply chain risk continues to be a significant threat in 2024, as we’ve already seen from major incidents this year involving MoD and MITRE, which we covered in our blog outlining the top five cyber incidents of 2024 so far. Despite increased awareness, many businesses still struggle with effectively managing these risks. According to the UK Government’s Cyber Security Breaches Survey 2024, 31% of businesses and 26% of charities conducted cyber security risk assessments in the past year. Larger organisations are more proactive, with 63% of medium businesses and 72% of large businesses conducting these assessments.

However, only 11% of businesses review the cyber risks posed by their immediate suppliers, a figure that rises to 28% for medium businesses and 48% for large businesses. This limited oversight is concerning given the complex and interconnected nature of modern supply chains. The qualitative data indicates that while awareness of supply chain cybersecurity risks is growing, smaller organisations often lack the formal procedures necessary to manage these risks effectively.

As organisations continue to migrate their operations to the cloud, vulnerabilities within cloud environments have become a critical concern in 2024. The flexibility and scalability of cloud services come with a unique set of security challenges that can be exploited by well-versed threat actors.

A significant factor contributing to cloud vulnerabilities is misconfiguration. According to reports as far back as 2019, misconfigured cloud settings were responsible for nearly 70% of all cloud security incidents, and according to IBM’s Cost of a Data Breach Report, 45% of reported breaches were cloud-based. These misconfigurations can lead to unauthorised access, data leaks, and compliance issues. Additionally, the shared responsibility model of cloud security often leads to confusion about where the provider’s security obligations end and the client’s responsibilities begin, leaving gaps that can be exploited. The NCSC has published free guidance on cloud security and shared responsibility models.

The rise in cloud-based attacks has also been driven by increasingly sophisticated threat actors targeting cloud infrastructure. For instance, the recent breaches involving high-profile cloud services such as the recent Microsoft Azure incident have shown that attackers are leveraging advanced techniques and exploiting zero-day vulnerabilities to bypass security controls, escalate privileges, and access sensitive data. These incidents highlight the importance of robust security practices, including regular audits, comprehensive monitoring, and strict access controls.

To mitigate cloud vulnerabilities, organisations should focus on improving their cloud security posture through continuous monitoring to identify any vulnerabilities or misconfigurations exposing their cloud infrastructure’s attack surface, employee training and implementing robust policies for access, user groups and data handling, and of course, adherence to best practices for cloud configuration and management. The Center for Internet Security (CIS) has published the CIS Benchmarks, a series of prescriptive recommendations for configuring over 25 cloud and network vendor product families including AWS, Azure and Google Cloud Platform (GCP).

By addressing these vulnerabilities proactively and implementing industry best practices and benchmarks, businesses can better protect their data and maintain trust with their customers.

The top five cyber threats of 2024 so far serve as a stark reminder of the evolving threat landscape. By understanding the risk and implementing a layered and strategic approach to cyber security, organisations can better protect their people, data, and customers.

The finance sector is one of the biggest targets of cyber threat actors with 65% of organisations hit by ransomware in 2024, according to recent research by Sophos. As the financial technology (Fintech) sector continues to revolutionise the way we handle money, the stakes for cyber security have never been higher.

The integration of innovative digital solutions, from AI-driven financial services to blockchain technology, has opened up new opportunities for growth, but it has also expanded the threat landscape.

This blog explores the current cyber security challenges facing the financial technology industry, the impact of these threats, and the best practices that companies can adopt to safeguard their operations and customer trust.

The Fintech industry, characterised by its rapid adoption of cutting-edge technologies, is a prime target for cybercriminals. According to recent reports, the financial services sector experiences cyber-attacks 300 times more frequently than other industries, with Fintech companies being particularly vulnerable due to their digital-first nature. The rise of AI and machine learning in Fintech has further complicated the threat landscape, as these technologies can be both a tool for defence and an instrument for sophisticated attacks.

Key Threats Facing Fintech Today

• Data Breaches: With vast amounts of sensitive financial data at stake, data breaches remain one of the most significant risks for Fintech and financial services firms. Recent breaches, such as the SolarWinds attack, have highlighted the vulnerabilities in supply chains and third-party providers, making it clear that no organisation is immune.
• AI-Driven Cyber Attacks: The same AI technologies that enable personalised financial services are also being used by cybercriminals to automate attacks, enhance phishing campaigns, and exploit vulnerabilities faster than traditional methods. For instance, AI can create highly convincing deepfake videos and emails, making it easier to deceive even the most vigilant employees.
  One of the most alarming examples occurred in earlier this year, when cybercriminals targeted a Hong Kong-based financial services firm in a first-of-its-kind heist. Using advanced deepfake technology, the attackers impersonated the firm’s Chief Financial Officer (CFO) during a video conference call. They convincingly replicated the CFO’s voice and appearance, deceiving an employee into transferring nearly £20 million to a fraudulent account. [source: Ars Technica]
• Regulatory Challenges: With evolving regulations such as GDPR and PSD2 in Europe, and new guidelines from the FCA and other financial authorities worldwide, Fintech companies must navigate a complex web of compliance requirements. Failure to comply not only risks legal substantial penalties, but also damages brand reputation.