Generative AI is being used more and more in cyber attacks

Last updated: June 26, 2026

Generative AI in cyber attacks has become a true accelerator: it reduces preparation times and costs, increases the credibility of messages, and enables more scalable campaigns. In 2025, attacks exploiting AI grew rapidly, and in Italy, a significant share of severe cases involves generative AI tools. For decision-makers, the priority is combining technical controls, governance, and training.

Why generative AI is making cyber attacks faster, cheaper, and more credible

Generative AI (models capable of creating text, code, images, and audio) is a dual-use technology in cybersecurity: the same Large Language Models (LLMs) that assist defensive teams can amplify offensive capabilities. The advantage for attackers is clear: fewer skills required, less preparation time, and higher persuasiveness.

According to the “AI Threat Landscape 2025” report by Maticmind (covered by Il Sole 24 Ore, 2025), in the first half of 2025, attacks exploiting AI increased by 47% compared to the previous year. In parallel, 77% of security professionals expect an increase in attack volume and speed over the following two years.

The most common malicious uses include:

• more credible phishing and spear-phishing
• deepfakes and voice cloning for impersonation
• generation/refinement of malicious code
• large-scale social engineering automation

The key takeaway for enterprises, public administrations, banks, and universities is that 2025–2026 marks a shift toward more convincing and, above all, more industrialized attacks.

How cyber attacks are changing with generative AI

Cyber attacks are changing because generative AI makes it easier to produce credible, personalized, and "error-free" content. Phishing (scams designed to trick users into clicking, paying, or sharing credentials) remains the most widely used tactic: in the report cited by Maticmind, over 80% of phishing emails exploit generative AI models (2025), eliminating "classic" red flags like grammatical errors.

The same applies to spear-phishing (phishing targeted at a specific person or role): the report indicates that 91% of campaigns were supported by generative AI (2025). Within this scenario, Business Email Compromise (BEC) (email fraud imitating CEOs or suppliers to obtain wire transfers or sensitive data) is on the rise because messages align perfectly with style and context, even when misusing consumer tools like OpenAI or Google Gemini.

Generative AI is a dual-use technology: it can significantly improve cyber defence, but it also lowers the barrier of entry for criminal actors by automating the creation of highly convincing phishing emails, malware code and social engineering scripts.

— Europol Innovation Lab, Innovation Lab analysts, Europol

Deepfakes (synthetic audio or video imitating a real person) are also surging: from 500,000 cases in 2023, the number is expected to exceed 8 million by the end of 2025, and today 1 out of 20 cases in identity verification checks is due to deepfakes (estimates reported within the scope of the 2025 report). The practical difference is twofold: attacks are more convincing, and attacks are more scalable.

Which attack techniques generative AI enhances beyond phishing and deepfakes

Beyond phishing and deepfakes, generative AI powers chained offensive techniques, frequently combining LLMs (text/code), OSINT (Open Source Intelligence), and automation. The typical objective is not "inventing" an entirely new attack, but rather increasing speed, precision, and volume.

• Assisted malware generation: AI helps write or refine portions of malicious code and scripts, lowering the technical entry barrier (Microsoft, 2025: The danger of AI’s role in cyber attacks).
• Vulnerability prioritization: AI can summarize CVE (Common Vulnerabilities and Exposures) bulletins and suggest which systems to target first, accelerating the targeting phase.
• Personalized social engineering: combining OSINT and LLMs makes it simpler to create highly credible pretexts for HR, procurement, and administrative offices, increasing conversion and response rates.
• Voice cloning and spoofing: voice cloning and identity falsification in BEC scenarios, particularly for urgent payment requests or changes to bank details (UNODC, 2025: Emerging threats).
• “Criminal LLM” platforms: names like WormGPT or FraudGPT circulate as market labels; for defenders, they serve as a signal of commoditization rather than "magical tools."

We are starting to see the first cyber-espionage operations fully orchestrated by AI agents, capable of scanning, identifying and exploiting vulnerabilities almost autonomously.

— Matteo Lucchetti, Curator of the ICT Security Forum

In practice, AI does not replace the attacker: it amplifies their ability to generate variants and rapidly adapt to security controls.

When AI attacks AI: prompt injection, data poisoning, and model theft

The battlefield has expanded: it is no longer just data and infrastructure at risk, but the AI tools themselves—such as prompts, datasets, and models. This makes certain traditional security paradigms less reliable, as the attack surface now encompasses linguistic interactions and data pipelines.

«We are experiencing the paradox of artificial intelligence: on one hand, it increases attack vectors and broadens the exposed surface of those who use it; on the other hand, if correctly managed, it can significantly strengthen defenses».

— Pierguido Iezzi, Cyber Business Unit Director, Maticmind

Three recurring threats, also featured in the OWASP Top 10 for LLM Applications guidelines, are:

• Prompt injection (manipulating inputs to alter output): can induce a chatbot to reveal confidential data or bypass embedded policies.
• Data poisoning (contaminating training or retrieval data): degrades response reliability and can inject malicious "facts."
• Model theft (unauthorized theft/replication of models and know-how): impacts intellectual property and competitive advantage.

For organizations adopting internal chatbots, copilots, or LLMs, defense requires rigorous application controls and data security measures, as recommended by NIST. For a technical deep dive into these vectors, see AI model poisoning techniques like prompt injection and data poisoning.

Traditional cyber attacks vs. generative AI attacks: what truly changes

The three operational differences are: speed (shorter cycles), personalization (tailored messages), and scalability (more variants for the same amount of effort). Sources like IBM and Microsoft describe AI as a multiplier for both offense and defense, especially across phishing, social engineering, and automation.

On the defensive side, automation and AI tools can reduce the average time to identify and contain a data breach by 108 days (IBM Cost of a Data Breach Report, 2024: source). The core issue is that the same automation logic accelerates adversary campaigns: "victory" belongs not to whoever simply has AI, but to whoever better governs processes, data, and controls within the SOC (Security Operations Center) and SIEM (Security Information and Event Management) platforms.

Which sectors are most exposed: banking, universities, and the digital supply chain

Risk is not uniform: certain sectors offer higher economic incentives, more identities to protect, and processes that are more easily “mimicked” via AI. In Italy, nearly 50% of attacks in 2024 targeted **Public Administration** (30.3%) and **Finance & Insurance** (18.18%).

Banking: banks and financial institutions are highly exposed to fraud, **BEC**, and **voice cloning** within payment processes, alongside operational resilience risks. Here, DORA (Digital Operational Resilience Act) and GDPR take center stage, as incident response and traceability shift from best practices to strict requirements. To explore secure setups, see private AI architectures and compliance in the banking sector.

Universities: higher education institutions and research centers maintain an expansive attack surface (students, faculty, guests) and sensitive assets (credentials, research data, intellectual property). Spear-phishing targeted at administrative offices and research groups is a recurring pattern; in Italy, the Education sector grew by up to +43% (Clusit 2025). Regarding the Italian context, see Italian university partnerships with OpenAI and AI innovation.

Digital supply chain: a Managed Service Provider (MSP) or an external vendor can become the gateway. AI makes it easier to impersonate roles and apply pressure on ticketing platforms, help desks, and procurement systems, raising the risk of lateral movement compromises.

How to defend against attacks enhanced by generative AI

Enterprises mitigate risk by combining technical controls, clear policies, and continuous training: no single measure suffices against faster, more credible attacks. An approach aligned with the NIST Cybersecurity Framework (Identify–Protect–Detect–Respond–Recover) and **Zero Trust** principles (continuous verification) helps build resilient defenses even when dealing with synthetic content.

1. Prevention: anti-phishing training updated for the GenAI era (simulations featuring realistic texts, focusing heavily on BEC and urgent requests).
2. Prevention: enforcing MFA (Multi-Factor Authentication) everywhere and establishing out-of-band verification for payments/bank detail changes (a call to a known number, not the one provided in the email).
3. Detection: anomaly monitoring and behavioral analytics across emails, logins, and transactions; fine-tuning **SIEM** environments and **SOC** playbooks.
4. Response: rapid procedures for “stop payment” situations and internal emergency communication; specialized runbooks for BEC and voice-cloning fraud.
5. Deepfakes: strict verification protocols for suspicious audio/video interactions (challenge phrases, requesting specific live video actions, two-person escalation paths).
6. Internal LLMs (e.g., Microsoft Copilot, Google Workspace): comprehensive logging, API access control, strict data segregation, regular prompt injection testing, and filters on sensitive data input.
7. Governance: clear corporate policies on data shared with public models alongside risk/compliance evaluations (NIS2, GDPR, EU AI Act).

Within regulated contexts (banking and public administration), change governance is the real differentiator: clear roles, deep auditability, and repeatable controls. For regulatory and risk-management insights, see AI governance and risk management strategies under the EU AI Act and the impact and alignment requirements of the EU AI Act.

Practical note: many operational discussions on deepfake detection and BEC surface directly within professional communities like r/cybersecurity on Reddit (2025–2026 threads), frequently highlighted by platforms like Perplexity for field validation.

FAQ: common questions regarding cyber attacks powered by generative AI

How do I recognize phishing written with generative AI?

AI-generated phishing is often grammatically flawless and contextually logical, so "classic" indicators are no longer enough. Focus instead on the action being requested (urgency, payments, credentials verification), double-check domains and communication channels, and use out-of-band verification for any anomalous requests, especially in BEC scenarios.

Can generative AI really build malware from scratch?

Yes, it can assist in writing or adjusting malicious code and script variants, but it typically amplifies pre-existing, known techniques rather than "inventing" entirely novel malware types. The main risk lies in the lowering of technical barriers and the rapid iteration speed, which increases the volume of variants and adds pressure to defensive security controls.

How difficult is it to detect a convincing deepfake within an enterprise setting?

It is significantly more difficult than before because synthetic audio and video improve rapidly, easily bypassing initial human checks. In 2025, it was estimated that 1 out of 20 identity verification checks was tied to deepfakes. Mitigating this risk requires strict verification protocols, explicit escalation paths, and the use of pre-arranged “challenge phrases.”

Which sectors are most vulnerable to AI-driven attacks in Italy?

Finance, Public Administration, and Education are among the hardest hit and/or fastest-growing targets according to the Clusit Report 2025. Financial institutions and insurance companies are prime targets for fraud and BEC; universities and research centers are targeted for credentials and data theft; and the digital supply chain is vulnerable because a compromised supplier can open up a cascade of lateral entry points.

What controls can I implement immediately without overhauling my entire security architecture?

The fastest actions include enforcing MFA everywhere, requiring out-of-band verification for payments or wire detail changes, launching updated phishing simulations, tuning logging alert systems for anomalies, and creating specialized incident response playbooks for BEC and deepfakes. If you deploy internal LLMs, introduce prompt injection testing and clear rules regarding data inputs.

Related deep dives: the evolution of Large Language Models (LLMs) and generative AI; AI models integrating traditional techniques and generative AI; using digital twins for traceability and cross-sector compliance.