What AI changed for attackers
The single biggest change is quality at volume. Fluent, contextually accurate phishing in any language now costs nothing to produce, which removes the language and grammar cues that people were trained to look for over two decades.
Beyond text, models generate convincing voice clones from a few seconds of audio, video avatars usable in live meetings, fake documents and identity images, and working variations of malicious code. Reconnaissance is also faster: publicly available profile information can be summarised into a personalised pretext in seconds.
- Fluent phishing at scale in every language
- Voice cloning from short public audio samples
- Deepfake video in investment adverts and live calls
- Faster reconnaissance and personalised pretexts
- Automated variation of malicious code to evade signatures
What AI has not changed
Attacks still need a way in and an action from you. Payment still has to move to an account, credentials still have to be entered somewhere, and remote access still has to be granted. Those choke points are where defences work regardless of how the approach was generated.
This is why verification procedures beat detection instincts. You do not need to judge whether a voice is real if your rule is to call back on a number you already hold.
Deepfake fraud in practice
Two patterns dominate. In the family emergency, a cloned voice of a relative calls from an unfamiliar number in distress and needs money immediately. In the corporate version, a video call with apparently familiar colleagues authorises an urgent transfer.
The countermeasures are procedural. Agree a family code word that is never shared online. In organisations, require a second approver and an out-of-band confirmation for any payment above a threshold or any change of bank details, and make it explicit that nobody will be criticised for delaying a payment to verify it.
Risks of the AI tools you use
Information typed into a consumer AI assistant may be retained, reviewed by humans or used for training depending on the plan and settings. Treat client data, credentials, health details and unpublished business information as unsuitable for those tools unless you are on an agreement that prohibits training.
There is also a supply of malicious imitations: browser extensions and mobile apps posing as popular AI products that harvest credentials or inject advertising. Install only from the vendor's official site or store listing, and check the publisher name rather than the icon.
- Check whether your inputs are used for model training
- Never paste credentials, client data or identity documents
- Install AI apps and extensions only from official publishers
- Treat AI output as a draft to verify, not a source of fact
How defenders use AI
Security products have used machine learning for years in behavioural detection, spam filtering and anomaly analysis, and the current generation is better at correlating weak signals and summarising incidents for people who are not specialists.
For consumers the practical benefit appears as improved scam detection in messaging and browsing protection, faster identification of unfamiliar malware behaviour, and clearer explanations of alerts. It does not remove the need for updates, backups and multifactor authentication.
A realistic personal defence
Move the accounts that matter to passkeys or hardware security keys, because they cannot be handed over by a persuasive caller or a perfect replica login page. Keep money movement behind a verification habit rather than a judgement call.
Reduce the raw material available for cloning by limiting public voice and video, and talk to older relatives specifically about emergency-call scams: awareness of the technique is the defence that transfers best to people who will not change their technology.
