How big is it? An emerging risk, with numbers
Voice cloning and deepfake video are real, and the official bodies warn about them, but they are still a small part of the picture. Scamwatch says scammers can use a few seconds of a voice recording to make an AI clone, and that they use AI to make fake videos and audio that look and sound real. The FBI's 2025 report counts more than 22,000 complaints that mention AI in some way, with adjusted losses above US$893 million, and says businesses reported over US$30 million in losses to BEC scams involving AI.
To keep that in proportion: the same report puts all BEC at about US$3.05 billion. On our own arithmetic, US$30 million is roughly 1% of US$3.05 billion, so in that report the BEC scams that mention AI are a small share of all BEC losses. That does not mean AI voices can be ignored: they are cheap to try, and the checks in this course (call back, second person, slowing down) defeat them as well as the ordinary tricks.
Remember
Facts and sources for this part
- Scamwatch says scammers can use a few seconds of a voice recording to make an AI clone of a voice, then call pretending to be a friend or family member in distress. [KS7]
- Scamwatch says scammers use AI to make fake videos and audio recordings that look and sound real. [KS7]
- The FBI's 2025 report counts more than 22,000 complaints that mention AI, with adjusted losses above US$893 million, and says businesses lost over US$30 million to BEC scams involving AI. [KS16]
- The FBI's 2025 report ranks business email compromise as the second-largest source of reported losses in the United States, after investment fraud, at about US$3.05 billion. [KS16]
How an attack can sound
INCIBE defines a deepfake as a fake video, manipulated with AI tools that can swap faces in images and change the voice. The Australian Cyber Security Centre describes vishing as a voice scam that uses voice cloning, deepfake technology or fake caller IDs to sound real. It can be harder to detect because a familiar voice makes you more likely to trust the caller, the phone creates urgency, and caller ID and voices can be faked. The US Federal Trade Commission notes that a call that sounds like your boss asking for bank account numbers makes you more likely to act, and that a scammer needs only a short audio clip, which can come from content posted online.
The FBI warns that criminals generate video for real-time video chats with people claiming to be company executives, law enforcement or other authority figures, and use AI audio of a person to try to get into that person's bank accounts. FinCEN says criminals have used generative AI to pretend to be an executive or trusted employee and tell victims to transfer large sums.
Remember
Facts and sources for this part
- INCIBE defines a deepfake as a fake video, manipulated with AI tools that can swap faces in images and change the voice. [KS38]
- The Australian Cyber Security Centre describes vishing as a voice scam that uses voice cloning, deepfake technology or fake caller IDs to sound real. [KS11]
- It says vishing can be harder to detect because a familiar voice makes you more likely to trust the caller, the phone creates urgency, and caller ID and voices can be faked. [KS11]
- The US Federal Trade Commission says a call that sounds like your boss, asking for bank account numbers, or a family member in an emergency makes you more likely to act, which is why scammers use voice cloning. [KS22]
- The same agency says a scammer needs only a short audio clip, which can come from content posted online, and a voice-cloning program. [KS21]
- The FBI warns that criminals generate video for real-time video chats with people claiming to be company executives, law enforcement or other authority figures. [KS14]
- The FBI warns that criminals use AI-generated audio of a person to get into that person's bank accounts. [KS14]
- The US Treasury's financial crimes unit (FinCEN) says criminals have used generative AI to pretend to be an executive or trusted employee and tell victims to transfer large sums. [KS31]
Three cases from the sources
INCIBE describes a Spanish company where staff heard a voice exactly like the CEO's, joined a call in which he said an audit was imminent and a machine had to be bought, and paid straight after the call. Days later they learned it was not him. In another INCIBE case, a fake CEO messaged an employee on the company's Teams chat asking for an urgent EUR 150,000 purchase and emailed a document with forged signatures.
A third INCIBE case shows the defence working: a woman heard her husband's voice asking her to send a message to a number, called his usual number instead, and he confirmed he had not called. Notice what the people who were fooled had in common: a request outside the normal process, urgency, and no callback.
Remember
Facts and sources for this part
- INCIBE describes a Spanish case where staff heard a voice exactly like the CEO's, joined a call where he said an audit was imminent and a machine had to be bought, and paid straight after the call; days later they learned it was not him. [KS35]
- In another INCIBE case a fake CEO messaged an employee on the company's Teams chat asking for an urgent EUR 150,000 purchase, and sent an email with forged signatures to look credible. [KS36]
- INCIBE gives a case where a woman heard her husband's voice asking her to send a message to a number; she called his usual number, and he confirmed he had not called. [KS39]
Do not rely on spotting the fake
The FBI lists signs in fake images and video, such as odd hands, irregular faces, wrong shadows, lag in the voice and unnatural movement, and says to listen to tone and word choice. It also says AI-generated content has advanced to the point that it is often difficult to identify and that voice clones can sound nearly identical. Europol adds that strengthening the approval process is currently seen as better than relying only on deepfake detection tools.
Europol's examples of stronger checks include approving with audio and video rather than audio alone, demanding a live video connection, and asking the person to perform random, complicated acts live on camera. Our advice: for money and passwords, the best check is not "does it sound right?" but "have I confirmed it through a channel the criminal does not control?".
Remember
Facts and sources for this part
- The FBI says AI-generated content has advanced to the point that it is often difficult to identify. [KS17]
- The FBI lists signs in fake images and video, such as odd hands, irregular faces, wrong shadows, lag in the voice and unnatural movement, but also warns that these may be hard to see. [KS17]
- The FBI says to listen closely to tone and word choice, because an AI voice clone can sound nearly identical to a real person. [KS17]
- Europol advises organisations that rely on face or voice checks to look at the whole approval process, and says strengthening the process is currently seen as better than relying only on deepfake detection tools. [KS30]
- Its examples of stronger checks include using audio-visual rather than audio-only approval, demanding a live video connection, and asking the person to do random, complicated acts live on camera. [KS30]
Calm habits that work
The FTC's advice is not to trust the voice: call the person back on a number you know is theirs and verify the story. The FBI suggests an agreed secret word or phrase. INCIBE advises confirming identity through another known channel and, after its voice-clone case, authorising payments by email or in person, not by phone. Peru's data protection authority advises using the privacy options on social platforms and not posting videos, audio or personal information publicly.
None of this needs fear. It needs a habit: when a voice or video asks for money or access, pause, hang up and confirm.
Remember
Facts and sources for this part
- Its advice is not to trust the voice: call the person back on a phone number you know is theirs and verify the story. [KS21]
- The FBI suggests agreeing a secret word or phrase with family members to verify identity. [KS14]
- INCIBE advises confirming the identity of contacts through another known channel, such as a direct phone line or in person, especially for sensitive operations. [KS36]
- After that case INCIBE recommended that payments be authorised by email or in person, never by phone, and that larger payments need more than one person. [KS35]
- Peru's data protection authority says scammers need only a few seconds of audio, which they can take from a social media video, to clone a voice and phone relatives asking for money for a fake emergency. [KS42]
- The same authority advises using the privacy options of social platforms and not posting videos, audio or personal information publicly. [KS42]
Scenario 1: a phone call
Valeria in finance takes a call from what sounds exactly like her director: "I'm in a closed meeting with an auditor. We need to buy a piece of equipment today to pass the audit. Transfer $19,500 to this account and don't mention it to anyone until I'm out." The number shown is a number she does not recognise.
Red flags: A voice that sounds right, asking for an urgent confidential payment; A story that cannot be checked right now; A request to tell no one and to pay a new account.
What to do: Say you will take care of it and call straight back. Hang up. Call your director on the number saved in your phone, and tell your manager or a second approver. Do not transfer anything on the strength of a voice. Report the call number to IT.
Remember
- A voice that sounds right, asking for an urgent confidential payment
- A story that cannot be checked right now
- A request to tell no one and to pay a new account
Facts and sources for this part
- INCIBE describes a Spanish case where staff heard a voice exactly like the CEO's, joined a call where he said an audit was imminent and a machine had to be bought, and paid straight after the call; days later they learned it was not him. [KS35]
- Its advice is not to trust the voice: call the person back on a phone number you know is theirs and verify the story. [KS21]
- INCIBE advises confirming the identity of contacts through another known channel, such as a direct phone line or in person, especially for sensitive operations. [KS36]
- After that case INCIBE recommended that payments be authorised by email or in person, never by phone, and that larger payments need more than one person. [KS35]
Scenario 2: a video call
Marcus is invited to a video call "with the CFO and the lawyer" about a confidential acquisition. The CFO's face and voice look right, but the picture is slightly soft and the lips lag a little. The CFO asks Marcus to approve a $85,000 deposit to the lawyer's trust account today and asks everyone to keep the call private.
Red flags: A confidential, urgent payment requested on a video call; A slight lag and soft picture, which are possible signs but cannot be relied on; The only evidence is what is on screen, with no independent confirmation.
What to do: Say the payment needs the usual written approval and a second approver. Confirm with the CFO by a call to their saved number and by internal email, and ask for the invoice and payment details through the normal process. Do not pay from a call. Report the invitation to security.
Remember
- A confidential, urgent payment requested on a video call
- A slight lag and soft picture, which are possible signs but cannot be relied on
- The only evidence is what is on screen, with no independent confirmation
Facts and sources for this part
- The FBI warns that criminals generate video for real-time video chats with people claiming to be company executives, law enforcement or other authority figures. [KS14]
- The US Treasury's financial crimes unit (FinCEN) says criminals have used generative AI to pretend to be an executive or trusted employee and tell victims to transfer large sums. [KS31]
- Europol advises organisations that rely on face or voice checks to look at the whole approval process, and says strengthening the process is currently seen as better than relying only on deepfake detection tools. [KS30]
- Its examples of stronger checks include using audio-visual rather than audio-only approval, demanding a live video connection, and asking the person to do random, complicated acts live on camera. [KS30]
- After that case INCIBE recommended that payments be authorised by email or in person, never by phone, and that larger payments need more than one person. [KS35]
Scenario 3: a chat message
Priya gets a voice note on a messaging app from a number saved as "Boss": "Hey Priya, my phone is dying, can you quickly send our client list and pay the supplier invoice I've forwarded? Do it now please." The voice is a bit flat, but it sounds like her boss.
Red flags: A voice note asking for both data and a payment; A "dying phone" story that explains why you cannot call; Urgency and a forwarded invoice that has not been checked.
What to do: Do not send the list or pay. Contact the boss another way (their office line, a colleague next to them, or the company chat) and use the team code word if you have one. Check the invoice against purchase orders. Tell your manager.
Remember
- A voice note asking for both data and a payment
- A "dying phone" story that explains why you cannot call
- Urgency and a forwarded invoice that has not been checked
Facts and sources for this part
- The FBI warns that criminals use AI-generated audio of a person to get into that person's bank accounts. [KS14]
- The FBI suggests agreeing a secret word or phrase with family members to verify identity. [KS14]
- INCIBE advises confirming the identity of contacts through another known channel, such as a direct phone line or in person, especially for sensitive operations. [KS36]
- Its advice is not to trust the voice: call the person back on a phone number you know is theirs and verify the story. [KS21]
Check what you have learned
Three questions from the test, each explained. Tap an answer.
Question 1 of 3
In the FBI's 2025 report, businesses reported over US$30 million in losses to BEC scams involving AI, against about US$3.05 billion for BEC overall. How should this be read?
Show the answer
Right answer: D, AI-related BEC is real but still a small share, so basic payment checks matter most. On our own arithmetic, the AI share is roughly 1%. The same checks defeat both. [KS16]
Question 2 of 3
A caller who sounds exactly like your director asks you to transfer $19,500 today to buy equipment "for an audit" and not to tell anyone. What do you do?
Show the answer
Right answer: B, Say you will call back, then ring the director on a saved number with a second approver. INCIBE describes a Spanish case with this pattern. The checks are the same as for any urgent payment request. [KS35]
Question 3 of 3
What does Europol say is currently a better course of action than relying only on deepfake detection tools?
Show the answer
Right answer: D, Making the whole approval process more robust and harder to get around. Process strength matters more than trying to spot a fake by eye or ear. [KS17]