NEWS: On November 13, 2024, the Financial Crimes Enforcement Network (FinCEN) released this alert to address identity schemes involving “deepfake media” that use generative artificial intelligence (GenAI) tools. The alert also addresses different typologies associated with fraud schemes, identifies red flags related to suspicious activity, and serves as a tool to keep financial institutions informed of the “opportunities and challenges” that derive from artificial intelligence (AI).
What is deepfake media or deepfakes?
According to the Department of Homeland Security (DHS):
Deepfake media, or deepfakes, are a type of synthetic content that use artificial intelligence/machine learning to create realistic but inauthentic videos, pictures, audio, and text. The threat of deepfakes and synthetic media comes not from the technology used to create it, but from people’s natural inclination to believe what they see, and as a result, deepfakes and synthetic media do not need to be particularly advanced or believable to be effective in spreading misinformation or disinformation.
What is artificial intelligence and generative AI?
According to this White House Executive Order:
Artificial intelligence is a machine-based system that can, for a given set of human-defined objectives, make predictions, recommendations, or decisions influencing real or virtual environments. The term generative AI (GenAI) means the class of AI models that emulate the structure and characteristics of input data in order to generate derived synthetic content. This can include images, videos, audio, text, and other digital content.
Many GenAI tools contain information that appear to be realistic (i.e., real events), when in fact, they are not (i.e., a person saying or doing something they did not say or do). FinCEN calls attention to this issue by referencing the National Security Agency (NSA), Federal Bureau of Investigation (FBI), and Cybersecurity and Infrastructure Security Agency (CISA) Cybersecurity Information Sheet, that includes deepfake and “generative AI” examples that are synthetically generated and/or manipulated.
Criminals are also targeting financial institution customers (members) and employees by using GenAI social engineering supporting other scams and types of fraud, such as business email compromise (BEC) schemes, spear phishing attacks, elder financial exploitation, romance scams, and virtual currency investment scams. For example, scammers may use deepfake audio or “voice cloning” to impersonate a family member, friend, or other trusted individual. These scammers also use GenAI tools to target companies by impersonating executives or other employees to make transfers to accounts the scammers control.
How are deepfake fraudulent identities created and used?
Criminals use new technology, such as GenAI, to exploit financial institutions’ identity verification processes. FinCEN has analyzed BSA data that shows criminals are using GenAI to falsify documents, photographs, and videos that financial institutions use as part of their Customer Identification Program (CIP) processes (i.e., identification and verification) and Customer Due Diligence (CDD).
FinCEN shared its concerns about the use of deepfake identities in the alert, saying:
“…financial institutions have reported that criminals employed GenAI to alter or generate images used for identification documents, such as driver’s licenses or passport cards and books.”
This means criminals are using deepfake images to alter and/or modify actual sources of documentation (i.e., driver’s licenses) and replacing them with synthetic images that appear to be valid. Criminals are also using GenAI images with stolen or fake personal identifiable information (PII) to create the synthetic identities. FinCEN’s analysis of BSA data further demonstrated that malicious actors are opening accounts using fraudulent identities resulting from GenAI. As a result, the accounts are being used to receive and launder funds and/or being used as funnel accounts to commit online scams, consumer fraud, loan fraud, and unemployment fraud.
To mitigate these risks, financial institutions must form a reasonable belief that they know the “true identity” of their customers (members). Maintaining risk-based procedures to monitor ongoing due diligence ensures financial institutions understand the nature and purpose of their customer (or member) relationships and customer (or member) risk profiles.
How to detect and mitigate deepfake identity documents
BSA data indicates that financial institutions can detect GenAI and synthetic identifying information used at account opening when the account opening documents are reviewed. For instance, if a financial institution suspects “deepfake” images are used, it can conduct a reverse-image search (i.e., Google Image Search or Bing Visual Search) and use other open-source tools to detect if any photos provided match online galleries of faces using GenAI.
Some financial institutions rely on third-party providers to verify identities at account opening. These third-party providers typically use sophisticated verification software tools to detect potential deepfakes. For example, different metadata examination tools may be used to assess if the metadata is authentic, manipulated, or obtained through a social media platform or other process.
Financial institutions should provide additional scrutiny in situations where:
- Multiple identity documents are submitted by a customer (or member).
- A customer (or member) is unable to identify his or her identity through sources of income or other aspects related to their profile.
- There are inconsistencies with the identifying documentation and the customer (or member) profile.
Examining deepfake documents extends beyond account opening and can often be detected when enhanced due diligence is applied. FinCEN noted that separate identifying indicators may warrant further due diligence including situations where:
- Access to an account is made from an IP address that is not consistent with the customer (or member) profile.
- Coordinated activity appear in patterns among multiple accounts.
- High payment volumes are made to potentially risky payees (i.e., gambling websites or digital asset exchanges).
- Rapid transaction patterns are evident on new accounts or accounts that have prior or little transaction history.
- Patterns of immediate withdrawals are made when funds are deposited. In some cases, these funds are applied to payments that are difficult to reverse or may be suspected of fraud (i.e., international bank transfers to offshore digital asset exchanges or gambling sites).
As FinCEN reiterates, it is important to “conduct ongoing monitoring to identify and report suspicious transactions, and on a risk basis, to maintain and update customer information.” Implementing and enforcing due diligence processes and procedures is one way to validate a financial institution’s reasonable belief it knows the “true identity” of its customers (or member).
What are financial red flag indicators of deepfake media abuse?
FinCEN has identified “financial red flag indicators” of deepfake media abuse to help financial institutions detect and prevent suspicious activity that may be related to GenAI tools. Identifying a single red flag is not indictive of suspicious or illicit activity. FinCEN reminds financial institutions to consider the facts before making a final determination that any transaction and/or activity is “suspicious or associated with illicit use of GenAI tools.”
Below are the red flags FinCEN has identified:
- A customer’s photo is inconsistent (i.e., visual signs of being altered) or is inconsistent with other identifying information (i.e., a customer’s date of birth indicates they are much older or younger than the photo suggests).
- A customer presents multiple identification documents that are not consistent with each other.
- A customer uses a third-party webcam plugin during a live verification check. Alternatively, a customer attempts to change communication methods during the live verification check due to excessive or suspicious technological glitches during remote verification of their identity.
- A customer declines to use multi-factor authentication to verify their identity.
- A reverse-image lookup or open-source search of an identity photo matches an image in an online gallery of Gen-AI-produced faces.
- A customer’s photo or video is flagged by commercial or open source deepfake detection software.
- Gen-AI-detection software flags the potential use of GenAI text in a customer’s profile or responses to prompts.
- A customer’s geographic or device data is inconsistent with the customer’s identity documents.
- A newly opened account an account with little prior transaction history has a pattern of rapid transactions; high payment volumes to potentially risky payees, such as gambling websites or digital asset exchanges; or high volumes of chargebacks or rejected payments.
If a financial institution identifies suspicious activity that may be linked to Gen-AI, FinCEN has requested the financial institution reference this alert in the Suspicious Activity Report (SAR) by “including the key term “FIN-2024-DEEPFAKEFRAUD” in SAR field 2 (‘Filing Institutions Note to FinCEN’) and in the narrative to indicate a connection between the suspicious activity being reported and this alert. Financial institutions should also include any applicable key terms indicating the underlying typology in the narrative.”
What are some best practices?
FinCEN and the NSA, FBI, and CISA have identified best practices to help reduce a financial institution’s vulnerability to “deepfake” identity documents.
Best practices include:
- Incorporating multi-factor authentication (MFA), including phishing-resistant MFA.
- Performing live verification checks (“real-time verification”) to confirm an identity through an audio or video process. Criminals will try to confirm their identity by using synthetic audio or video responses that may be inconsistent.
- Monitoring actors using deepfake identities to avoid or circumvent live verification checks. For instance, criminal actors may try opening an account using GenAI documentation and claim they are having technical glitches or request to change communication methods.
- Utilizing third-party webcam plugins where a customer (or member) can display their video.
- Develop plans for employees to respond to deepfake techniques.
- Train personnel to identify deepfake techniques.
- Report and share experiences of malicious deepfakes with U.S. Government partners, including the NSA Cybersecurity Collaboration Center for Department of Defense and Defense Industrial Base Organizations and the FBI (including local offices or CyWatch) to spread awareness of trending malicious techniques and campaigners.
For questions or comments regarding this alert, please contact FinCEN.