485 VIEWS
Pinterest LinkedIn LINE

It appears that the economic damage caused by deepfakes is becoming staggering.

Watch this video on YouTube.
Playing the video connects to YouTube.

According to the SBBIT article “The ‘Improved Accuracy’ of Deepfakes Is Extremely Alarming: In 2027, Damage Will Exceed ¥6 trillion”, the following points have been raised.

Article Summary

Improved Deepfake Accuracy

  • Deepfake technology has advanced rapidly with the evolution of AI.
  • Early deepfakes were low quality and obviously fake.
  • From 2018 to 2019, AI-based image generation technology improved.
  • The emergence of GANs significantly improved image quality.
  • Since 2020, Transformer technology has improved consistency in longer videos.
  • In 2023, deepfake content increased 3000% year over year.

Damage Caused by the Malicious Use of Deepfakes

  • According to Deloitte estimates, fraud losses in 2023 are expected to rise from 12.3 billion dollars to 2027 levels of 40 billion dollars.
  • The compound annual growth rate is 32%, and over 4 years, losses will increase by more than 3 times.
  • New generative AI tools make it possible to create deepfakes at low cost.
  • The financial services industry in particular is increasingly being targeted.
  • In 2023, deepfake incidents in the fintech industry increased 700%.
  • Annual losses from contact center fraud using voice deepfakes are approximately 5 billion dollars.
  • In 2024, deepfake-related incidents are projected to increase 60% year over year and reach 150,000 cases worldwide.
  • There are concerns about non-consensual sexual content and forged identity documents.
  • An underground industry has formed in which fraud software is sold on the dark web.

Real-World Fraud Losses Caused by Deepfakes

  • Deepfake fraud targeting corporate executives is on the rise.
  • An example of a WhatsApp scam targeting the CEO of WPP, the world’s largest advertising agency group.
  • An example of an executive impersonation incident in Hong Kong that caused tens of millions of dollars in losses.
  • Reports indicate that cyberattacks involving the malicious use of AI are increasing.

AI-Powered Cyberattacks Beyond Deepfakes

  • According to an Ivanti survey, many companies report an increase in cyberattacks involving the malicious use of AI.
  • AI-driven cyberattacks are expected to increase further.
  • Threats of particular concern include phishing (45%), attacks targeting software vulnerabilities (38%), ransomware attacks (37%), and attacks targeting API vulnerabilities (34%).

Current State of Deepfake Countermeasures

  • Banks and other financial institutions have introduced fraud detection systems that use AI and machine learning.
  • JPMorgan uses large language models to detect email fraud.
  • Mastercard has developed a “Decision Intelligence” tool that predicts whether transactions are legitimate.
  • Existing risk management frameworks may not be able to keep pace with new AI technologies.

Nationwide Efforts to Counter Deepfakes

  • It has been pointed out that identifying deepfakes through visual inspection is becoming difficult.
  • OpenAI plans to provide a deepfake detection tool that uses its own AI. However, deepfakes are rarely created with a single tool, which limits the effectiveness of such tools.
  • The C2PA initiative is developing a standard that presents the production process of AI-generated content in a form similar to food ingredient labeling.
  • The UK government has conducted a “Deepfake Detection Challenge.”
  • Public awareness campaigns are underway.

Thoughts from an Identity Perspective

Generative AI affects identity in many ways. Deepfakes are one aspect of that impact.

In terms of measures for managing deepfake risks, the following will likely be necessary:

  • Originator authentication
    • Rather than relying on people to judge voices or facial images, always authenticate the originator using strong authentication before important transactions (a technical measure)
    • Organizational measures to ensure that this happens
    • Promoting digitization as a countermeasure against forged identity documents
  • Indicating the nature of the information being disseminated
  • Human-centered measures for implementing these countermeasures

These and other measures will likely be necessary.

Originator Authentication

One example of originator authentication is to respond to a request made by phone or video by always using CIBA1 to send a push notification to the preregistered device of the person the requester claims to be, and then authenticate that user.

At the same time, it is also important to assure employees that they will not be fired for insisting on this procedure when someone calls them. A typical tactic in this kind of fraud is to pressure a hesitant person by saying something like, “The company’s survival is at stake. If you don’t do it right now, you’re fired.” People must be protected from this kind of pressure. Technical measures alone cannot easily achieve this; organizational measures such as internal rules are needed.

For identity documents forged using deepfakes, transitioning to digitally signed documents is effective. Fortunately, Japan has services such as the Japanese Public Key Infrastructure and the Digital Agency’s Digital Authentication App. I believe these should be relied upon to conduct identity proofing at a high level of assurance.

Indicating the Nature of the Information Being Disseminated

Indicating the nature of the information being disseminated involves both how the information was generated and who originated it. This is extremely important for preserving the consistency of identity. For example, what would happen if someone created and disseminated non-consensual sexual content, or a video depicting a person committing a crime? If people believed it, their perception of that person would change and the person’s reputation would undoubtedly be damaged.

C2PA and Originator Profile serve as infrastructure for this purpose. They can indicate whether a video or image was created using generative AI and identify its originator. However, some care is needed here in relation to freedom of expression.

C2PA and Originator Profile (OP) are technologies that improve the trustworthiness of digital content, but each may affect freedom of expression differently.

C2PA and Freedom of Expression

C2PA is a technology for proving the origin and editing history of digital content, and is intended to prevent the spread of fake news and deepfakes. However, if this technology is misused, it could lead to restrictions on freedom of expression. For example, there are concerns that the C2PA system could be used to identify journalists, allowing governments to use it to restrict expression. Content tracking through C2PA could also be used to enforce specific laws.

Originator Profile and Freedom of Expression

Originator Profile is a technology for verifying the authenticity and trustworthiness of web content originators. It aims to deter disinformation and advertising fraud, but identifying originators could eliminate anonymity and restrict freedom of expression. In particular, if originator information is used improperly, it could encourage self-censorship.

Impact on Freedom of Expression

  • Privacy concerns: Both technologies collect and manage information about originators, raising concerns about privacy violations. This could make it more difficult for originators to express their opinions freely.
  • Risk of misuse: If these technologies are misused by governments or other powerful actors, there is a risk that freedom of expression will be restricted. Journalists and activists may be particularly vulnerable to targeting.
  • Transparency and accountability of the technologies: Transparency is needed regarding how these technologies are used and how data is managed. Without appropriate accountability, freedom of expression may be threatened.

These technologies are important for improving the trustworthiness of digital content, but protecting freedom of expression requires careful consideration of how they are used and managed.

Human-Centered Measures

The final category, human-centered measures, is also extremely important. After all, implementing technical measures is pointless if they are not used. This is quite difficult, however. Within an organization, members such as employees can be required to comply through organizational education and penalties. It is much harder to do this with the general public. I think this remains a challenge.

Conclusion

While the capabilities of attackers’ tools evolve exponentially, human skills do not advance in the same way. It is therefore impossible to counter these threats through skills alone without technological support. Strongly promoting technical countermeasures is essential.

At the same time, freedom of expression is also important when it comes to public communication, so excessive measures must be avoided. We must also remain aware of the difficulty of human-centered measures.

Taking all of these factors into account, it is essential to implement countermeasures in a balanced manner.

Footnotes

  1. a standard developed by the OpenID Foundation for authenticating users through a different channel

Related posts