In recent weeks, media attention has been dominated by reports of a bug in Meta AI that allowed attackers to take over accounts on Instagram and other Meta services. At first glance, it may have seemed like yet another error caused by a “hallucinating” chatbot. In reality, however, the problem was much deeper and concerned the way AI had been integrated into account recovery and authorisation processes, experts from Check Point Research warn.
For some time, Meta had been developing AI systems designed to support users in recovering accounts, reporting abuse and resetting passwords. In theory, this was supposed to streamline the process and shorten response times. The problem was that AI was not merely acting as an “assistant” — it had been given real operational permissions. It could initiate actions that would normally require strong identity verification, such as changing the linked email address or sending password reset links.
This shifted the chatbot from an advisory role to an executive one, without sufficiently robust control mechanisms. According to cybersecurity specialists from Check Point Research, this was the mistake that made cyberattacks easier.
How Were the Accounts Taken Over?
Attackers used a simple but effective method. They provided the victim’s username, used a VPN to align their location with signals considered “trusted” by the system, and manipulated the facial analysis system until they were eventually able to force the tool to perform an operation changing the account recovery details.
As a result, the AI initiated a change of email address and launched the password reset procedure, making it possible to take over accounts — including profiles with high market value and strong public recognition.
Check Point experts stress that the core of the problem did not lie solely in a prompt injection vulnerability. The key architectural flaw was that the AI had access to administrative functions, lacked hard authentication checkpoints, and based its decisions mainly on the context of the conversation rather than on independent identity verification. In practice, this meant that if a user “sounded” convincing, the system could carry out critical operations.
A Broader Problem Than It May Seem
The incident is part of a wider trend involving the deployment of so-called AI agents — systems that not only answer questions, but also perform various actions on behalf of users.
“The key issue is not necessarily that the AI ignored instructions or was manipulated by a clever prompt. Rather, it appears that the AI was able to initiate or facilitate sensitive account recovery actions without sufficient, independent verification. In other words, the security failure may have involved the process itself, not the model,” says Steve Giguere, Lead AI Security Advocate at Check Point.
This creates new risks. AI may be manipulated into performing unauthorised actions, while the boundary between assistance and authorisation becomes blurred. At the same time, traditional security models — passwords, two-factor authentication and device signals — may be bypassed if AI becomes a gateway into the system.
“AI does not need to be directly attacked if it is granted permissions that go beyond the safeguards. As organisations deploy more and more agentic systems, security teams need to assess not only what AI can say, but also what it can do,” adds the Check Point expert.
Meta confirmed the vulnerability and implemented fixes, securing the affected accounts. The incident, however, has triggered a broader industry debate about the use of AI in security systems and the design of “trust boundaries” for AI agents.
Source: CEO.com.pl
