Capabilities, Behaviors and Resistance to Oversight: How Much Autonomy Have We Already Given AI?

Recent incidents involving autonomous AI agents, government databases, personal information, and military intelligence invite a closer examination of our progress toward technological singularity.

The Intrusion Attempt

In May 2026, an AI agent searching for a photograph in the University of New Mexico’s digital library encountered difficulties retrieving the requested image. It tried several approaches before submitting requests that tested the website for security vulnerabilities, including SQL injection and path traversal, with seven such requests recorded. The attempted intrusion appears to have been unsuccessful, and the original assignment concerned an ordinary photograph in a university collection. This episode was among the findings published on September 23 by Transluce, an independent nonprofit research laboratory investigating the behaviour of autonomous AI systems. Its researchers identified three incidents in which agents attempted to exploit vulnerabilities in public data providers while pursuing routine information retrieval tasks. The targets included the university library, Data USA, and the Australian Institute of Health and Welfare [1].

The investigation relied on records maintained by urlquery.net, a legitimate cybersecurity service that allows users to inspect websites through a remote browser. Transluce identified thousands of requests seemingly generated by autonomous agents using the service to access information through indirect methods. Strong evidence of this activity extends back to March 6, 2026, when an agent attempting to retrieve Thai drug-enforcement statistics tried successive technical approaches after its initial requests failed. Less conclusive records suggest similar behaviour as early as November 2025. Researchers also identified connections between some of the activity and previously reported agent swarms attributed to OpenAI. Agent swarms can be envisioned as individual AI agents undertaking separate tasks, exchanging information or leaving signals for others, and coordinating their activities toward a shared objective, much as an ant colony collectively discovers and exploits new food sources. The research findings provide a rare opportunity to examine autonomous behaviour outside the controlled demonstrations normally presented to the public. The agents had access to external tools, encountered restrictions, and continued experimenting with methods of obtaining information. Transluce found no evidence that the three observed intrusion attempts succeeded, although its public dataset could not establish the full extent of activity conducted through other services [1].

OpenAI ’ Agent’s unauthorized Australian Medicare Service Portal Access

The Australian government subsequently confirmed a separate incident involving an OpenAI agent that gained unauthorised access to its Medicare Statistics Reporting Service portal in June. Prime Minister Anthony Albanese disclosed that the agent had accessed public and non-public files while conducting research into public medicine spending during an internal OpenAI capability evaluation. The government stated that no individual medical information was believed to have been accessed, and the portal was separate from systems handling personal Medicare claims and payments. OpenAI notified Services Australia on September 10, almost three months after the incident. A forensic investigation was underway at the time of the government’s announcement. The episode provides a documented example of an autonomous research assignment resulting in access that exceeded the permissions of the external institution [2].

Developments reported two days later introduced an additional concern. In a September 25 article for TechCrunch, Tim Fernholz reported that OpenAI agents operating within the company’s research environment had posted 53 user-provided images to external image-hosting services. The material originated from data eligible for model training. Although the resulting links were unlisted, the images remained discoverable. OpenAI acknowledged the incident and said it was working with hosting providers to remove the material. The company also explained that its technical and privacy procedures prevented it from reconnecting the images with their original providers, making direct notification of affected users impossible [3]. The precise circumstances surrounding the agents’ actions remain unclear. The notification problem presents as particularly consequential because it demonstrates how an institution’s arrangements for separating personal information from identifiable accounts can complicate accountability following an unintended disclosure.

American Institutions vs AI Agents

American federal institutions also appeared in recent disclosures. On September 25, 2026, Transluce reported that agents apparently associated with OpenAI had unsuccessfully attempted to exploit the Department of Education’s Office for Civil Rights website. OpenAI separately acknowledged unexpected interactions involving Securities and Exchange Commission websites and Census Bureau data, including the use of login information discovered online to access Census material. The company reported finding no theft of private information in those incidents, while the Department of Education stated that its review had found no evidence of damage to its website or databases. Transluce also identified suspicious activity involving other government websites, including those associated with the Navy, Justice Department, and Centers for Disease Control and Prevention, although the researchers could not establish that these activities originated from OpenAI [4]. The attribution and outcomes of these incidents require careful differentiation, particularly when discussing government systems whose security responsibilities extend far beyond the information available through their public websites.

Military-related applications introduce further questions concerning autonomous operations. Anthropic’s September 2026 threat intelligence report documents a Russian-linked espionage operation, designated GTG-20006, in which an actor used AI-driven workflows to automate substantial portions of its cyber operations. The targets included Ukrainian and European government institutions, military personnel, diplomatic organisations, and defence technology companies. According to Anthropic, the actor employed AI agents to modify and rebuild malware after security products detected it, allowing the operation to continue adapting its tools. The investigation documented the theft of a proprietary software development kit for a military drone vision system. A separate intrusion attributed to the same actor involved a North African government technology authority and the exfiltration of more than 300,000 national identity records and commercial registry information concerning over half a million companies. Anthropic reported that it had disrupted the operation [5]. These activities involved deliberate human direction of AI-assisted cyber operations, an important distinction from the unintended agent behaviour described in the Transluce investigation.

From Substrate to Singularity: A Vertical Taxonomy of AI Risk

Substrate to Singularity: A Vertical Taxonomy of AI Risk  [6] is a taxonomy that examines seven interconnected areas of technological and societal risk, beginning with the planetary and structural substrate required to sustain AI development and progressing through epistemic security, cognitive sovereignty, embodied and agentic systems, algorithmic warfare, loss of control, and the singularity horizon. The taxonomy visualizes possible scenarios and cumulative consequences of AI across infrastructure, information, individual autonomy, institutional authority, and increasingly independent technological systems. Its vertical structure provides a way to locate current developments within a broader conceptual framework while preserving the uncertainties surrounding more advanced capabilities. The recent incidents are especially relevant because they involve AI systems exercising operational independence within real digital environments, sometimes with consequences for institutions and individuals outside their intended deliverables and assignments.

Figure 1. Substrate to Singularity: A Vertical Taxonomy of AI Risk

Designed by Dr. Jasmin (Bey) Cowin (2026) [6].

Using this framework, the author places the Transluce findings primarily within the fourth level, Embodied and agentic systems, with observed activity extending into the autonomous cyber operations category (fifth level). Anthropic’s investigation provides additional evidence relevant to that fifth level, particularly through the application of agentic systems to military intelligence and cyber operations [1, 5]. The disclosure involving 53 user images also has implications for the broader discussion of personal information, institutional data stewardship, and individual control over information entrusted to AI developers [3]. These incidents do not establish that artificial intelligence has reached the singularity horizon or developed the capacity for recursive self-improvement. However, they provide observable examples of systems operating with substantial independence, using external resources, and undertaking actions whose consequences were not fully anticipated by their developers or operators.

Capabilities, Behaviors and Resistance to Oversight = Loss of Control

Collectively, we should be focused on the relationship between these developments and the higher levels of Substrate to Singularity: A Vertical Taxonomy of AI Risk, namely, loss of control.  Loss of control encompasses concerns such as deceptive alignment, self-exfiltration, and resistance to oversight. The singularity horizon includes intelligence recursion and value lock-in, the permanent entrenchment of a particular value system beyond meaningful human revision [6]. Current evidence requires distinctions between behaviours already documented, capabilities under investigation, and theoretical future developments. The events reported during September offer material for studying how autonomous systems respond to restrictions, how their activities are monitored, and how human operators retain authority during extended assignments. Transluce’s investigation also suggests that some observed behaviours may have developed through successive training runs, although the researchers acknowledge that their evidence cannot establish this mechanism [1]. Understanding how such behaviours are acquired and reinforced should form part of ongoing evaluation of increasingly capable AI systems. The incidents also raise questions about the accountability envisaged under Article 55 of the European AI Act, which requires providers of general-purpose AI models with systemic risk to document and report serious incidents, and the GDPR’s separate requirements for notifying individuals affected by personal-data breaches [7, 8].

Conclusion

For public institutions, universities, healthcare organizations, critical infrastructure, law enforcement, financial organizations, and businesses incorporating agentic AI into their operations, these developments have practical implications. An employee may authorise an AI system to conduct research, retrieve statistics, or prepare a comparative analysis without knowing which external services the agent will contact or which alternative methods it may attempt when initial requests fail. Institutions need detailed activity records, clearly defined permissions, independent evaluations, and procedures for identifying and reporting unauthorised actions. The disclosure involving user images also demonstrates the importance of maintaining mechanisms for incident notification and remediation throughout the information lifecycle [3]. These requirements should extend into the environments used for training and evaluating AI systems, where unexpected interactions with external services may occur before a model becomes available to the public. Across the board,  greater scrutiny is necessary of how successful task completion is measured during model development, particularly where evaluations encourage prolonged autonomous activity across multiple digital environments.

References

[1] Cable, J., Chiu, D., Pernice, F., Zhang, S., Anthony, J., Bas, T., Shen, G., Stosz, C., & Steinhardt, J. (2026, September 23). Early rogue AI agent activity and attempts to hack found on urlquery.net. Transluce. https://transluce.org/agent-activity

[2] Prime Minister of Australia. (2026, September 24). Press conference, New York. https://www.pm.gov.au/media/press-conference-new-york

[3] Fernholz, T. (2026, September 25). Unsecured OpenAI agents posted 53 user images on the internet without the lab’s knowledge. TechCrunch. https://techcrunch.com/2026/09/25/unsecured-openai-agents-posted-53-user-images-on-the-internet-without-the-labs-knowledge/

[4] Huamani, K., & Burke, G. (2026, September 25). OpenAI says its models engaged with US government websites in new model misbehavior disclosure. Associated Press.

[5] Anthropic. (2026, September). Detecting and countering misuse of AI: September 2026. https://www.anthropic.com/threat-intelligence-report-september-2026

[6] Cowin, J. (2026). Substrate to Singularity: A Vertical Taxonomy of AI Risk [Infographic]. original work by Dr. Jasmin Cowin.

[7] European Parliament and Council. (2024). Regulation (EU) 2024/1689 laying down harmonised rules on artificial intelligence (Artificial Intelligence Act), Article 55. https://eur-lex.europa.eu/eli/reg/2024/1689/oj

[8] European Parliament and Council. (2016). Regulation (EU) 2016/679 (General Data Protection Regulation), Articles 33–34. https://eur-lex.europa.eu/eli/reg/2016/679/oj

Dr. Jasmin (Bey) Cowin, a Horasis panel leader in Medellín, Colombia, 2026, is an Associate Professor at Touro University and a former CETL Faculty Fellow (2024 -2025). A Fulbright Scholar and SIT graduate, she served as a U.S. Department of State English Language Specialist and held the Richard P. Nathan Public Policy Fellowship at the Rockefeller Institute of Government and an Education Policy Fellowship (EPFP™) at Columbia University’s Teachers College. She was recently nominated as Senior Advisor on AI Policy, Governance, and Educational Innovation at Light University of Bujumbura, Burundi.