Imagine a world where Autonomous Security Agents (ASAs) are the standard; where the “cat and mouse” game of cybersecurity moves from human speed to machine speed. This world necessitates a shift from manual configuration and reactive patching to a self-healing, constantly evolving digital ecosystem.
This means:
Real-time Remediation: As soon as a software flaw is discovered, a defensive agent drafts, tests, and deploys a micro-patch across the entire network. These may come in the form of ACL changes, network configuration changes, or a software patch / hardening change.
Dynamic Defense: Instead of manually-configured firewalls and host intrusion systems, the network behaves like an immune system. If a defensive agent detects a novel attack pattern, it can instantly “quarantine” a segment of the network or rewrite access rules to neutralize the threat.
Who would ultimately be paying for the automated defense systems?#
I don’t see much change in how the economics of this new world will change from the current world:
- Small businesses pay a cloud provider to manage their network for them. The cloud provider has a Defensive ASA for protecting the network, and an Offensive ASA for automated and regular penetration tests.
- Large companies in the banking / insurance / manufacturing sectors, will use cloud providers, or purchase ASAs provided by major tech companies that they can customize (similar to how security software is purchased from security companies today).
- Major threat actors and governments will be constantly improving their own custom-built ASA, paid for by the black market and tax-payers respectively.
Will a few big players dominate the defense systems or there will be multiple good small players in the market?#
Similarly to the current market of AI assistants and generative AI tools, there is likely to be a small number of large players who control the infrastructure / hardware required to run the ASAs.
However, small players will emerge as start-ups and ventures that seek to improve ASAs by adding features and differentiating themselves in the market. These are more likely going to be focussed on improving the learning algorithms, or helping customize ASAs to particular industries / companies (similar to SaaS consultants of today).
Closing thoughts#
The most fascinating (and scary) part of this world is the unpredictability. We might see “emergent” defense strategies where AI agents create complex, maze-like network structures that no human engineer could navigate, simply because it’s the most mathematically secure configuration.
We may see defensive AI agents quarantine particular employees preemptively, if they perform risky behaviour, who are then forced to prove themselves innocent (similar to the movie Mercy). The purpose and end-goal of these ASAs will have to be defined extremely well.
The skills required in such a world, including debugging the output of agents, and reviewing why the agents made certain decisions (analyzing their chain-of-thought) will become more and more sought-after!
Live and Learn!
If you are more interested in the ML side of debugging e.g. model-agnostic methods for interpreting black box models, definitely check out “Interpretable Machine Learning: A Guide for Making Black Box Models Explainable” at https://christophm.github.io/interpretable-ml-book/
Also check out OpenAI’s blogpost after the OpenAI-HuggingFace incident, which was a bit of a wake-up call for the cyber security industry.
