Observations
Defining AI Agents and the Hype: AI agents are envisioned as autonomous systems that can perceive their environment, act proactively, interact socially, and pursue goals without human intervention. This vision fuels significant interest, with search trends surging and a growing number of IT executives planning near-term investments, often based on a poor understanding of the efficacy of the current technology. However, this excitement clashes with a core technical limitation.
The DeepMind Proof – Why a Causal Understanding is Essential: the paper ‘Robust Agents Learn Causal World Models’1 provides a crucial insight. It mathematically proves that any agent adapting to a wide range of environments must learn a causal model of the data-generating process. This means that to truly act autonomously and effectively in varied situations (like a self-driving car navigating different road conditions), an AI needs to understand cause and effect, not just correlations.
- Causal Models vs. Associative Models: the difference is critical. An associative model identifies patterns in data, while a causal model understands how actions lead to specific outcomes. For example, an associative model might note that people who go to the doctor are more likely to be sick. A causal model understands that going to the doctor doesn’t cause sickness; rather, sickness causes people to visit the doctor. Failing to distinguish between correlation and causation can lead to faulty assumptions and incorrect conclusions.
- The Richness and Difficulty of Causal Models: causal models are far more informative than associative models, essentially capturing a complete understanding of a system. However, they are also significantly harder to develop, with theoretical limits to what can be known.
The Current State of Causal Modelling: today’s causal modelling, which often relies on experimentation and causal inference, is a slow, narrowly focused process. It requires significant human input and isn’t easily automated.
- Identifiability Problem: causal models aren’t directly derivable from data. Modellers must make assumptions based on their understanding of the system, which are often difficult to validate.
- Limitations of Causal AI: causal AI requires high-quality data that captures both correlations and context. In practice, such data is often scarce or costly, posing challenges for establishing accurate causal relationships. Even when data is available, it may be incomplete or biased, skewing causal inferences. Causal models often assume that all relevant variables have been identified and correctly measured, but unmeasured confounders can distort causal estimates. Scalability is also a significant challenge, as building and validating models is a complex and resource-intensive process. These models often require tailored adjustments for new contexts, limiting their generalisability.
Potential Breakthroughs: while the limitations are significant for agentic AI solutions, the field is evolving. One potential avenue is the application of increasingly powerful reasoning models, which have shown promise in complex mathematical or logical tasks, to causal modelling.
- Reasoning Models and Causal Inference: reasoning models may help automate the intuitive reasoning required for causal modelling. However, unlike mathematics, causal inference lacks a definitive ground truth.
Ground truth refers to established, unchanging fundamental principles and foundational information, similar to axioms in mathematics, that can be definitively known and used to build upon. The lack of ground truth makes applying reasoning models challenging.
For example, in mathematics, ground truth is knowing 2+2=4: a universally accepted, verifiable principle. However, a causal inference would be a marketing campaign where there is no single, undeniable principle proving that a particular action was the sole cause of a sales increase or decline. Other unmeasurable factors may have contributed, making it challenging to isolate the genuine causal link. - Causal AI Revolution: causal AI is considered a key enabler of the next wave of AI, moving towards greater decision automation, autonomy, robustness, and common sense. It can efficiently identify critical information in datasets, discarding irrelevant correlations and creating simpler, more powerful models for superior predictions. Causal AI addresses the limitations of current AI systems, such as failing when faced with novel data, struggling to transfer learning to new tasks, failing in deployment when the world changes unexpectedly, and being susceptible to adversarial attacks.
Applications of Causal AI: causal AI is rapidly gaining traction across industries. In healthcare, it enhances diagnostic accuracy and treatment planning by identifying causal relationships between symptoms, treatments, and outcomes. In finance, it improves risk assessment and fraud detection through causal analysis. In marketing, it helps maximise strategies and increase returns by identifying factors driving consumer action. Causal AI also aids in supply chain optimisation, policy-making, and fraud detection. For example, causal AI analyses how soil quality, weather patterns, and irrigation practices impact crop health in agriculture, enabling precise interventions.
Next Steps
- Understand the limitations of Agentic AI: even with impressive outputs, agentic AI and reasoning AI solutions still suffer from the limitations inherent in their associative model foundations. This can be applied to tasks where association is beneficial, such as research and basic conversational services, as they are not able to incorporate causal relationships into their outputs. In short, agentic AI can be used to ask “What is related?” but not “Why is this related?”
- Focus on Hybrid Approaches: when implementing AI systems into core business processes, consider both associative and causal modelling. This might involve using AI to identify patterns and then employing causal inference to understand the underlying relationships.
- Embrace Experimentation: where feasible, prioritise experimentation to gather data that can inform causal models. A/B testing and other controlled experiments are crucial for understanding the impact of actions. However, be sure to consider how any such experimentation could be taken into production.
- Develop Robust Evaluation Metrics: create evaluation metrics that go beyond simple accuracy and assess an AI agent’s ability to generalise to new environments and understand cause-and-effect relationships.
- Explore Open-Source Projects: leverage open-source causal AI projects like PyWhy and CausalNex to accelerate development and adoption.
Footnotes
- ‘Robust Agents Learn Causal World Models’, Google Deepmind, 2024.
