907: Neuroscience, AI and the Limitations of LLMs — with Dr. Zohar Bronfman
Zohar Bronfman discusses why current LLMs are not on a path to AGI, contrasting their combinatorial creativity with the transformational, domain-general intelligence of humans. He argues that predictive models, not generative ones, deliver the most business value and explains how his platform, Pecan AI, automates the critical data preparation bottleneck to democratize predictive analytics for all businesses.
LLMs are a remarkable technological advancement, but they are not necessarily bringing us closer to Artificial General Intelligence (AGI). True intelligence is the ability to solve complex and unique problems in your environment, especially by transferring knowledge from one domain to another. This is a hallmark of both intelligence and consciousness.
The Limits of LLMs and the Path to AGI
Current AI models are highly effective in specific domains but show no signs of general problem-solving capabilities, especially when encountering novel problems that require leaning on knowledge from a different domain.
A key concept for achieving this is Unlimited Associative Learning (UAL), which describes the ability to apply a lesson learned in one context (e.g., making a sacrifice in chess) to a completely different one (e.g., a business negotiation). This capability is intrinsically linked to consciousness because it requires a unified, central value system—a sense of what is “good” or “bad” for the entity. LLMs lack this; they merely optimize a cost function related to dataset error, with no concept of survival or intrinsic value.
The next qualitative leap toward more general intelligence will likely come from new architectures, not just larger models. These architectures may draw inspiration from the brain’s complexity, which is far more than the simple neural networks we use today. Key areas for development include:
- Meta-learning: Architectures that can learn about and modulate the networks themselves, akin to the executive control function of the prefrontal cortex.
- Embodiment: Fusing neural networks with robotics or other forms of physical interaction with the world. This provides the necessary substrate for true knowledge transfer and the development of real value systems beyond just a loss function.
Combinatorial vs. Transformational Creativity
It’s important to distinguish between two types of creativity:
- Combinatorial Creativity: Assembling existing ideas in new ways. LLMs excel at this, for example, by combining joke structures from different comedians to create a new joke.
- Transformational Creativity: Taking a completely new perspective on a problem to create a new paradigm. This is what Albert Einstein did with the theory of relativity. This higher form of creativity requires the domain-general intelligence that current models lack.
Businesses risk conflating these two, overestimating the innovative power of generative AI while overlooking the foundational value of predictive models.
Predictive Models: The True Engine of Business Value
For businesses, the most significant value from AI comes from prediction, not generation. The success of a majority of tech giants like Google, Amazon, and Uber was built on predictive machine learning that optimized core business processes. While LLMs have utility, the primary vector for creating a real, needle-moving impact within a business remains predictive analytics. There is still massive untapped potential, with perhaps only 5% of machine learning’s potential being utilized by companies today.
This is partly because our brains make decisions hundreds of milliseconds before we are consciously aware of them, as shown in Benjamin Libet’s famous experiments. This means much of human behavior is predetermined by unconscious factors. For a business, this implies that customer behavior—purchases, churn, lifetime value—can be predicted well in advance by analyzing their historical data, allowing for proactive optimization of their experience.
The Data-Centric Challenge
The biggest challenge in building a predictive model is not the model training but preparing the data. An estimated 90% of a data science team’s time is spent on “modeling the data”—transforming it from its raw state into a format suitable for a predictive model.
This process is incredibly challenging because every company has a unique “data fingerprint.” It involves numerous complex steps:
- Defining the entity and the prediction label.
- Structuring the data and defining its frequency.
- Consolidating features from different sources.
- Preventing data leakage and model drift.
- Handling anomalies and class imbalances.
- Ensuring the final data aligns with the business framework.
Furthermore, in a business context, the framing of the problem is often more critical than marginal gains in statistical accuracy. A model that predicts churn with 70% accuracy 14 days in advance is more valuable than one with 95% accuracy only 7 days in advance if the latter provides no time to act.
Pecan AI: Democratizing Predictive Analytics
Pecan AI was created to solve this data-centric bottleneck. The mission is to make predictive analytics accessible to everyone, not just data scientists. The platform automates the complex and time-consuming data structuring and transformation tasks required to build a valuable business model.
It functions as a predictive agent or “copilot” with a conversational interface. It uses LLMs to understand the business goal and data semantics, then guides the user—who may be a data-savvy business analyst or marketer—through the process of building a model. It automates querying, feature engineering, and validation, enabling a rapid, iterative process. The goal is to empower organizations without dedicated data science teams to harness the power of predictive AI, while also allowing data scientists to accelerate their workflow and focus on more nuanced, complex challenges.