The limits of deep learning in achieving artificial general intelligence

Renewed public interest in the potentials and hazards of artificial general intelligence (AGI) has been sparked by Sam Altman’s recent employment saga and speculation about OpenAI’s groundbreaking Q* model.

AGI has the ability to acquire and perform intellectual tasks comparable to that of humans. Optimism and apprehension regarding the emergence of AGI have been stirred by rapid advancements in AI, especially in deep learning, which have led to several companies aiming to develop AGI, including OpenAI and Elon Musk’s xAI. This leads to the question: Are current AI advancements leading towards AGI?

Perhaps not.

Limitations of deep learning

Deep learning, a machine learning (ML) method based on artificial neural networks, is used in ChatGPT and much of contemporary AI. Its popularity is due to its capacity to handle various data types and its reduced need for pre-processing. Many believe that deep learning will continue to advance and play a crucial role in achieving AGI.

VB Event

The AI Impact Tour – NYC

We’ll be in New York on February 29 in partnership with Microsoft to discuss how to balance risks and rewards of AI applications. Request an invite to the exclusive event below.

Request an invite

However, deep learning has constraints. Large datasets and expensive computational resources are needed to create models that mirror training data. These models abstract statistical rules from real-world phenomena and then apply those rules to current real-world data to generate responses.

Due to the sensitivity of these rules to the uncertainty of the natural world, they are less suitable for the realization of AGI. This was evident in the June 2022 crash of a cruise Robotaxi, attributed to the vehicle encountering a new situation for which it lacked training, rendering it incapable of making decisions with certainty.

The ‘what if’ conundrum

Humans, the models for AGI, do not formulate exhaustive rules for real-world events. Instead, they perceive the world in real-time, relying on existing representations to understand the situation, the context, and other incidental factors that may influence decisions. Humans repurpose existing rules and modify them as necessary for effective decision-making, rather than constructing rules for each new occurrence.

For example, if you are on a forest trail and come across a cylindrical object on the ground and need to decide your next step using deep learning, you need to gather information about different features of the object, categorize it as a potential threat or non-threatening, and act based on this classification.

Conversely, a human would likely begin to assess the object from a distance, update their information continuously, and opt for a robust decision drawn from a distribution of effective actions from previous analogous situations. This approach focuses on characterizing alternative actions in relation to desired outcomes, rather than predicting the future — a subtle but distinctive difference.

Achieving AGI might require moving from predictive deductions to enhancing an inductive “what if..?” capacity when prediction is not feasible.

Decision-making under deep uncertainty, a way forward?

Decision-making under deep uncertainty (DMDU) methods such as Robust Decision-Making may provide a conceptual framework to realize AGI reasoning over choices. DMDU methods analyze the vulnerability of potential alternative decisions across various future scenarios without requiring constant retraining on new data. They evaluate decisions by pinpointing critical factors common among those actions that fail to meet predetermined outcome criteria.

The goal is to identify decisions that demonstrate robustness — the ability to perform well across diverse futures. While many deep learning approaches prioritize optimized solutions that may fail when faced with unforeseen challenges, DMDU methods prize robust alternatives that may trade optimality for the ability to achieve acceptable outcomes across many environments. DMDU methods offer a valuable conceptual framework for developing AI that can navigate real-world uncertainties.

Developing a fully autonomous vehicle (AV) could demonstrate the application of the proposed methodology. The challenge lies in navigating diverse and unpredictable real-world conditions, thus emulating human decision-making skills while driving. Despite substantial investments by automotive companies in leveraging deep learning for full autonomy, these models often struggle in uncertain situations. Due to the impracticality of modeling every possible scenario and accounting for failures, addressing unforeseen challenges in AV development is ongoing.

Robust decisioning

One potential solution involves adopting a robust decision approach. The AV sensors would gather real-time data to assess the appropriateness of various decisions within a specific traffic scenario. If critical factors raise doubts about the algorithmic rote response, the system would then assess the vulnerability of alternative decisions in the given context. This approach could enhance AV performance by redirecting focus from achieving perfect predictions to evaluating the limited decisions an AV must make for operation.

Decision context will advance AGI

As AI evolves, there may be a need to move away from the deep learning paradigm and emphasize the importance of decision context to advance towards AGI. Deep learning has been successful in many applications but has drawbacks for realizing AGI. DMDU methods may provide the initial framework to pivot the contemporary AI paradigm towards robust, decision-driven AI methods that can handle uncertainties in the real world.

Swaptik Chowdhury is a Ph.D. student at the Pardee RAND Graduate School and an assistant policy researcher at nonprofit, nonpartisan RAND Corporation.

Steven Popper is an adjunct senior economist at the RAND Corporation and professor of decision sciences at Tecnológico de Monterrey.


Welcome to the VentureBeat community!

DataDecisionMakers is where experts, including the technical people doing data work, can share data-related insights and innovation.

If you want to read about cutting-edge ideas and up-to-date information, best practices, and the future of data and data tech, join us at DataDecisionMakers.

You might even consider contributing an article of your own!

Read More From DataDecisionMakers

Leave a Reply

Your email address will not be published. Required fields are marked *