No Generalization, No AGI

A significant drawback of DL is its lack of generalization capabilities. This limitation becomes evident in industries like autonomous driving, where substantial investments have been made based on DL. The technology has struggled to handle unexpected scenarios, leading to considerable financial losses. For instance, a deep neural network can only recognize a bicycle if it has been specifically trained to identify it beforehand.

In contrast, a person can recognize a bicycle immediately, even without prior exposure. This instantaneous recognition involves understanding various aspects such as shape, size, and spatial relationships with other objects. This ability to generalize is crucial; without it, intelligent systems cannot adapt to new situations effectively.

In the context of DL, perception is contingent upon prior recognition. This dependency is a critical flaw for achieving AGI. While one might argue that a person may not fully comprehend a bicycle's function upon first glance, it’s important to note that DL models, despite exposure to numerous bicycle images, also lack true understanding. In essence, classification does not equate to comprehension.

Hubert Dreyfus, Martin Heidegger, and the Concept of Generalization

Generalization is essential for intelligence, allowing for the immediate recognition of a wide array of objects and patterns. The approach of storing representations—common in DL—is not only impractical but also inefficient. In contrast, the human brain discards most perceptual details shortly after processing, retaining only what is necessary.

The late philosopher Hubert Dreyfus criticized the idea of creating stored representations as a flawed approach. He emphasized that intelligence must be context-aware and continuously responsive to changes in the environment. This principle aligns with the notion of generalized perception, which is vital for developing AGI.

Scaling as a Misguided Solution

The failure of the autonomous vehicle sector to produce fully self-driving cars can be attributed to DL's inability to generalize. Despite efforts to scale DL, this strategy has not resolved the fundamental issues. The infinite variability of edge cases makes it impractical to gather sufficient training data.

Similar to previous rule-based expert systems, deep neural networks exhibit fragility when faced with novel situations. Adversarial patterns pose significant challenges, highlighting the necessity for systems that can generalize effectively.

Generalization: An Innate Ability, Not a Scaled One

Some proponents of Large Language Models (LLMs) suggest that with enough data, generalization might emerge. However, this perspective overlooks the fact that true generalization is an intrinsic capability, not a result of scaling. Even small creatures like bees exhibit this ability, which is crucial for survival in complex environments.

Cracking AGI Without Massive Resources

Many in the AI community believe that substantial computing power is required for AGI development. However, we argue that a standard desktop computer could be sufficient for breakthroughs in this field. Generalization does not necessitate vast neural networks; even organisms with limited neurons, like bees, can navigate complex spaces effectively due to their ability to generalize.

Most future intelligent systems may not require human-level cognition. For instance, we estimate that full self-driving technology could be achieved with significantly fewer neurons than those found in the human brain.

The Pursuit of Generalized Intelligence

A number of researchers advocate for modifying DL to incorporate generalization and causal reasoning. However, we contend that generalization must be a foundational aspect of any intelligent system. Every component, including sensors and effectors, should be designed with generalization in mind.

The concept of signal timing and symmetry is fundamental to this approach. Biological neurons emit spikes to signify events, and this mechanism plays a crucial role in learning. Observations of nature suggest that integrating these principles could lead to a more effective understanding of AGI.

Conclusion

In summary, we argue that deep learning is not synonymous with true intelligence due to its inability to generalize. Scaling and superficial modifications will not yield AGI. Instead, a focus on generalization, signal timing, and symmetry may pave the way for future advancements. We are also preparing to introduce a speech recognition application that reflects our approach.

Section 1.1: Limitations of Deep Learning

Deep learning's dependency on prior representations poses a significant barrier to achieving AGI.

Subsection 1.1.1: The Case of the Autonomous Vehicle

Section 1.2: Philosophical Perspectives on Intelligence

Dreyfus and Heidegger's ideas provide valuable insights into the need for context-aware intelligence.