It’s the intelligence, stupid
The Next Breakthrough Is Intelligence, Not Hardware
I am continuing my series on the computer vision ecosystem. Everything needed to build a human-augmentation hardware device is within our reach: a camera module smaller than a fingernail, with enough resolution to read signs from across the street; a wireless radio that transmits video with no perceptible lag; and, amazingly, an AI accelerator chip, affordable even for a not-too-full purse, small enough to fit in a pair of glasses, and capable of running a real neural network at an operating temperature low enough not to need a fan.
Cloud-based AI can be slow, but for a device designed to be worn on, attached to, or embedded in your body or clothing, even a one-second delay is too much. To run directly and respond in milliseconds it requires making large models even smaller and faster through compression, quantization, and distillation. Making powerful AI work on tiny, low-power chips is still a major challenge.
Think, now, about a wearable device for blind and low-vision users. AI can recognize what is happening in a single image, but real life is a continuous video. Think about what your own eyes do when walking down a street. They do not describe every object in the visual field; they filter almost everything in a tiny fraction of a second: the object is moving towards me, that one is new since the last look. A perception system that announces every object in the frame, frame by frame, is not an assistant; it is noise. So, a wearable AI device of this nature needs a contextual and temporal understanding to remember what it saw a few seconds ago and use that information to understand what is happening now. For example, it should know that an object moved into view rather than simply appearing.
Embodied AI focuses on how AI understands and interacts with the real world, rather than just answering questions about images. This is also where more talent and money are moving.
Robotics is facing many of these same problems. Robots need to understand their surroundings, follow instructions, and move safely in real time. The solutions being developed for robots could also be important for future wearable AI.
The key point is that the next big advances in human augmentation probably will not come from smaller cameras or better-looking hardware. They’ll come from solving the harder problem: AI that can understand the world in real time, using little power, and know both what to say and when to stay quiet. This reality was on display at the second World Humanoid Olympics in China this past August where it became painfully clear that the hardware is there, but the software is simply not stepping up to the plate yet.
The hardware is no longer the main problem. The intelligence is. It is thus fundamental to improve the software models as much as the hardware.
