PetaVision

Can computers see as well as people do?

Brian Fishbine

The gift of sight is truly amazing. You “instantly” know what everything is in your field of view without seeming to think about it, without asking yourself, “What am I looking at?”
But to give you this knowledge, your brain must quickly make sense of the huge amounts of visual information constantly gathered by your eyes. How does your brain do it?
Scientists don’t know exactly.

“Brain research is in a pre-paradigm state,” says Garrett Kenyon, a Los Alamos neuroscientist and member of the Laboratory’s Synthetic Visual Cognition Team. “We know lots of things about the brain, but we don’t really know how it works.”

As a result, computer programs designed to emulate the way the brain processes visual information don’t begin to approach human levels of performance. For example, an MIT-developed computer-vision program—currently the most-accurate program at identifying objects—misidentifies what it sees 10 percent of  the time.

“Imagine that when you crossed the street, 10 percent of the time what you thought was a billboard was actually an oncoming truck,” says Luis Bettencourt, leader of the Synthetic Visual Cognition Project. “Clearly this sort of inaccuracy can be lethal  in the real world.”

So what’s missing in the computer programs? What do computers need in order to see as well as people do?

Finding the answer could one day help robots navigate through buildings and cities without running into walls or getting run over and let computers take the wheel of your car in an emergency. It could also allow rapid, automated analysis of the huge volumes of data beamed down each day from reconnaissance satellites or enable computers to identify faces in video taken at airports—a task at which existing computer methods fail dismally.

Understanding how the brain sees requires a good theory of how the brain works. But neuroscientists disagree about exactly what’s needed to formulate such a theory. Research teams all over the world, including the Laboratory’s Synthetic Visual Cognition Team, are exploring various possibilities, often aided by advanced supercomputers.

In collaboration with researchers at MIT and elsewhere, the Los Alamos team plans mainly to explore several mechanisms that could improve our understanding of how the brain processes visual information, which should lead to a better understanding of how the brain does all of its tasks. One of the team’s major goals is improving the performance of computer-vision software to human levels.

In addition to Laboratory neuroscientists and advanced-computing specialists, the Synthetic Visual Cognition Project features a fairly unique piece of Laboratory hardware—the Roadrunner supercomputer. Roadrunner set the record for supercomputer speed last summer, running software developed by the Synthetic Visual Cognition Team.