PetaVision

Plans B

However, team members doubt that scaling-up alone will do the trick. So they are developing other schemes in parallel with the scaling-up approach.

One tack is based on the facts that lateral and feedback neural connections kick in after the feed-forward processes do and that humans identify objects more accurately when scenes are presented to them for at least 50 milliseconds. If the second fact follows from the first, including lateral and feedback connections could improve the model.

In fact, last summer's Roadrunner speed record was set by including lateral connections in a team-developed program called "PetaVision."

PetaVision simulated only area V1—not the entire "what" pathway. Nor did PetaVision include feedback connections. So, PetaVision couldn't test whether lateral and feedback connections can together or separately improve the performance of the entire visual-cortex model. However, PetaVision did show that lateral connections can be important in processing visual information, and the code paved the way for testing the effects of lateral connections in models that include all four major areas of the visual cortex. It also tested some other promising approaches.

PetaVision's neurons were edge detectors whose lateral connections to other neurons were "weighted" to detect smooth curves in the simulated visual field; the weighting was derived from the results of experiments. First, as in PetANNet, each small square of pixels in the visual field was analyzed by a stack of feature detectors—in this case, only edge detectors. However, unlike in the feed-forward model, a PetaVision neuron that detected its targeted edge orientation sent lateral signals to other neurons. If a nearby neuron detected an edge that made a small angle with the edge detected by the first neuron, the weighting caused both neurons to send more signals to each other, generating a local feeding frenzy of neural activity. In this way, the neurons corresponding to segments of a smooth curve became highly active, while neurons corresponding to squares that were blank or contained edges with comparatively large angles were suppressed and became listless.

This weighting of lateral connections allowed PetaVision to do something PetANNet could never do in its present state: find the border of a circle (see figure below). This may not seem particularly earthshattering, but finding the borders of an object in one's visual field—which is called "segmentation"—is essential to identifying the object's location. Simulations of this sort could pave the way for exploring the more poorly understood "where" pathway of the visual cortex.

Bettencourt also points out that PetaVision found the circle using a prescribed weighting. However, the team plans to modify the software so the neurons learn how to find smooth contours on their own, just as we do.

Finally, in contrast to the very-simple neuron model in PetANNet, the electrical signals sent between PetaVision's neurons—and the neurons' responses to those signals—were modeled in biological detail.

Biological neurons talk to each other by sending out impulses, "spikes," of voltage. Each spike lasts about a millisecond. PetaVision modeled each spike's amplitude and duration, along with the spike's precise placement in time.

Precise spike timing is known to be used by the cortical tissue that processes auditory information, for example, in bats, "who are geniuses of sound," Bettencourt says. Some of the neurons in the auditory cortical tissue of bats locate sounds by measuring the difference between the placements in time of two spikes to a precision as small as 10 percent of a spike's duration. (The distance from a source of sound is usually slightly different for each ear, so the associated neural signals are slightly displaced from each other in time.) Moreover, Kenyon has studied spike timing in cat retinas, where its importance has also been shown. PetaVision's accurate spike-timing model could help the Synthetic Visual Cognition Team see if spike timing could be important in other cortical activities as well.

The "Input Image" was input to the PetaVision program, which was run on Roadrunner last summer. Using only the simple-cell and complex-cell neurons described on pp. 4 and 5, with lateral connections between the simple-cell neurons that were weighted to emphasize smooth curves, PetaVision found the circle embedded in the input image, as shown in the "Model Output."  Such "segmentation" tasks are key to determining the positions of objects in the visual field.

Click to enlarge