Science 1663

The Cibola Satellite Tests a Reconfigurable Computer

Can an innovative supercomputer, built from off-the-shelf FPGAs, work in space? Los Alamos’ latest experimental satellite will find out.

As part of DoD’s Space Test Program, an Atlas V rocket carried aloft two Defense Advanced Research Projects Agency satellites and four small, experimental satellites, including Cibola. An innovative “launch adapter ring” allowed the four small spacecraft to “piggyback” into space. Ben Cooper/launchphotography.com

“Go Atlas!” The team in the launch control room shouted encouragement as the Atlas V rocket arced gracefully through the sky. After months of schedule changes and launch delays, Diane Roussel-Dupré and her Cibola Flight Experiment team were at last seeing their Cibola satellite—a sophisticated box no bigger than an armchair—ferried into space.

Cibola (pronounced see-bo-lah) is the newest satellite from Los Alamos National Laboratory. Its primary goal is to prove that an innovative supercomputer—developed over the past six years by Roussel-Dupré s close-knit team—can perform reliably in the harsh, radiation-filled environment of near-Earth space. If successful, the supercomputer could have a huge impact on the next generation of space-borne sensors.

"Los Alamos is in a unique position to field these pathfinder missions, which are necessary to validate new technologies," says Roussel-Dupré. "The DOE, DoD, NASA, and commercial enterprises cannot afford to fly something that may fail. The Laboratory has a very successful track record validating new technologies with the ALEXIS and FORTE satellite projects, two previous space-validation experiments undertaken at Los Alamos."

Artist’s conception of Cibola in orbit. The three radio antennas beneath the satellite are designed to sense the electromagnetic pulse generated from aboveground nuclear detonations as well as collect data about lightning storms.

The March 8 launch from Cape Canaveral, Florida, marked the end of a three-year marathon to prepare Cibola for space flight. Los Alamos was responsible for the entire mission. It built the all-important supercomputer payload and proved the satellite was space-ready with a series of exhaustive tests. The Laboratory even monitored the contract for the spacecraft. The satellite body itself was built in just 27 months by England’s Surrey Satellite Technology, Ltd.

At this writing, the diminutive spacecraft is safely in orbit. Systems are being methodically activated, and the ground crew is learning the ins and outs of flying its "bird."

The shakedown hasn’t been exactly storybook. There were a few surprises uncovered after launch. But that's par for the course.

(Left to right) Scott Robinson, Kimberly Katko, Diana Esch-Mosher, and Steve Knox watch the launch from the Laboratory’s Satellite Operations Center.

"Several are not problems, just 'undocumented' features," remarks Roussel-Dupré. "Actually, everything is functioning reasonably well, and the satellite is proving to be quite robust. But, yes, I do have a few more gray hairs."

Nuclear Lightning?

Underpinning Cibola's pathfinder mission is the desire to improve the ability of the United States to detect and locate above-ground nuclear explosions. A nuclear weapon emits a burst of radiation—neutrons, x-rays, and gamma rays—when detonated. The gamma rays in particular can collide with atoms in the atmosphere, freeing electrons that become accelerated in Earth’s magnetic field. Those electrons emit a broad spectrum of radio waves known as an electromagnetic pulse, a portion of which (30–300 megahertz) will penetrate clouds and Earth’s upper atmosphere and can be detected by a satellite.

Lightning can mimic the electromagnetic pulse of a nuclear explosion. Cibola’s on-board supercomputer will analyze detected events and determine which is which. Photo by Harald Edens, © 2003, www.weatherscapes.com

Unfortunately, ordinary lightning is a similar pulse of energy. To be effective, nuclear-detection sensors have to be able to distinguish lightning from a true weapon-generated electromagnetic pulse. The task is sufficiently complex to require analysis by a supercomputer.

On most detection satellites, the computing power is too limited to analyze the mass of data generated by either event. Thus, FORTE—Cibola’s immediate predecessor—relayed its data back to Earth for processing and analysis. But the communication could take place only when the satellite passed over a ground station, and there was only one such station. It’s a dish antenna located on the roof of the Laboratory's Physics Building. Between transmissions, the satellite stored events in its computer memory.

"We would fill FORTE’s memory in several seconds with a good lightning storm," says Roussel-Dupré. "Then we couldn't do anything else for the rest of the orbit, until it saw the ground station again and downloaded its data."

Cibola likewise has a downloading problem in that it talks to the Los Alamos ground station for only 10 minutes at a time, 6 times a day. But Roussel-Dupré and her collaborators envisioned a way to stretch the satellite’s memory: use an on-board supercomputer to extract and save the cream of an event, then discard the rest. Send only the processed data down to Earth.

It was a great idea, except for one thing: supercomputers don’t work well in space. Not well at all.

Chips in Space

Space is a harsh environment. Far from being empty, the ethereal region surrounding Earth is filled with radiation, primarily energetic charged particles from the sun and the ever-present cosmic-ray background.

Computer chips like the ones found in a desktop computer fare poorly in this harsh environment. The high-energy particles can smash into a chip and cause permanent damage.

The chips can also experience another type of radiation trauma known as single-event upsets. These are "soft errors,' wherein no physical damage occurs, but the output of a memory bit (or some other chip feature) changes, say, from 1 to 0. The result could be benign or, if the upset occurs in the wrong bit at the wrong time, radically change the outcome of a calculation. The latter did not bode well for the reliability of an orbiting computer.

One way to eliminate single-event upsets is to "harden" chips to radiation damage by giving them larger features. Unfortunately, that strategy lowers the overall chip speed and increases power consumption. Additionally, all satellite components need to be in final form years before launch, so by the time they reach space, hardened satellite computers are at least 10 if not 20 years behind their ground-based counterparts in both speed and functionality.

“We wanted to have a fast, reliable computer in space,” says Michael Caffrey, the payload computer’s chief engineer, “but also take advantage of the technology and cost advantages coming out of the commercial chip industry.” So in order to have their chips and launch them too, Caffrey’s team spearheaded a new approach to the computing-in-space problem.

A Space-Capable Computer

The idea was to investigate the use of an off-the-shelf, commercial chip known as a "field-programmable gate array" (FPGA). An FPGA can host millions of elements wired together into cells that carry out logic functions. By linking various cells, one can "configure" an FPGA to perform more-complex tasks that, in turn, can be strung together to create a data-analysis program.

Daniel Seitz inspects Cibola’s solar panels. The four deployed and two body-mounted panels on average provide 110 watts of precious electric power, barely enough to power a bright light bulb.

The beauty of the FPGA is that the internal linking isn't permanent but is established through programming. By optimizing the links, one can make the computer run very efficiently. Thus, Cibola’s FPGA-based supercomputer is very fast—roughly 100 times faster than what is currently available for space flight. Plus, the linking can be “tweaked” while the satellite is in orbit if better ways are found to differentiate lightning from a true electromagnetic pulse.

Furthermore, the computer can be reconfigured in less than a second to tackle a completely different science mission. So Cibola was planned with several missions in mind, to study lightning, for example, as well as to understand how conditions in the upper part of the atmosphere—the ionosphere—affect radio c

ommunications and other space operations. Still, there was a hitch. "The FPGAs are not radiation hardened," says Caffrey. "They will have single-event upsets."

Could they be hardened? Collaborating with teams from Brigham Young University and Xilinx, the FPGA manufacturer, Caffrey’s team worked for three years to study the problem, develop strategies, and test ideas.

The solution involved a clever tactic known as "triple modular redundancy." Suppose you could run an analysis program simultaneously on three identical (redundant) computers. Then, assuming that no more than one computer at a time can be corrupted by a single-event upset, you could compare the three outcomes to identify the correct result. In other words, you take a vote.

One of Cibola’s radio antennas during the satellite’s testing phase. Once Cibola was in orbit, a 2.3-meter-long antenna successfully deployed from the gold-colored canister.

Limited in volume and power, Cibola could not carry three computers. But Caffrey and his team could identify critical points within the analysis program where a single-event upset would affect the result. They could then configure redundant computational pathways and “voter” circuits at those points and thereby harden the analysis program.

But finding the critical points is challenging, often requiring months or even years of software design time. So Caffrey’s team and the Brigham Young team developed a software tool (the Brigham Young, Los Alamos triple modular redundancy tool, or BLTMR) to analyze FPGA configurations and produce a program appropriate for use in space. Caffrey states confidently that the BLTMR’s output "is not only space-qualified but also is more reliable than a program produced by a software engineer."

Diane Roussel-Dupré is the project leader for the satellite. ”I am honored to have been a part of the Cibola team.”

The BLTMR can also be applied to programs used on Earth. Massively parallel supercomputers, with thousands of processors all working simultaneously, are also subject to single-event upsets from terrestrial neutrons, a problem that will only get worse as chip features get even smaller and more numerous. The semiconductor industry is very interested.

Cibola is flying at the relatively low altitude of 350 miles and will likely stay in orbit for three to five years, depending on the amount of solar activity. Solar flares eject huge numbers of charged particles into Earth’s upper atmosphere. Those particles increase the drag on objects in low Earth orbit, causing them to lose energy and altitude until they eventually fall and burn up in a fiery descent. With any luck, Cibola will not meet this fate too soon, but it all depends on the sun. And no one can program that.