Director of Test & Evaluation Pursues Standard, Structured Testing Approach
The Defense Information Systems Agency (DISA) recently named Luanne Overstreet as the Director of Test and Evaluation (T&E). Overstreet has served as the acting T&E Executive since October 2011; prior to that she served as the deputy T&E Executive since May 2006.
Overstreet has worked in the T&E field for 26 of her 31 years with the Department of Defense (DoD).
Early in her career, she was selected to participate in an intern program at the organization that would eventually become the Joint Interoperability Test Command (JITC). (At the time, it was called the Joint Interface Test Force - Joint Interoperability of Tactical and Command & Control Systems, or JITF-JITACCS.)
“I remember I was being shown around by the executive officer and she was explaining the mission to me: ‘Our mission is to make systems talk to each other… sounds easy, doesn’t it?’ she asked me. I thought about it, and agreed that it did sound straightforward. ‘Well it isn’t,’ she told me. ‘We’ve been doing it for six years already.’”
That was in 1984; nearly 30 years later, it is still a critical mission.
“Interoperability testing is a mission that never ends,” said Overstreet.
It’s also a mission she is passionate about.
“A defining moment for me occurred in 1987,” said Overstreet. “The Air Force had developed a system intended to translate C2 information between the air C2 systems and the ground C2 systems.
In order to be certified interoperable, the system had to exchange data with a representative system from each service branch, as well as with the National Security Agency.”
During testing, the JITC test team could see and interpret the Air Force transmissions, but all other system owners were receiving unintelligible data. The test was stopped, and Overstreet worked with another programmer, an airman, for two weeks to isolate and identify the root cause of the problem.
They eventually identified that the Air Force system added one extra mark bit after a parity stream (essentially, an extra “1” in the binary code), which altered the message format and kept the other services’ systems from recognizing it as an authorized C2 system.
“I faxed the details to the airman and he changed the system code in an hour. We brought up the Army system and ran the test again… The Air Force system successfully communicated with it,” Overstreet said. “That was the defining moment; I knew as a tester I could make a difference.”
Twenty-five years later, she keeps a copy of her handwritten notes identifying the solution in her desk drawer as a reminder of what’s possible.
Overstreet is already in pursuit of new possibilities, leading DISA toward providing testing as a service.
The traditional testing model supports only one program at a time; it creates an individual test process for the program under review, dedicates a team to it, and keeps that team with the program for the lifecycle.
“We need to focus more on a standard and structured testing approach,” said Overstreet, “so that we don’t have to redesign a new testing process for each program.”
Her goal is to implement an enterprise testing strategy that can be tailored for use on all programs under test. The goal is to select from defined performance parameters to determine the level of “goodness” for each program, and in doing so we execute the “right testing at the right time”. This concept aligns with the Information Technology Infrastructure Library (ITIL) processes (a collection of best practices from information technology service organizations around the world).
The enterprise strategy will also “blend mobility and cyber T&E needs into it, keeping […] technologies synchronized through evaluation methods and adding flexibility to the [mobility and cyber] concepts.”
Accomplishing the transition to a testing as a service model and leading the organization through a time of tightening budgets is a challenge Overstreet is well-prepared for.
“[The testing community] is used to working with less and coming up with more efficient ways to do what we do… Testing is typically the tail of an acquisition lifecycle, where money is always tight,” she said. “Once our strategy is in place, our next step is to design a standard and structured process [to achieve it.]”
“Ultimately, T&E is responsible for protecting the enterprise and the users from poorly performing capabilities, as well as ensuring the delivery of effective, suitable, interoperable, and secure capabilities,” said Overstreet. “Staying focused on doing what is right and on what [we] set out to do is definitely the key to helping us accomplish our mission… even with less resources.”