Researchers at Space and Naval Warfare Systems Center Pacific (SSC Pacific) entered into a Cooperative Research and Development Agreement (CRADA) with San Diego-based National University and its School of Engineering and Computing to study the effectiveness of cybersecurity tools from the human operator’s perspective.
These operators are tasked with completing an ever-changing job using upwards of 75 different, complicated tools that aren’t intuitive for the user.
Through the CRADA, a team of researchers will investigate human performance during simulated cyber defense tasks to better understand how cyber operators use the bevy of tools available to them, and learn what functionalities and interfaces within these tools are the most effective. This is a new, untapped area of research with growing importance, as the Department of Defense enters the Third Offset Strategy focused on technology and information warfare, and private industry is increasingly under threat of cyber breaches.
“What we’re really trying to do is assess how well people are doing in this task of cyber defense,” said Dr. Robert Gutzwiller, a scientist in the user-centered design and engineering group at SSC Pacific and lead researcher in the CRADA. “If we throw attacks at operators in a simulated network environment, are they catching them? Are they catching them quickly? How hard was it for them to catch them? Is the software helping them or hindering their situation awareness of malicious activities?”
National University’s cybersecurity program is recognized as a Center of Academic Excellence in Cyber Defense Education by the National Security Agency and Department of Homeland Security — the only such program in San Diego County to achieve this status. Through the agreement, student volunteers who are in the University’s cybersecurity program will have the opportunity to contribute to the research by finding flaws in a simulated environment using different sets of tools and interfaces. Faculty, who provide on-site project oversight, will also contribute through research assistance.
Surprisingly little research has been done to learn how cyber defense analysts actually perform their job, Gutzwiller said. He added that the evolving nature of cyber threats, coupled with only a loose and recently introduced architecture for cyber training, means different professionals approach problems using different methods and different tools. There are few studies demonstrating which methods, tools and interfaces are the most effective.
“If you’re making cyber tools really complicated you’re introducing two problems; the first is a steep learning curve for a new user. And the second is the sheer complexity of completing a task. What we know from decades of research is that under stress and under fatigue, operators can’t handle those things,” Gutzwiller said. “They will make mistakes. cyber defense is one of those areas where we can’t afford to make any mistakes.”
Eventually, the collaborative team hopes to gather enough information to create recommendations for developers of cybersecurity platforms to design their tools with the human user in mind, knowing at what point an interface becomes too complicated to use effectively, and what the most successful aids are for the cyber defender.
"This is a great opportunity for our students who will have access to additional realistic and challenging labs to practice their cybersecurity skills, while at the same helping advance important research being undertaken by SSC Pacific to refine assessments of cybersecurity tools,” said Chris Simpson, lead faculty for the Master of Science program in Cyber Security and Information Assurance at National University.
SSC Pacific provides the U.S. Navy and military with essential capabilities in the areas of command and control, communications, computers, intelligence, surveillance, and reconnaissance (C4ISR) through the full spectrum from concept to capability, and regularly enters into CRADAs with industry and academia to support its research and development mission.