Until the 1980s, industrial systems such as water treatment plants, electric grids and recycling centers were inherently disconnected, each operating independently of other industrial systems. The commercialization of the internet allowed these disparate structures to be connected to each other and even larger networks, which makes their monitoring and management easier, but also makes it easier for nefarious actors to hack into one system and gain access to a whole topology of connected machinery.
These are called SCADA systems — supervisory control and data acquisition systems. As the number of these control systems grows — such as energy monitors for your home, smart meters, air conditioning units, and more — and cybercrime becomes an increasingly pressing concern, cybersecurity firms are filling this gap, offering standalone SCADA security tools or adding SCADA security onto existing products.
How effective are these tools, and do they really do what they promise? A 10-member team at the Space and Naval Warfare Systems Center Pacific (SSC Pacific) has been answering those questions for three years, and recently signed a limited purpose cooperative research and development agreement (CRADA) with a cybersecurity firm to evaluate the company’s tool designed to prevent phishing attacks and malware. This is the 13th such collaboration for the Cyber-SCADA Evaluation Capability (C-SEC) team; each time the team tests the partner company’s product and evaluates it on a set of 150 metrics.
The C-SEC project, which is funded by the Office of Naval Research, has several aims.
“One aspect of C-SEC is the ability to survey the market and understand what’s available in terms of cybersecurity technologies with a bent towards critical infrastructure,” said Dr. Jose Romero-Mariona, C-SEC project lead. “This also allows the company to understand a bit more about the environment that we’re bringing it into.”
Not only does the company have a chance to have its product vetted and make improvements if desired, but various Department of Defense stakeholders can access the test results to know which tools might be most effective in military settings.
“At the end of the day, the goal was to get a copy of a company’s product, be able to understand what the product does, and ultimately integrate it in our laboratory to study how well it performed based on what it promises to do and what it actually delivers,” Romero-Mariona said.
Some of the metrics on which each tool is evaluated, in addition to the standard detection, firewall and antivirus capabilities, include price, ease of use, and whether it unintentionally adds vulnerabilities to the network.
The C-SEC laboratory is unique, and mimics a variety of SCADA systems with equipment not typically present in a cyber lab. The lab has a water pump, smart meter, electric motor and scaled-down version of a manufacturing line that are used for stress testing each cyber tool evaluated. This approach allows for a clearer picture of how the product would perform in a real-world situation.
Eventually, Romero-Mariona would like to see the C-SEC tool become a standardized platform on which to evaluate new cyber products — like the Consumer Reports of the cyber world. Additionally, he would like the tool to provide some sort of predictive recommendations of suitable products for users, based on what they deem as important criteria and the scenario that they need to use the tool.
The C-SEC team is starting to develop what they call C-SEC On The Move — a mobile kit of critical infrastructure equipment that can be taken on a ship or installation to measure the effectiveness of tools in use, and offer suggestions of tools to fill any gaps.
Roger Hallman, a scientist working on C-SEC, added that having unbiased, objective metrics to grade cyber tools in an infrastructure environment is necessary since many people in the infrastructure industry don’t have backgrounds in cybersecurity.
“This is a space where a lot of the people who make decisions don’t have expertise in cybersecurity,” Hallman said. “So a good salesman can tell you what your needs are, whether or not that’s actually the case. This is a tool that can help that non-security expert understand what their needs are, apart from the salesman.”
Space and Naval Warfare Systems Center Pacific provides the U.S. Navy and military with essential capabilities in the areas of command and control, communications, computers, intelligence, surveillance, and reconnaissance (C4ISR). SSC Pacific provides the full spectrum of C4ISR capabilities from basic research and prototype development, to extensive test and evaluation services, through systems engineering and integration, to installation and life-cycle support of fielded systems. SSC Pacific is a recognized leader in the cyber domain and for autonomous unmanned systems, and is providing the technological and engineering support critical to ensuring the Navy’s information warfare superiority.