Award Abstract #0430474
Experiments in CyberSpace

NSF Org:	CNS Division of Computer and Network Systems
Initial Amendment Date:	September 2, 2004
Latest Amendment Date:	June 24, 2008
Award Number:	0430474
Award Instrument:	Standard Grant
Program Manager:	Karl N. Levitt CNS Division of Computer and Network Systems CSE Directorate for Computer & Information Science & Engineering
Start Date:	September 1, 2004
Expires:	August 31, 2009 (Estimated)
Awarded Amount to Date:	$1350000
Investigator(s):	Roy Maxion roy.maxion@cs.cmu.edu (Principal Investigator) Daniel Siewiorek (Co-Principal Investigator)
Sponsor:	Carnegie-Mellon University 5000 Forbes Avenue PITTSBURGH, PA 15213 412/268-8746
NSF Program(s):	ITR-CYBERTRUST
Field Application(s):	0000912 Computer Science
Program Reference Code(s):	HPCC,9218,7254
Program Element Code(s):	7456

ABSTRACT

Experiments in CyberSpace

Roy Maxion, Carnegie-Mellon University

Award 0430474

Abstract

It is important to be able to place high confidence in a detection system of any kind, particularly one intended for detecting attacks against the nation's critical information infrastructure. One requirement for establishing such confidence is to have a complete understanding of a detector's "sweet spots" and operational limits, so as to calibrate the detector optimally for the conditions under which it performs best. Due to a lack of standard test data sets and measurement procedures, such calibrations have not previously been done.

The proposed research will address methods of achieving high confidence in intrusion and malicious-insider detectors by developing: (a) metrics for gauging the effectiveness of detection algorithms; (b) gold-standard reference data sets, with calibrated ground truth, to be shared among producers and consumers of detection technologies, particularly for replication of scientific experiments that determine detection efficacy for new algorithms across a range of data conditions; and (c) a data synthesizer for producing reference and calibrated data sets.

This work will put decision makers in a position to know the flaws, the strengths, and the weaknesses of detectors before deployment. Knowing the operational limitations of one detector provides the opportunity to design a companion detector whose strengths compensate for the weaknesses of the other, enabling accurate and efficient composition of detectors for the first time.

Experiments in CyberSpace

Roy Maxion, Carnegie-Mellon University

Award 0430474

Abstract

It is important to be able to place high confidence in a detection system of any kind, particularly one intended for detecting attacks against the nation's critical information infrastructure. One requirement for establishing such confidence is to have a complete understanding of a detector's "sweet spots" and operational limits, so as to calibrate the detector optimally for the conditions under which it performs best. Due to a lack of standard test data sets and measurement procedures, such calibrations have not previously been done.

The proposed research will address methods of achieving high confidence in intrusion and malicious-insider detectors by developing: (a) metrics for gauging the effectiveness of detection algorithms; (b) gold-standard reference data sets, with calibrated ground truth, to be shared among producers and consumers of detection technologies, particularly for replication of scientific experiments that determine detection efficacy for new algorithms across a range of data conditions; and (c) a data synthesizer for producing reference and calibrated data sets.

This work will put decision makers in a position to know the flaws, the strengths, and the weaknesses of detectors before deployment. Knowing the operational limitations of one detector provides the opportunity to design a companion detector whose strengths compensate for the weaknesses of the other, enabling accurate and efficient composition of detectors for the first time.

PUBLICATIONS PRODUCED AS A RESULT OF THIS RESEARCH

(Showing: 1 - 5 of 5).

El-Arini, Khalid and Killourhy, Kevin S..  "Bayesian Detection of Router Configuration Anomalies,"  SIGCOMM-05 Workshop,  v.1,  2005,  p. 1.

Hansen, Jeffery P.; Tan, Kymie M.C. and Maxion, Roy A..  "Anomaly Detector Performance Evaluation Using a Parameterized Environment,"  Ninth International Symposium on Recent Advances in Intrusion Detection (RAID-2006) (LNCS),  v.4219,  2006,  p. 106.

Maxion, Roy A. and Roberts, Rachel R. M..  "Methodological Foundations: Enabling the Next Generation of Security,"  IEEE Security and Privacy,  2005,  p. 54.

Rachel R. M. Roberts, Roy A. Maxion, Kevin S. Killourhy, and Fahd Arshad.  "User Discrimination Through Structured Writing on PDAs,"  International Conference on Dependable Systems & Networks (DSN-07),  2007, 

Tan, Kymie M. C. and Maxion, Roy A..  "The Effects of Algorithmic Diversity on Anomaly Detector Performance,"  International Conference on Dependable Systems & Networks,  v.1,  2005,  p. 216.


(Showing: 1 - 5 of 5).

Please report errors in award information by writing to: awardsearch@nsf.gov.