Detection, Enforcement, and Inspection
2. Imaging Metrology
In 2005, the Detection, Enforcement, and Inspection program started a new and broad program in imaging quality as applied to systems and devices used by emergency responder, law enforcement, criminal justice, and security providing agencies. Thus far, this program includes developing minimum performance requirements for the X-ray backscatter systems used for personnel screening, the portable X-ray systems used by bomb squads, the infrared cameras used by firefighters, visible-light digital video cameras used by police and other agencies performing surveillance, and radio-frequency through-wall surveillance and imaging systems. Each of these activities has unique requirements for evaluating the performance of the imager, including for example, unique test objects and test scenarios. This will be discussed in detail below.
This general program is directed to the development of metrology for a broad range of imaging technologies used in security, surveillance, and evidence-gathering applications. Although the imaging systems may use different sensor technologies, such as X-ray, infrared, visible light, and radio frequency sensors, there are many common requirements that can be addressed simultaneously. The common components include, for example, display quality, human-in-the-loop, the effects of algorithmic processes, and dissemination of the measurement capability to qualified testing laboratories.
The NIJ has a standard, NIJ Std-0603.01, "Portable X-Ray Systems for Use in Bomb Identification," for the portable X-ray systems used by bomb technicians to identify improvised explosive devices and subsequently plan strategies for their interdiction has been recently completed. OLES maintains this NIJ standard. This latest revision introduced mechanical tolerance, electromagnetic immunity, and environmental tolerance requirements. Another revision has been started to address imaging quality requirements. Presently, the image quality requirement is not rigorous nor does it lend itself to objective evaluation. We plan on separating this requirement into several metrics that can be objectively evaluated. We are also examining the possibility of establishing protocols for including the effects of human perception in these requirements.
2.1 Personnel X-Ray Anomaly Imagers
These anomaly imagers use x-ray backscatter and transmission technologies. Backscatter systems are currently being deployed at various types of security checkpoints for evaluation.
Figure 10. X-ray backscatter image of person with weapons concealed on front and back.
Goals
To define and develop an industry-accepted requirement for imaging quality along with associated test methods.
Customer Needs
Presently there are no methods defined to measure the performance of x-ray backscatter systems to image concealed objects. Technical Strategy Work in collaboration with manufacturers, users, and other government agencies to develop minimum requirements for imaging performance, which will become IEEE/ANSI Std. N42.47.
Deliverables
Minimum performance standard.
2.2 Portable X-ray Systems
Portable x-ray imaging systems are used by bomb squads to identify bombs in leave-behind packages and concealed inside packages. These systems comprise three separable components, the x-ray source, the imaging device, and the control electronics (usually a computer). The sources may use pulsed, continuous wave, or dual energy. There are several different technologies that can be used for the imaging device.
Figure 11. X-ray image of pipe bomb hidden in a box.
Figure 12. Bomb technician using a portable x-ray system to look inside a briefcase. The portable x-ray system include a source (in hand), an imaging device (on the side of the brief case opposite the source), and a remote control unit (shown in inset)
Goals
To replace the existing imaging quality requirement with a set of objective test metrics and develop associated test methods.
Customer Needs
The present image quality requirement is nebulous and does not lend itself to objective evaluation, yet imaging quality is the primary performance characteristic of these portable x-ray systems. To assist the bomb squad users and procurement officials in procuring a portable x-ray system and have assurances it will work as desired, the imaging quality requirement must be objective and measureable.
Technical Strategy
- Work with the National Bomb Squad Commanders Advisory Board, the International Association of Bomb Technicians and Investigators, and bomb squad technicians in identifying their operational requirements.
- Translate these operational requirements into objective test metrics.
- Develop test methods for the test metrics.
- Develop appropriate transfer standards.
- Establish laboratory test facility for assessing applicability of imaging requirements and ability of systems to meet these requirements.
- Develop objective performance metrics that include the effect of human perception on imaging performance.
Deliverables
- List of performance metrics.
- Revised NIJ standard.
- Development of test facility.
2.3 Visible-Light Video Surveillance and Security
These are the camera systems that will soon be replacing the analog video systems used in a plethora of security, surveillance, and evidentiary applications. The imaging system may include several components that are not part of the camera but are important for the utility of the imaging system for their targeted purposes. Applications of this work include police in-car video, perimeter/site security, and general surveillance.
Figure 13. Multiple video cameras on a pole.
Goals
To develop minimum performance requirements and associated test methods applicable to a broad range of video cameras used in surveillance and security.
Customer Needs
Presently there are no quantitative test methods for evaluating the quality of still and motion imagery acquired using surveillance and security video cameras. Although most applications of these types are cameras are primarily used for real-time identification of intrusion, theft, etc., the focus of this activity will be on the use of these cameras for forensic, evidentiary, and law enforcement tactical purposes.
Figure 14. License plate images taken using two different video cameras, one camera has sufficient resolution to provide legible characters.
Technical Strategy
-
Develop minimum performance requirements, in collaboration with manufacturers and users.
-
Develop test methods and standards for evaluating requirements.
-
Develop visible-light complex scene projector.
-
-
Develop reference test bed from which to accredit third-party labs.
Deliverables
-
List of performance metrics for in-car digital video system.
-
Test methods for performance metrics.
-
Identification of performance limits.
-
Prototype visible-light complex scene projector.
2.4 Infrared Surveillance and Security
These are the camera systems that will be replacing and have replaced analog cameras in security, surveillance, and emergency response applications. Applications that will be addressed include the cameras used by firefighters to find downed people inside of collapsing and burning structures and to assess conditions within burning structures. The firefighter cameras have unique requirements relative to conventional IR cameras in that they are exposed to high temperatures, a small temperature differential between target object and background, and the requirement for automatic control of certain functions. The unique requirements will also be addressed.
Figure 15. Infrared image of a person climbing over a fence at night.
Goals
To develop minimum performance requirements and associated test methods applicable to a broad range of IR video cameras used in surveillance, security, and by emergency responders.
Customer Needs
Presently there are no quantifiable and reproducible methods for evaluating technology and/or systems.
Technical Strategy
-
Develop minimum performance requirements, in collaboration with manufacturers and users.
-
Develop test methods and standards for evaluating requirements.
-
Develop IR complex scene projector.
-
-
Develop reference test bed from which to accredit third-party labs.
Deliverables
-
List of performance metrics for firefighter cameras.
-
Test methods for these performance metrics.
-
Identification of performance limits.
-
Prototype IR complex scene projector.
2.5 Microwave / Millimeter-Wave Anomaly Imagers
Microwave / millimeter-wave anomaly imaging systems operate in the microwave (typically from 1 GHz to 300 GHz) and millimeter wave (typically from 100 GHz to 1 THz) parts of the electromagnetic spectrum. These systems are being used and developed to image through thin nonmetallic barriers, such as clothing and packaging, for the purposed of identifying and locating concealed objects. This technology is relatively new for this application and, consequently, there is a lack of performance standards of any type.
Figure 16. Photograph (on left) and microwave holographic image (right) of a manikin with weapons concealed under clothing.
Figure 17. Photograph (left) and millimeter-wave image of a man with a concealed weapon.
Goals
To develop minimum performance requirements and associated test methods for these types of anomaly imagers.
Customer Needs
Presently there are no quantifiable and reproducible methods for evaluating technology and/or systems.
Technical Strategy
-
Develop minimum performance requirements, in collaboration with manufacturers and users.
-
Develop test methods and standards for evaluating requirements.
Deliverables
-
List of different applications for microwave / millimeter-wave anomaly imagers for which there are commercially-available systems.
-
List of applications selected for development of test methods.
-
List of performance metrics for selected applications.
2.6 Human Perception on Imaging Task Performance
The use of images by humans in security and law enforcement applications is a form of object recognition wherein the human observer must make an assessment of what is contained in the image and then formulate an appropriate response. Because task performance, such as identification, categorization, etc. is dependent on the observer, it becomes extremely difficult to develop objective test metrics. Consequently, the effect of human perception on task performance through the different imaging performance metrics must be quantified.
Goals
To develop methods for quantifying the effects of human perception on objective imaging performance metrics through assessment of the effects of human perception on task performance.
Customer Needs
The quality requirements presently used for still and motion imagery in law enforcement, security, and emergency response applications is not adjusted for the effects of human perception. This situation precludes making objective comparisons and selections between commercially available systems. Furthermore, it is difficult for manufacturers to improve product when the performance targets are ill defined.
Technical Strategy
-
In collaboration with the US Army's Night Vision Lab (NVL), develop methods for measuring and characterizing the effects of human perception on the quality of images acquired using firefighter IR cameras.
-
Determine appropriate algorithm for removing the effect of human perception on the previously identified performance metrics.
Deliverables
