The Thermo robot with the SmartTRACK
sensor attached

Intelligent Systems Program

Measurement Science for Manufacturing Robotics and Automation Program

Annual FTEs: 10 NIST staff

5.5 guest researchers

15.5 total FTEs

Challenge

Manufacturers anticipate a future in which people will work side by side on the shop floor with versatile and adaptable robots and other automation systems. For this vision to come about, equipment will need to perceive the geometry, location, color, features, and movements of items and people in its vicinity and have an intrinsic ability to adjust for variations in parts, environment, and other conditions, rather than relying on fixed automation and costly infrastructure. The long-term challenge is to build a performance measurements science to characterize constituent components for intelligent manufacturing systems, define the target performance goals, and measure how well a component or overall system meets the goals.

Overview

NIST is working with manufacturers and developers of robotics and automation systems to define performance requirements that will help manufacturers build higher quality, higher-precision and more complex parts or assemblies, in a predictable, adaptive, safe, and efficient way. These requirements are used to establish metrics and quantitative, reproducible test methods that will spur innovation in the robotics and automation systems communities and pave the way for the needed advancements in capabilities, quality, and productivity.

NIST has long been a leader in establishing performance measures for mobile and intelligent robots, components, and algorithms. ISD expertise has drawn significant external funding from defense, transportation, and security agencies. This work has given us a strong foundation from which to apply the knowledge and tools gained to manufacturing robots and automation. In addition to defining performance metrics and test methods for overall systems, ISD efforts also address sensing, planning (including navigation), human interaction, mobility, and other essential component evaluation and characterization.

Key Accomplishments and Impacts:

• Organized a workshop on Dynamic Measurement and Control for Automated Manufacturing, through which industry defined their requirements for more autonomous and responsive robots.
• Developed a testbed for evaluating sensors, particularly a new class of 3D Flash LADARs (Laser Detection and Ranging). This enabled ISD to provide characterization data to industry where none previously existed.
• Applied ISD-developed techniques for evaluating the performance of 6D sensing and motion by robots, providing researchers and industry with quantitative data about their robots, sensors, and algorithms. Obtaining high-resolution data for robots in motion within a dynamic environment had previously not been possible. ISD involvement has led to the successful adoption of non-contacting safety sensors as vehicle obstacle detection devices (ANSI B56.5), increasing the safety zone around vehicles.
• ISD’s robotics expertise feeds into the development of world-class advanced control and positioning systems for nanoscale measurements and standards. A half-dozen micro-, meso-, and nanoscale positioners (some multi-axis) have been built and tested. These devices are being used to demonstrate 2D and 3D micro- and nano-assembly and to validate performance measures now under development. Such studies provide industry and researchers with knowledge about device characteristics crucial for their design and utilization.

Future Direction and Plans:

• Working with a newly-developed consortium of Automated Guided Vehicles (AGV) companies, ISD will develop requirements and standards for 3D imaging systems that can detect overhanging obstacles, which currently pose a major danger to these vehicles.
• Develop requirements for AGVs (which will become mobile robots) and robots arms to meet the flexibility, adaptability, and safety challenges articulated by users. ISD’s work will focus on the next generation of sensing, planning, and control to enable robots to operate safely in unconstrained and dynamic environments and in cooperation with humans. Detailed requirements and metrics will be used to generate standard test methods for quantifying and validating the performance of components and integrated systems.

  Awards and Recognition:

  Board Membership

  Staff	Board Membership
  Roger Bostelman	Board member on the ANSI/ITSDF B56.5 Sub-Committee for “Safety Standard for Guided Industrial Vehicles and Automated Functions of Manned Industrial Vehicles.”
  Raj Madhavan	IEEE Robotics and Automation Society (RAS) Conference Editorial Board (CEB)
  Elena Messina	Advisory Board, Texas A&M Engineering Extension Service (TEEX)

  Leadership

  Staff	Leadership
  James Albus	Editorial board for Autonomous Robots, Robotics and Autonomous Systems, Journal of Robotic Systems, Intelligent Automation and Soft Computing
  Nicholas Dagalakis	Co-organizer, 4th International Conference on Safety of Industrial Automation Systems SIAS-2005 . Organizer, Workshop on Metrology Needs for Micro Nano Systems Technologies. Technical Advisor, ANSI US TAG for ISO TC 184, SC 02, Robots and Robotic Devices
  Raj Madhavan	Founded the Washington DC section of the IEEE Robotics and Automation Chapter Chair of IEEE Robotics and Automation Society Washington Chapter
  Raj Madhavan, Steve Balakirsky	Originators and Chairs, IEEE Robotics and Automation Systems New Initiatives Competition “Advancing Robotic Research through an Open Source High-Fidelity Simulation Framework and Competition” (2007).
  Raj Madhavan, Elena Messina	Co-chairs (along with Angel del Pobil) of Workshop on Performance Evaluation and Benchmarking for Intelligent Robots and Systems at IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), San Diego, U.S.A., Nov. 2007.
  Craig Schlenoff and Steve Balakirsky	Chairs, Workshop on Knowledge Representation for Autonomous Systems at the 2005 ACM Conference on Information and Knowledge Management.
  Craig Schlenoff	Chair of the NIST Research Advisory Committee, 2007

  Excellence

  Staff	Excellence Recognized
  James Albus	Distinguished Lecture at University of Texas, Arlington on Toward A Computational Theory of Mind, 2007
  Roger Bostelman	Defense Manufacturing Conference (DMC) 2007 award for Best Presentation at the December 07 “Gee Whiz Technologies”
  Adam Jacoff, Raj Madhavan, Elena Messina	Guest Editors, Special Issue on Quantitative Performance Evaluation of Robotic and Intelligent Systems, Journal of Field Robotics, John Wiley & Sons, Inc., 2007, Volume 24, Issue 8-9 (August - September 2007).
  Elena Messina	Guest Editor, Integrated Computer-Aided Engineering, Volume 12, Issue 3, 2005, Special Issue on Performance Metrics for Intelligent Systems.

Projects

Measurement Science for Manufacturing
Robotics and Automation Program

Introduction

The Next Generation Robot (NGR), as defined by industry groups, is a machine that will incorporate inherent safety design and benign operating features in order to facilitate and promote lean manufacturing. ISD has worked with a broader definition of NGR, in which more advanced autonomy and intelligence is integrated into mobile vehicles and industrial arms. Work has been carried out in three complementary thrust areas. Performance Simulation for Mobile Manufacturing Robots tightly integrates real and virtual environments to develop advanced shop floor robotics concepts and creating performance requirements and test methods. Industrial Autonomous Vehicles focuses on transferring technologies from defense robotics to the materials handling and transport industries, and Next Generation Robot Safety concentrates on safety standards for industrial robots. Going forward, these projects will become more tightly connected to each other and to the division’s perception and sensing standards projects, which also support the Measurement Science program.

Measurement Science for Manufacturing
Robotics and Automation Program

Performance Simulation (PerfSim) Project
(status: initiated in 3QFY07; to be completed in FY09)

Challenge/Problem Addressed:

This project aims to facilitate safe operation of AGVs and robots in unstructured, dynamic environments through the use of simulation to perform initial validation of safety systems against standard test methods. The use of simulation is expected to also accelerate the introduction of more advanced robotics for manufacturing (both arms and AGVs) by allowing for greater flexibility and speed in experimentation and validation.

AGVs play an important role in today’s manufacturing processes. Companies use them on factory floors for jobs as diverse as intra-factory transport of goods between conveyors and assembly sections, movements of parts, and loading and unloading of truck trailers. Bishop Consulting’s report on AGV Industry Next-Generation Technology Priorities said that “In the eyes of the system vendors, the most prominent technology development area is in moving from today’s AGVs, which require highly structured environments and reference markers installed throughout the plant, to operating in less structured or unstructured environments. In fact, the site preparation required to site and install these reference markers are a significant portion of the system cost… In implementing AGVs for unstructured environments, another need is for sophisticated simulation capability… no capability currently exists to simulate AGVs for unstructured environments.”

Objectives:

• Build performance requirements and test methods for AGVs and robot arms by applying ISD’s world-class expertise in performance evaluation of mobile robots.
• Develop and refine test methods in a virtual factory simulation and translate them to easily realizable physical artifacts that remain coupled to the simulation environment. This coupling will permit more rapid and less costly review of potential test methods by constituents. Use of test environments that combine real and virtual elements will allow for performance quantification of systems that are in the vicinity of humans or in collaboration with them without posing any risk to human participants.

Planned Future Accomplishments:

• Demonstrate first test method for both simulated and real AGVs. The test method will measure how narrow a passageway an AGV is able to negotiate. This type of information about AGVs is not currently available and will eventually provide users and vendors with data to help them make buying and design decisions. (FY08Q3)
• Publish a document summarizing user-articulated requirements for AGVs. ISD has begun gathering input from users, particularly small and medium enterprises that may not currently be considering use of AGVs because of their costly infrastructural requirements. A requirements document will serve as a roadmap for developing test methods. (FY09Q3)

Customers and Collaborators

• We are in the process of reaching out to manufacturers to gather their requirements. Thus far, we have established ties with Dixon Valve and Patton Electronics
• IEEE
• Georgetown University
• George Mason University
• University of Maryland Eastern Shore

Measurement Science for Manufacturing
Robotics and Automation Program

Industrial Autonomous Vehicles Project
(Status: to be completed in FY2010)

Challenge/Problem Addressed:

The material handling industry needs greater flexibility, efficiency, and autonomy in its automated vehicles. This project aims to transfer into the Automated Guided Vehicle (AGV) and other types of vehicles advances gained in the development of measurement methods and new technology leveraged from other agencies, notably defense (Army Research Laboratory, DARPA) and transportation (Department of Transportation).

Objective(s):

• Transfer advanced defense and transportation technology and performance metrics to the automated guided vehicle industry
• Set standards to allow the use of advanced sensor and control technology

Accomplishments:

• ISD proposals to the ANSI/ITSDF B56.5 standards sub-committee resulted in changes to the standards to allow advanced, non-contact safety sensors as vehicle obstacle detection devices.
• Developed software for and demonstrated large facility navigation on a mobile robot equipped with only a color camera and an onboard computer to sense lines or patterns on the floor. This prototype demonstration builds end-user confidence in attaining greater efficiency and worker safety.
• Demonstrated, in collaboration with Transbotics, Inc., an AGV manufacturer and integrator, a prototype system to measure and verify palletized loads and truck volumes and orientations. This verified the feasibility of a generic solution for autonomous truck loading that is expected to also lead to the more difficult task of truck unloading.
• Funded a study on the AGV industry, which includes both manufacturers and users, that documented the current state of the art and prioritized the industry’s technology needs. This guides the requirements and standards development.
• In 2007, created a Consortium of three AGV companies – Danaher Motion, Egemin, and FMC – that will collaborate on advanced 3D image sensor systems to detect overhanging obstacles in the vehicle path currently requiring several 2D imaging sensors per vehicle. This addresses a major safety concern with AGVs.

Planned Future Accomplishments:

• 1st Q/2009: develop a consortium of AGV and forklift manufacturers to develop the concept for an advanced, light-duty, semi-autonomous forklift to carry relatively small loads through manufacturing and distribution facilities while posing no danger to people
• 2nd – 4th Q/2009: support design and development to build and then measure performance of a safe, light-duty, semi-autonomous forklift

Customers and Collaborators:

• Material Handling Industry of America (MHIA)
• Robotic Industries Association (RIA)
• ANSI
• Boeing
• Tranbotics
• Danaher Motion
• Egemin
• FMC

Measurement Science for Manufacturing
Robotics and Automation Program

Next Generation Robot Safety Project
(Status: to be completed in 2011)

Challenge/Problem Addressed:

Collaborative human-robot interaction holds promise for a growing number of applications, but robot technology has not changed significantly over the last ten years. Safety concerns arising in human-robot interaction are currently dealt with by restricting human access to significant portions of valuable manufacturing production floor space and investing significant resources in protective equipment. At least 100,000 industrial robots are in use in the U.S. and approximately 700,000 throughout the world, roughly 10% of which are replaced annually. Unit cost ranges from approximately $30K to $50K, but the cost of protective equipment is approaching the cost of the robotic units themselves. The Next Generation Robot (NGR), with its built-in safety technology, will promote the development of markets in those many classes of products requiring repeatable autonomous operation, handling of hazardous or heavy loads, etc. For example, this technology can reduce the cost of automobile manufacturing, microelectronic manufacturing, surgical operations, and rehabilitation and elderly care.

Objectives:

• Promote NGR inherent safety design and benign operating features in order to promote lean manufacturing.
• Develop metrology technology and sensors, which will facilitate the development of the NGR.
• Develop industrial robots safety standards.

Accomplishments:

• Produced and authenticated a USMS metrology need on “Next Generation Robots.” http://www.isd.mel.nist.gov/next_generation_robots/Manu_13_30Mar06_NextGen_Robots_Final.pdf. This documents the industry need for a wide audience and serves as a platform for establishing standards development and collaboration.
• Obtained informal oral permission to install experimental NGR devices in a GM plant for the purpose of collecting performance data. This will provide real-world data for developing concrete requirements for NGR safety standards.
• Identified the source for data on the injury tolerance of human beings and biometric devices appropriate for an industrial environment. This information is a necessary prerequisite in any plans for humans to work in proximity with robots.
• Contributed to the development and balloting of the industrial robot safety standard-ISO 10218-1. This standard specifies requirements and guidelines for the inherent safe design, protective measures, and information for use of industrial robots. It describes basic hazards associated with robots, and provides requirements to eliminate or adequately reduce the risks associated with these hazards.
• Hosted four International Standardization Organization meetings on the following subjects, all of which are aimed at having the standards evolve as industry and technology advances:
• Project Team 1 (PT1) meeting for the revision of ISO 10218 robot safety standard.
• Advisory Group 1 (AG1) meeting for the study of mobile service robots.
• Project Team 2 (PT2) meeting for the study of robots in personal care.
• ISO TC184/SC2 (Robots and Robotic Devices) annual plenary meeting.

Planned Future Accomplishments:

• Contribute key technical content and facilitate collaborations to develop the industrial robot safety stan¬dard-ISO 10218-2. This standard specifies requirements and guidelines for the installation and safe operation of industrial robots.
• Contribute key technical content and facilitate collaborations to develop a Technical Report on Guidelines for Implementing ANSI/RIA/ISO 10218-1-2007.
• Contribute key technical content and facilitate collaborations to develop a Technical Report on Guidelines for Implementing ANSI/RIA/ISO 10218-2.
• Develop and test prototypes of robot safety MEMS microsensor

Collaborators and customers:

• Robotic Industries Association (RIA).
• Liaison between the RIA office of standards development and the ANSI Panel on Nanotechnology Standards (ANSI/NSP).
  

Measurement Science for Manufacturing
Robotics and Automation Program

Perception for Advanced Intelligent Manufacturing Project
(Status: Exploratory project completed in FY07)

Challenge/Problem Addressed:

Investigate the roles and capabilities of novel 3D range sensors for use in manufacturing environments.

General and flexible 3D vision systems would have great payoffs for manufacturers as they could be quickly adapted to new processes, would require less investment in fixtures to orient parts, and could permit new applications in tasks where perception had previously been too difficult. Standards do not yet exist by which researchers can verify their approaches, vendors certify their products, and manufacturers evaluate sensors.

Objectives

• Investigate 3D vision perception systems for manufacturing.
• Determine current practice and identify where NIST could assist in metrology and standards for these systems.
• Conduct research to better understand a new class of sensors, 3D flash LADARs.
• Establish a testbed system for 3D sensors and characterize a set of sensors.

Accomplishments

• Attended workshops and trade shows and interacted with industry representatives and leadership from the Society of Manufacturing Engineers and the Automated Imaging Association (AIA). This, together with visits to factories, informed us on the state of the art and practice of industrial vision systems.
• Established a working group of NIST and GM researchers to investigate vision methods and algorithms for flexible manufacturing with specific application to fixtureless assembly and line tracking. The result is a white paper (in progress) on “Evaluation of 3D Imaging Systems for Dynamic Sensing Applications in Manufacturing.”
• Determined that no standards effort focuses on 3D sensors under general and dynamic conditions. Existing efforts emphasize 3D static metrology and not real-time applications.
• Worked with members of the Automated Guided Vehicle community on a CRADA to explore the use of flash LADARS for obstacle detection.
• Established a testbed to gather optimal data from the sensors, including work on calibration, data quality optimization, and super-resolution. The objective of the calibration work is to calibrate a heterogeneous set of sensors so as to simultaneously determine intrinsic[2] parameters and relative extrinsic[3] parameters for all the image sensors. This yielded an automatic correspondence-less[4] algorithm that simultaneously calibrates the sensors.
• Developed software that computes optimal parameter settings for a general class of 3D flash LADARS and applies those settings in real-time to reduce data variance. The software runs a general interface to flash LADARS from multiple vendors so all are treated in a common framework.
• Evaluated flash LADARs from several vendors to begin to develop standards for range data. Tested the sensors for range accuracy, precision, sensitivity to material reflectance, background illumination, and uniformity of active illumination. Determined that there are aspects of these sensors that require further testing, such as cross-talk between sensors, light scattering from one object that contaminates other objects, and dynamic behavior. Found that dynamic array range sensors will need additional test protocols beyond standards for static scanning LADARs.

Customers and Collaborators

• General Motors
• Egemin
• FMC
• Danaher
• Rensselaer Polytechnic Institute (RPI)

Measurement Science for Manufacturing
Robotics and Automation Program

Dynamic Metrology Project
(Status: Exploratory Project to be completed in FY2008)

Challenge/Problem Addressed:

To develop performance measures and standards applicable to six degree-of-freedom (6DOF) sensors for use in dynamic manufacturing environments where parts motion is unconstrained.

The ability to measure the positions and orientations of components as they move would result in considerable cost savings by replacing expensive fixed installations with more intelligent combinations of sensing and automation, and would help U.S. manufacturers compete with foreign firms that have recently made great investments in robotic technology. The automotive and airplane industries, as well as others, would gain substantial and immediate benefits, but the technology is fundamental and could be widely applied. A reference standard would also assist the academic community in establishing clear performance metrics for research systems and algorithms.

Objective(s):

• Develop methods for continuous measurement in manufacturing situations and evaluate and minimize the sources of error in the measurements.
• Investigate candidate technologies for reference measurements.
• Organize a Dynamic Metrology workshop.
• Establish the data collection infrastructure for simultaneous continuous measurements with two systems so the measurements can be compared and validated, and work on protocols and metrics for the comparison.
• Develop techniques to calibrate the reference and test systems, to synchronize measurements for comparisons, evaluate the raw sensor data that is used to compute 6DOF pose[5], and track the contribution and propagation of errors in subsystems.

Accomplishments:

• A report on the current state of the art in LADAR sensor technology was completed, updating the previous survey (NISTIR 7117) from 2004. The report will help determine the suitability of LADAR technology for 6DOF metrology.
• Took delivery of a laser tracker during fall 2007. The laser tracker is being used in experiments as a reference standard for 6DOF position and orientation.
• Used the laser tracker to validate sensor calibration algorithms developed at General Dynamics Robotic Systems (GDRS) as a first step towards standards transfer.
• Used the laser tracker with its 6DOF dynamic pose sensor to establish a system for time stamped simultaneous data collection. This involved working with the vendor to update and modify their system so that it can work with a hardware trigger and report at a high data rate, and the development of a trigger system for simultaneous measurements from the tracker and a sensor system under test. The data collection system was prototyped at NIST using a robot arm and an off-the-shelf vision system. This led to experiments at the Robot Vision Laboratory at Purdue University where the tracker evaluated the performance of a visual servoing system controlling a robot tracking a moving, swaying engine part.
• Held a workshop in October, 2007 with the title “Dynamic Measurement and Control for Automated Manufacturing.” Over 44 people attended the one and a half day event. A preliminary report on the workshop has been prepared.

Collaborators:

• General Motors
• Purdue University
• General Dynamics Robotic Systems
• NASA

Measurement Science for Manufacturing
Robotics and Automation Program

Advanced Control Systems and Positioning for Nanoscale Measurements and Standards Project
(Status: to be completed in 2011)

Challenge/Problem Addressed:

Metrology and fabrication of micro- and nanostructures are two key technologies for nanomanufacturing. A central challenge of micro- and nanomanufacturing is the development of methods to build complex three-dimensional (3-D) structures and devices on these scales that can readily interface with the macro scale world (scale-up). The metrology of micro- and nanostructures, such as nanowires, nanoparticles, proteins, cells, etc., is crucial for the advancement of micro and nano technology and manufacturing, which presents many opportunities for novel micro and nano electromechanical and biomedical systems.

Objectives:

• Develop world-class advanced control and positioning systems, and appropriate standards, for nanoscale measurements.
• Develop world-class advanced control and positioning systems, and appropriate standards, for nanoscale manipulation.
• Develop scale-up interfaces and standards to connect micro and nano manufacturing tools to the macro scale world.

Accomplishments

• Fabricated of numerous devices that advance the state of the art for control and positioning systems at the nanoscale, including:
• A two degrees of freedom (X-Y axes) planar dual parallel cantilever high precision meso scale micro positioner from Ti-6-4 (Ti, 6%Al, 4%V) material.
• Several 1D high precision MEMS nano positioners and thermal actuator calibration devices.
• Several X-Y precision MEMS nano positioners, with leaf-spring actuator coupling and single rod actuator coupling.
• Several micro assembly MEMS artifacts, successfully tested
• MEMS micro X-Y-Z nanopositioner, successfully tested
• A Probe Holding MEMS Micro Stage and Micro Coil.
• A 2 x 2 array of MEMS micro X-Y nanopositioning stages for nanoassembly operations.
• Completed numerous difficult experiments that advance the state of the art at the meso-, micro-, and nanoscales,
  including
• Tested several planar MEMS nano positioners inside a Scanning Electron Microscope (SEM) vacuum chamber.
• Quasi static and dynamic testing of nanopositioners
• Used MEMS Nano Positioning Devices to Perform Micro and Nano Material Testing
• Demonstrated Manipulation and Assembly of Microspheres.
• Collaborating RPI Researchers Demonstrated 3D Microassembly.

Planned Future Accomplishments:

• Develop embedded displacement metrology sensors for the MEL/ISD MEMS nano positioners.
• Develop embedded force metrology sensors for the MEL/ISD MEMS nano positioners.
• Advance and optimize the performance of the MEL/ISD MEMS nano positioners.
• Demonstrate the use of the MEL/ISD MEMS nano positioners for one or more nano metrology needs.
• Develop advanced metrology, control, and positioning systems, for nanoscale manipulation.

Collaborators and customers:

• RPI Center for Automation Technologies and Systems, Dept. of Electrical, Computer, & Systems Engineering
• George Washington University.
• APNanotech
• LightTime MEMS LADAR built based on NISTIR information
• Contributed to the RoboCup Nanomanufacturing Material Delivery and Removal competition demonstration in 2007