MANUFACTURING


PERCEPTRON, INC.

Machines that See in 3-D

View PDF Document                Text Only Version

Can machines see? Not as well as a healthy set of human eyes, but well enough to do certain tasks. Of course, machines at present don’t see using biological processes; rather they can be given a form of sight through the use of digital cameras and other imaging devices used in conjunction with computer processing. In fact, prior to recent advances in machine vision technology, machines could only see in two dimensions (2-D), which meant seeing images as flat.

 COMPOSITE PERFORMANCE SCORE
                 (based on a four star rating)
                  * * *

Limitations of 2-D Vision

The flat images delivered by 2-D vision are adequate for some industrial purposes, and several firms offer standard 2-D vision systems for use in a variety of markets, such as inspection for quality control of printed circuit boards. Many automated tasks, however, require geometric spatial information that can only be provided by three-dimensional (3-D) vision capability. This requirement for improved vision with depth perception made the development of affordable 3-D vision technology an important goal in industrial automation. Affordable 3-D vision, for example, is critical to complete automation of inspection tasks and to achieving greatly improved performance of robot guidance tasks.

Technical Barriers to Affordable 3-D Vision

The recent advent of affordable 3-D imaging devices—hardware components of 3-D vision systems—brought 3-D machine vision a step closer to realization, but the lack of general-purpose 3-D vision software was recognized as a serious technical barrier impeding further progress. Under existing conditions, it was necessary to develop specific software for each application, as well as customized hardware to work with that particular software. The high costs of such customization have limited the rapid adoption and diffusion of 3-D machine vision technology, despite its large potential for making a wide variety of manufacturing processes more reliable, less expensive, and safer.

A Proposal to Overcome Technical Barriers

Perceptron, Inc., a small company in Farmington Hills, Michigan, submitted a proposal to ATP’s 1993 General Competition with the goal of advancing 3-D machine vision. Perceptron received $1.219 million in ATP funding.


3D machine vision of a person moving
The company provided $865 thousand of its own funds to mount the $2.084 million project. The focus was on developing tools for general-purpose vision software.

The high costs of customization have
limited the rapid adoption and diffusion
of 3-D machine vision technology.

More specially, the project aimed at developing standard image-processing algorithms, a test environment for testing, and compatible hardware standards that would provide the basis for the development of a range of affordable machine vision products. These software tools, the test environment, and the hardware standards could then be made available to researchers and companies to push the technology forward and develop applications for various industrial processes, such as automotive drive train assembly and quality control inspections.


PROJECT HIGHLIGHTS

Project:
To create generic image processing algorithms—building blocks for cost-effective 3-D vision software needed to make industrial machines “see” better—as well as supporting test environment and hardware specifications.


Duration: 1/1/1994 – 3/31/96
ATP Number: 93-01-0071

Funding (in thousands):
 ATP           $1,219      58%
Company       865      42%
Total         $2,084

Accomplishments:
Perceptron met or exceeded its goals for the project. Among its accomplishments, it:

- developed image processing techniques and algorithms for developing specialized vision software at lower cost;

- constructed a test environment to test these techniques for use in a range of different industrial applications;

- specified the hardware standards to use these tools effectively and inexpensively in an industrial setting;

- developed an interface module that allows data generated by the image processing techniques to be communicated between a range of imaging devices and computing platforms used in industrial machine vision systems;

- published 12 technical papers in international conference proceedings and international journals on image processing, pattern recognition, machine vision and industrial automation; and

- worked with Trident Systems to develop and install two robot guidance systems using the machine-vision software advances for use in a lumber mill.

Commercialization Status:
The chief focus of Perceptron’s efforts in software was robot guidance, which demands a high level of recognition capability, also crucial to other applications such as measurement and inspection. Commercial applications of the new vision software technology have been demonstrated in a lumber mill (where it has reduced timber waste) and to inspection of steel processing furnaces (to achieve safety goals without increasing downtime). In addition, it is feeding into a collaborative effort with Ford Motor Company on the development of robot guidance to automate the assembly of automobile powertrains as part of an ATP project.

Outlook:
Perceptron’s software advances in the ATP project have highlighted the need for improved imaging devices. At present, imaging devices must often be customized, at considerable cost, to meet the precision demands of specific applications. While the development of generic software techniques has extended the range of cost-effective applications of machine vision, the costs associated with the continued need for customization of imaging devices is a remaining barrier that impedes the rapid, widespread applications of 3-D machine vision.

Composite Performance Score:    * * *

Number of employees: at project start: 70; number of employees at project end: 300

Company:
Perceptron, Inc.
47827 Halyard Drive
Plymouth, MI 48170

Contact: Don Holtz
Phone: (734) 414-4842

Subcontractors: ERIM International, a nonprofit research institute; University of Michigan.


The Project Team

Perceptron subcontracted some of the project’s research to the School of Engineering at the University of Michigan at Dearborn (UMD). At the time of the proposal, UMD’s Machine Vision Laboratory was engaged in several projects that involved using image analysis methods to solve practical machine vision problems such as on-line inspection and parts recognition.

Perceptron subcontracted some of
the project’s research to the School of Engineering at the University of
Michigan at Dearborn (UMD).

Two months into the project, researchers from ERIM International, a nonprofit research institute, joined the project team — made up at the time of Perceptron and University of Michigan researchers. ERIM, headquartered in Ann Arbor Michigan, brought significant experience in image-processing and laser radar systems ERIM was instrumental in the development of a standard test environment for testing image-processing algorithms and in determining the hardware requirements for executing these algorithms.

Perceptron brought to the research effort a background in producing machine vision systems for a number of industrial applications. The company is primarily engaged in the manufacture and sale of imaging devices. Prior to this ATP project, Perceptron had developed a general-purpose industrial measurement and inspection system, called the P1000, which was based on laser


triangulation. Also prior to this ATP project, Perceptron had developed a scanning laser radar called the LASAR, which was the only commercially available scanning laser radar device at the time. This device provided a fundamental advance in the use of machine vision for robot guidance.

ERIM, headquartered in Ann Arbor Michigan, brought significant experience in image-processing and laser radar systems.

Project Goals and Accomplishments

Perceptron, UMD, and ERIM researchers had four technical objectives in software and hardware development, centered on 1) image preprocessing, 2) image feature extraction, 3) testing, and 4) hardware standardization. The aim was to create a standard set of algorithms for 3-D image-processing and object feature analysis, and to demonstrate the effectiveness of these algorithms in a test environment that could simulate the demands of a variety of different manufacturing applications. Such a test environment would allow many different automation vendors to develop general-purpose 3-D vision software products. This would in turn spur hardware production, resulting in widely available 3-D machine vision systems at affordable prices. The advances were expected to make it much more cost-effective to use automated systems in a variety of industrial processes, thus enhancing U.S. manufacturing competitiveness.

Perceptron reached all four of the project’s technical objectives. Most significantly, the objectives for software tools were exceeded, with over 200 image processing algorithms developed. According to a company representative, the ATP award accelerated progress towards accomplishing the goals by five years or more.1

Progress Toward Commercialization

The chief focus of Perceptron’s subsequent efforts to apply project advances has been in robot guidance. Robot guidance is considered a leading application of machine vision technology because it demands a high level of recognition capability. Technology which can meet the stringent demands of robot guidance systems generally can move into other applications, such as measurement and inspection.

Perceptron worked with Trident Systems, a systems integrator, to develop two related machine vision systems for use in Gulf States Paper Corporation’s new $40 million lumber mill in Moundville, Alabama. The first


system decides how to best cut logs crosswise into shorter lengths. The second system calculates the mix of plank sizes that will yield the least waste and most profit from each log.

Perceptron also is applying project results in developing machine vision systems to inspect the lining of furnaces used in steel processing. These systems would be able to detect faults in the lining remotely while the furnace is still in operation. Currently, furnaces must be shut down in order to examine the lining for faults pursuant to safety regulations, resulting in substantial downtime and loss of production.

Perceptron is building on the advances in software from the machine vision project to develop prototype automatic inspection devices for automobiles on assembly lines.

Perceptron worked with Trident Systems, a systems integrator, to develop two related machine vision systems for use in Gulf States Paper Corporation’s new $40 million lumber mill in Moundville, Alabama.

The company is collaborating with Ford Motor Company as part of a new ATP project to use robot guidance applications in auto industry robotics. 2

A Remaining Impediment to a Generic System

Despite the technical advances in software and Perceptron’s progress in commercializing results from the project, further improvement is needed to achieve a truly generic machine vision technology. Before the project, software capabilities lagged behind the capabilities of imaging devices then on the market, and the software was considered the major technical barrier. Although the project made progress with software, it made little further progress in the capabilities of imaging devices. By the end of the project, the capability of existing imaging devices was a remaining impediment.

The level of precision required for many of the actual applications, including those currently being developed by Perceptron, can only be achieved by customizing existing imaging devices at considerable cost. The inability of existing imaging devices to meet the demands of varied applications without undergoing customization, limits the ability of companies to take full commercial advantage of the progress made in developing generic software under the ATP project. Thus there is a need for further improve-ments in imaging devices, and improvements are now being pursued.
1 Interview with Don Holtz of Perceptron, November 28, 2000.

2 Flexible Robotic Assembly for Powertrain Application (FRAPA). ATP awarded funds to this project in October 1997.


Knowledge Sharing

Perceptron and UMD informed other researchers in the machine vision and related industries of their research findings through a number of published papers. The

By the end of the project, the
capability of existing imaging devices
was a remaining impediment.

research led to the publication of 12 technical papers in international conference proceedings and technical journals on image processing, pattern recognition, machine vision, and industrial inspection. Developments were also shared with these industries through trade shows and communication to the Robotics Industry Association, regional organizations such as the Industrial Technologies Institute in Michigan (now the Michigan Manufacturing Technology Center) and the Edison Industrial Systems Center in Ohio.

Gains in Industry Productivity and Safety

The development of general-purpose 3-D vision software promises to make 3-D machine vision systems more cost-effective for a number of industrial applications, which will allow cost savings and increased safety. For example, the use of 3-D vision systems for robot guidance allows for cost savings in dunnage, that is, the pallets, baskets, bins and other containers or platforms from which parts are taken for use in industrial assembly. With the use of 3-D vision systems, general-purpose dunnage can be used, allowing firms to move more quickly from development to production without the need to design specific dunnage for each new project. Successful application of robot guidance in the forest products industry promises

In steel processing, the successful
application of software advances to
remote inspection will allow safety goals
to be met without increasing downtime.

to decrease waste substantially, allowing more lumber to be produced from the same amount of timber. In steel processing, the successful application of software advances to remote inspection will allow safety goals to be met without increasing downtime.


 

This status report was written during 1999-2000 and published in December 2001.