FDA Examining Computer Diagnosis

When Paul A. Mongerson of Marathon, Fla., had severe abdominal pains in 1980, a battery of tests showed he had an elevated blood level of the enzyme lipase, which could indicate cancer of the pancreas. Though the diagnosis was not confirmed by other tests, his doctor recommended surgery.

"I'm an engineer by training, so I made up a matrix, charting my symptoms and test results, the possible diseases that could cause my symptoms, and what other symptoms would be present with those diseases," Mongerson recalls. "My conclusion was that I could not have cancer of the pancreas, and the fifth doctor I consulted, at Mt. Sinai in New York, agreed with me."

Eventually the pain disappeared, and he was found to have a condition known as pseudolipase, which results in abnormally high readings for the enzyme in tests. Mongerson said he thinks he might have bruised his pancreas while working around the house.

"I said at that time, 'what I did is just what a computer would do.' Medicine has gone about as far as it can without computers. There's a limit to how much the mind can retain, even with the degree of specialization that we've seen in medicine," Mongerson said. "The field of knowledge is so big the human mind is incapable of grasping it all--but a computer could help."

So Mongerson formed a foundation that provided financial assistance to physicians working to develop computer diagnosis systems. Today, medical diagnostic software puts entire medical libraries a mouse-click away. The Food and Drug Administration already regulates some diagnostic software, and as the number of programs expands, so does FDA's review.

Anthony Voytovich, M.D., chief of staff at John Dempsey Hospital in Connecticut, who has worked with computerized diagnosis programs for more than a decade, said it's important to keep their function in perspective.

"These programs are kind of like those Nordic Track machines. You can't put a machine like that in your bedroom and suddenly look like a million bucks; it's going to take a lot of sweat," Voytovich said. "When these programs were first put out there, a lot of people thought they would replace the physician, but they can't do that anymore than a high-speed drill will replace the dentist. They're a tool--a very effective tool that makes the physician's job much easier, but a tool nonetheless."

Diagnostic software is enjoying increasingly wider use each year. Computers are helping in the general practitioner's office, where they can identify even exotic diseases the physician has never seen. In the laboratory, they might help find a precancerous cell from a pap smear or a lump in breast tissue.

The programs are particularly good at prompting physicians with possibilities they might not otherwise envision. G. Octo Barnett, M.D., of Harvard Medical School's Laboratory of Computer Science, who developed the diagnostic software DXplain (2,200 diseases and 5,000 symptoms in its knowledge base), told of a physician in Texas seeing an adolescent boy with a skin rash and high fever. The computer suggested Rocky Mountain spotted fever, which the physician had not considered. He learned the boy had been hiking in Colorado recently, and subsequent tests confirmed the diagnosis.

Jerome P. Kassirer, M.D., editor of the New England Journal of Medicine, wrote in an editorial June 23, 1994, about the use of a computer program in the case of an 18-year-old man who had symptoms of a heart attack. The software also suggested cocaine abuse, which had not been considered, but which could have been a cause.

A study at the University of California San Diego Medical Center, focusing on emergency room patients experiencing chest pain, found computers accurately diagnosed 97 percent of heart attacks, compared with 78 percent for physicians.

Closer Look

FDA is now taking a closer look at medical diagnostic programs. Harvey Rudolph, Ph.D., acting deputy director of the Office of Science and Technology in the Center for Devices and Radiological Health, said FDA's policy has been that as long as the programs provided for "competent human intervention," they would not be actively regulated.

"In effect, the stand-alone diagnostic software programs used by physicians are considered decision support systems, and are exempt from regulation," Rudolph said. "The programs are based on well-known, well-established data, and the person using the software is expected to be competent to interpret the results."

FDA has always regulated computer software that modifies data entered by a user to control a medical device, or modifies data from a medical device to present it to a user. This includes, for instance, software that scans biopsied cells for signs of cancer.

But as the use of computers in diagnosis grows, the line between the different types of software blurs. Currently, FDA has applications on file for premarket approval of programs that would, for instance, help in reading diagnostic images such as mammograms.

Later this year, FDA plans to publish a Federal Register notice about its proposed policy for assessing and regulating various software, said Jurriaan Strobos, M.D., who is also an attorney and heads the policy research staff in FDA's Office of Policy.

Possible Criteria

Strobos suggested several criteria for determining whether software has the potential to adversely affect human health. Among them:

• Is the software intended for critical use, such as pointing out imminent danger in an intensive care unit, or for a less crucial purpose, such as to store diagnostic images such as CT scans?
• Is the user aware of the limitations of the program, and whether any of its recommendations depart from conventional medical practice?
• How specific is the software to a particular patient? Does it provide general information on a condition, or does it manipulate specific data from a specific patient to develop a specific treatment plan?
• How quickly do the software's recommendations need to be implemented? For instance, with an electrocardiogram program, will it recommend defibrillation at the appropriate time, or will it merely point out that a particular rhythm should be checked for abnormalities?

And, as Strobos pointed out, software is constantly being upgraded and improved, and customized. Such revisions need to be checked for their impact on the safety and effectiveness of the device.

"The [evaluation] process should not end when the software leaves the laboratory," Strobos said. "Postmarketing controls, as part of a life-cycle approach to software design, are important and should not be overlooked in developing a new system of agency review."

Rapid Recent Developments

Diagnostic programs have been around since the 1950s. The early programs were not used widely because they were unreliable, ran on bulky mainframes, and required a doctor to double as a data processing clerk. But in the last decade, engineers have put together more accurate and convenient programs. Hardware has shrunk to the size of a desktop, while computer memory has vastly expanded. Modems or network cables can easily link them together.

In most stand-alone diagnostic systems the physician enters the symptoms, test results, and medical history into the program, which then suggests a list of possible diagnoses. Some even provide a probability--in a person with severe, recurring headaches, for instance, the computer might say there is a 97 percent probability that the patient has sinusitis, but only a 6 percent probability of a brain tumor.

The programs can also help by highlighting some of the more likely diagnoses.

"If I see someone with a given set of symptoms, I can usually think of 20 or 30 things that it could be, off the top of my head," Voytovich said. A decision support system "helps me to narrow it down to a few possible diagnoses."

So far, the results of the programs have been mixed. A study by Eta S. Berner, Ph.D., of the University of Alabama, and colleagues from around the country, published in the New England Journal of Medicine, rated four of the most popular computer programs for medical diagnosis. The ratings were based on their ability to diagnose 105 cases chosen by a panel of experts, including some so complex they would not often be seen by the average physician.

The study found the programs correctly diagnosed the conditions between 50 and 75 percent of the time, depending on the complexity of the case, particularly whether more than a single disease was present.

Berner, a specialist in medical education, said that evaluation of the programs must also consider the doctor's rate of accuracy.

"If the physician is making an accurate diagnosis only 30 percent of the time, these programs would be better than a doctor," Berner said. "The consumer expects the doctor to be right 90 percent of the time, and that these programs will help."

Increased Complexity

As the knowledge base expands, the systems become more complex, requiring ever larger amounts of memory. To help overcome that problem, PKC Corp. in Vermont, has begun marketing "Problem Knowledge Couplers," diagnostic software sold in modules that target a specific area. Under current FDA policy the software is not regulated.

The PKC system prompts the physician with on-screen questions until it roots out the possible causes of the symptoms, then directs the physician to couplers, or software packages, that cover the appropriate disciplines.

So far, PKC has developed 50 couplers, covering areas as diverse as hypertension, lung cancer, obesity, and male erectile problems.

A "triage" coupler is used to help physicians determine whether a patient who calls with a problem needs to go to the emergency room, at an average cost of $230, or just make an appointment to come to the office.

Doctor Still Needed

The programs' limitations preclude them from replacing the physician.

Barnett, who developed DXplain, contended that only the physician can comprehend the "gestalt" of the disease, or diseases, producing the symptoms.

If a patient has only one disease, DXplain would stand a good chance of coming up with the correct diagnosis, Barnett said. "If they come in with two or three different diseases, though, it gets beyond the competence of any of the expert systems." In those cases, a physician's intuition, years of experience, and ability to make leaps of logic are the patient's best hope for unraveling the underlying diseases.

Nor can the computer consider the patient's personality, stresses, and other factors, or judge how those might interplay.

And, Barnett said, only physicians can quantify the disease, taking into account the myriad factors to determine its course in a particular patient. Is the pain radiating down the left arm from angina? Is it late- or early-stage disease? How severe is it? Will it get worse?

Still, Barnett and others agree, diagnostic software can prompt consideration of diseases a physician might never have seen or remembered described since medical school.

Computers are particularly effective for calculations needed for diagnosis. Susan Madden, a spokeswoman for Western Psychological Services in Los Angeles, said computers are commonly used in psychological assessments, to determine, for example, the potential of a patient for suicide, depression and anxiety, or substance abuse.

"The clinician using one of these programs doesn't have to calculate the scores or write an interpretive report on their own," Madden said. "It frees them up to do more clinical work."

Help for Technicians

More recently, computer diagnosis has been used to supplement the skills of medical technicians.

For instance, a technologist reviewing pap smears is expected to recognize a few abnormalities out of 100,000 cells--the equivalent of finding a couple of typographical errors in a large novel. A good technician might look at 90 slides a day. Estimates are that as many as many as 5 in 100 of the slides contain abnormalities that are missed, according to an article on the pap test in the Feb. 3, l989, issue of the Journal of the American Medical Association.

But Papnet, a computer program under a premarket review by FDA, developed by Neuromedical Systems Inc. in Suffern, N.Y., can screen one slide every five minutes, and work round the clock if necessary. The computer program identifies the 1,000 most suspect cells under magnification, then does a secondary screening where it stores magnified colored images of the 128 most suspect cells. A technologist can review them later and determine which require a pathologist's review.

A retrospective study by a team at Johns Hopkins University found Papnet identified suspect cells on slides from women whose undetected abnormalities had later gone on to develop lesions.

"It has the potential to detect abnormalities much earlier," said Laurie J. Mango, M.D., medical director and vice president at Neuromedical Services. "The most accurate pap smear will always be one that's read in two independent ways. In that sense, this will always be a complementary screen. But this is also a task that's particularly suited to computers because it's so repetitive."

The company is also investigating whether the program can be modified to screen cell samples from the respiratory tract and esophagus.

Other programs are being developed to digitize the information from mammograms and detect abnormalities. Stephen Feig, M.D., of Thomas Jefferson University Hospital in Philadelphia, said digital enhancement was found to increase a radiologist's detection of abnormalities on mammograms, from 85 to 92 percent in one study, and from 81 to 90 percent in another.

"It's not going to substitute for a radiologist, because no computer yet comes close to the human brain in terms of judgment, so determining whether something is benign or malignant is beyond its capabilities. But it might be of great value to a radiologist in that it would bring attention to potential abnormalities that he or she might not appreciate on a mammogram," Feig said. "Of course, it might also pick up abnormalities that are not there, but a radiologist could quickly rule them out."

Systems Integration

There is also an effort under way to integrate different computer systems. For instance, some electrocardiograms now come with a computer analysis attached to the printout, which notes some things the physician might want to consider based on the test results.

Jerome H. Grossman, M.D., chairman of the New England Medical Center in Boston, said the programs can also be used to reduce unnecessary treatment or tests.

Grossman, who is also chairman of the Federal Reserve Bank of Boston, said he has seen major changes in the way banks do business, to drive down costs by reducing duplication of services, or services too costly to justify.

"Health care is in the final stages of a similar transformation," Grossman said. "There is a real market for efficiency and value in health care. We have a marketplace that never existed before."

And, he predicted, there will be a continual sophistication of the software, integrating all the different information systems so that information on a patient's medical record is automatically considered by the diagnostic software, and the outcome of treatment, good or bad, is automatically fed back into the system to guide future decisions.

In the meantime, there are gradual refinements of existing software programs, and each year they become a little better at helping physicians figure out what is wrong with each patient who comes through the door.

Robert A. Hamilton is a writer in Franklin, Conn.