Clinical decision support systems for neonatal care

Tan K, Dear PRF, Newell SJ

 

Background - Methods - Results - Characteristics of Included Studies - References - Data Tables and Graphs


 

Dates

Date edited: 22/08/2007
Date of last substantive update: 24/01/2005
Date of last minor update: 09/07/2007
Date next stage expected 13/07/2009
Protocol first published: Issue 2, 2003
Review first published: Issue 2, 2005

Contact reviewer

Dr Kenneth Tan
Assistant Professor
Department of Paediatrics
McMaster University
Division of Neonatology, Room 4G37
1200 Main Street West
Hamilton
Ontario CANADA
L8N 3Z5
Telephone 1: 1 905 521 2100 extension: 76486
Facsimile: 1 905 521 5007
E-mail: ktan@mcmaster.ca

Contribution of reviewers

KT developed the protocol and search strategy, performed the searches, analysed the search results and wrote the manuscript.
SN was involved in the protocol development, searches and search result analysis.
PD edited the manuscript.
The decisions about the inclusion and exclusion of studies involved all three investigators.

Internal sources of support

None

External sources of support

SPorts Aiding Research for KidS (SPARKS), UK

What's new

This updates the review "Clinical decision support systems for neonatal care" published in The Cochrane Library, Issue 2, 2005 (Tan 2005).

The search was updated on July 2007. No new studies were identified for inclusion in this review update review. The conclusion of the review remains the same.

Four additional studies were added to the excluded studies list.

Dates

Date review re-formatted: / /
Date new studies sought but none found: 09/07/2007
Date new studies found but not yet included/excluded: / /
Date new studies found and included/excluded: / /
Date reviewers' conclusions section amended: / /
Date comment/criticism added: / /
Date response to comment/criticisms added: / /

Text of review

Synopsis


Clinical decision support systems (CDSS) are computer systems that bring together medical and patient information to help doctors make decisions about health care.

Support systems may be helpful for managing illness and the survival of newborns in the first 28 days of life. They may also influence the performance of doctors treating these newborn infants. The review authors searched the medical literature and contacted experts to find studies on CDSS used with newborns. They identified three randomised controlled studies that met the criteria for the review. Two of these three studies were on computer-aided drug prescribing and one was on computerized physiological monitoring of newborns. One of the studies on computer-aided prescribing showed that the CDSS used resulted in fewer drug dosage errors. The studies found no other benefits. The studies did not consider long-term outcomes in the newborns, just short-term effects. Also, with rapid changes in computer technology, current CDSS are more advanced than those used in the three studies. The Cochrane review authors conclude that there is not enough data to determine whether or not CDSS are beneficial for newborn care.

Abstract



Background


Clinical decision support systems (CDSS) are computer-based information systems used to integrate clinical and patient information to provide support for decision-making in patient care. They may be useful in aiding the diagnostic process, the generation of alerts and reminders, therapy critiquing/planning, information retrieval, and image recognition and interpretation. CDSS for use in adult patients have been evaluated using randomised control trials and their results analysed in systematic reviews. There is as yet no systematic review on CDSS use in neonatal medicine.

Objectives


To examine whether the use of clinical decision support systems has an effect on
1. the mortality and morbidity of newborn infants and
2. the performance of physicians treating them

Search strategy


The standard search method of the Cochrane Neonatal Review Group was used. Searches were made of the Cochrane Central Register of Controlled Trials (CENTRAL, The Cochrane Library, Issue 2, 2007), MEDLINE (from 1966 to July 2007), EMBASE (1980 - July 2007), CINAHL (1982 to July 2007) and AMED (1985 to July 2007).

Selection criteria


Randomised or quasi-randomised controlled trials which compared the effects of CDSS versus no CDSS in the care of newborn infants. Trials which compared CDSS against other CDSS were also considered. The eligible interventions were CDSS for computerised physician order entry, computerised physiological monitoring, diagnostic systems and prognostic systems.

Data collection & analysis


Studies were assessed for eligibility using a standard pro forma. Methodological quality was assessed independently by the different investigators.

Main results


Two studies fitting the selection criteria were found for computer aided prescribing and one study for computer aided physiological monitoring.

Computer-aided prescribing: one study (Cade 1997) examined the effects of computerised prescribing of parenteral nutrition ordering. No significant effects on short-term outcomes were found and longer term outcomes were not studied. The second study (Balaguer 2001) investigated the effects of a database program in aiding the calculation of neonatal drug dosages. It was found that the time taken for calculation was significantly reduced and there was a significant reduction in the number of calculation errors.

Computer-aided physiological monitoring: one eligible study (Cunningham 1998) was found which examined the effects of computerised cot side physiological trend monitoring and display. There were no significant effects on mortality, volume of colloid infused, frequency of blood gases sampling (samples per day) or severe intraventricular haemorrhage (Papile Grade IV). Published data did not permit us to analyse effects on long-term neurodevelopmental outcome.

Reviewers' conclusions


There are very limited data from randomised trials on which to assess the effects of clinical decision support systems in neonatal care. Further evaluation of CDSS using randomised controlled trials is warranted.

Background


The body of new medical information is growing at a seemingly exponential rate. As computers can process immense quantities of data with great speed, they are indispensable tools for helping to manage information (Johnson 1995). Clinical decision support systems (CDSS) are computer-based information systems used to integrate clinical and patient information to provide support for decision-making in patient care (NLM 2001). The medical tasks in which CDSS have been successfully used included diagnostic assistance, the generation of alerts and reminders, therapy critiquing/planning, information retrieval, and image recognition and interpretation (Coiera 1997).

The first computerised systems for medical decision support included a system for the diagnosis of congenital heart diseases that used Bayes theorem (Warner 1961) and MYCIN, a system for providing advice on antibiotic use in patients with hospital acquired sepsis (Wraith 1976). Three types of CDSS have been developed so far: rule based systems, probabilistic systems, and cognitive models (Delaney 1999). Computerised physician order entry (CPOE) systems have been promoted heavily to aid in drug prescribing, as these systems have been shown to lower the incidence of adverse drug events (Evans 1998). In adult patients on warfarin, computerised dosing has led to better oral anticoagulation (Fitzmaurice 1998). Other major areas for CDSS include aiding the diagnostic process; for example, in patients with possible surgical acute abdomens (Adams 1986) and in the intensive care environment for intelligent monitoring and the operation of infusion devices (Hanson 2001).

Computerised systems have been developed to assist the care of newborn infants since Perlstein 1976 first described their system for incubator temperature control. Indeed, CDSS have been created for many areas of neonatal care including management of the ventilated neonate (Carlo 1986; Snowden 1997) and in prescriptions, for example of parenteral nutrition solutions (Ball 1985). Systems have also been used for the prediction of length of inpatient stay (Zernikow 1999) as well as prognosis of respiratory distress syndrome (Hermansen 1987). These systems were generally reported to have beneficial effects on neonatal care.

Any information system, including CDSS, ought to be systematically evaluated before being introduced for patient care (Wyatt 2000). The use of randomised controlled trials for evaluation of CDSS has been questioned. It was thought that, in a fast changing environment, other approaches to evaluation might also be required (Mowatt 1997). Other pertinent issues are that CDSS may influence the behaviour of a physician, which then carries over when treating control patients (contamination of the control group) and it is sometimes impossible to blind patients and staff to the presence of a CDSS (Randolph 1999). The de Dombal system for aiding diagnosis of abdominal pain in adult patients was evaluated in a large multi-centre study in over 16,000 patients using a "before-after" study design (Adams 1986). It showed increase in accuracy of diagnosis, reduction in complications and mortality as well as significant cost savings.

A number of CDSS, have been successfully evaluated using the randomised controlled trial design (Wyatt 1990). A systematic review of these rigorously conducted studies showed that CDSS were effective in improving physician performance and patient outcome, but this review did not investigate systems developed for use in newborn infants (Hunt 1998). Although there are general reviews on the use of CDSS in pediatrics, like the effect of CPOE on prescribing (Kaushal 2001), there are no systematic reviews on the effects of CDSSs on care of newborn infants.

The aim of this systematic review is to review the evidence from controlled clinical trials on the effects of CDSS on neonatal care.

Objectives


To examine whether the use of clinical decision support systems (or different types of CDSS) has an effect on:
1. the mortality and morbidity of infants within the first 28 days of life within the neonatal unit, hospital wards and community and
2. the performance of physicians treating them

Criteria for considering studies for this review



Types of studies


Studies that have a randomised or quasi-randomised controlled trial design were included. Included studies could have used either patient, staff (medical and nursing) or hospital/unit as the unit of randomisation.

Types of participants


Studies of CDSS design for use in newborn infants, or for conditions arising in the first 28 days of life were included. Eligible studies could include CDSS for the care of neonates admitted to intensive care/special care units, not admitted to a neonatal unit, or managed in the community.

Types of interventions


Studies comparing the use of computerised systems versus no CDSS for clinical decision support in neonatal care or studies comparing different types of CDSS.

Studies describing CDSS for the following indications were specifically included:
1. computerised physician order entry (CPOE)
2. computerised physiological monitoring
3. diagnostic systems
4. prognostic systems.

Types of outcome measures


Primary outcome measures:
i. mortality within the first 28 days of life
ii. mortality within the first year of life
iii. effects (e.g. improvement, or otherwise, in diagnostic accuracy, time-saving, more efficient use of resources) on physician or nursing staff performance

Secondary outcome measures:
i. staff's satisfaction or compliance
ii. costs (cost of introducing the system, cost reduction in patient care)

Outcome measures specific to the type of CDSS:
1. CPOE systems - incidence of adverse drug events
2. computerised physiological monitoring - short-term physiological parameters (e.g. arterial blood gases, blood pressure) within the follow-up period
3. diagnostic systems - accuracy (level of agreement with physicians) and reliability
4. prognostic systems - accuracy (level of agreement with physicians) and reliability

Search strategy for identification of studies


1. Published manuscripts:
MEDLINE (from 1966 to July 2007), EMBASE (1980 to 2007), CINAHL (1982 to July 2007) and AMED (1985 to July 2007) were searched. A search strategy utilising the following Medical Subject Heading (MESH) terms was used: "computer assisted decision making", "artificial intelligence", "hospital information systems", "computer-aided diagnosis", "computer-aided therapy" and "algorithms". A search filter for randomised controlled trials was used. Age restriction used was "infants (0 - 23 months)" .

The details of the search strategy used for searching the databases are given below:

Search Strategy
1. randomised controlled trial.pt.
2. controlled clinical trial.pt.
3. randomised controlled trials.sh.
4. random allocation.sh.
5. double blind method.sh.
6. single blind method.sh.
7. or/1-6
8. animal.mp. not human.sh. [mp=title, abstract, registry number word, mesh subject heading]
9. 7 not 8
10. clinical trial.pt.
11. exp clinical trials/
12. (clin$ adj25 trial$).ti,ab.
13. ((singl$ or doubl$ or trebl$ or tripl$) adj25 blind$).mp. or mask$.ti,ab. [mp=title, abstract, registry number word, mesh subject heading]
14. placebos.sh.
15. placebo$.ti,ab.
16. random$.ti,ab.
17. research design.sh.
18. or/10-17
19. 18 not 8
20. 19 not 9
21. comparative study.sh.
22. exp evaluation studies/
23. follow up studies.sh.
24. prospective studies.sh.
25. (control$ or prospectiv$).mp. or volunteer.ti,ab. [mp=title, abstract, registry number word, mesh subject heading]
26. or/21-25
27. 26 not 8
28. 27 not (9 or 20)
29. 9 or 20 or 28
30. exp Decision Making, Computer-Assisted/
31. computer assisted decision making.mp. [mp=title, abstract, registry number word, mesh subject heading]
32. (computer$ adj3 decision$).mp. [mp=title, abstract, registry number word, mesh subject heading]
33. (clinic$ adj3 decision$ adj3 computer$).mp. [mp=title, abstract, registry number word, mesh subject heading]
34. 30 or 31 or 32 or 33
35. exp Artificial Intelligence/
36. artificial intelligence.mp. [mp=title, abstract, registry number word, mesh subject heading]
37. (artificial adj2 intelligence).mp. [mp=title, abstract, registry number word, mesh subject heading]
38. 35 or 36 or 37
39. exp Diagnosis, Computer-Assisted/
40. computer assisted diagnosis.mp. [mp=title, abstract, registry number word, mesh subject heading]
41. (computer$ adj3 diagnos$).mp. [mp=title, abstract, registry number word, mesh subject heading]
42. exp Therapy, Computer-Assisted/
43. computer assisted therapy.mp. [mp=title, abstract, registry number word, mesh subject heading]
44. (computer$ adj3 therap$).mp. [mp=title, abstract, registry number word, mesh subject heading]
45. computer assisted treatment.mp. [mp=title, abstract, registry number word, mesh subject heading]
46. (computer$ adj3 treat$).mp. [mp=title, abstract, registry number word, mesh subject heading]
47. 39 or 40 or 41
48. 42 or 43 or 44 or 45 or 46
49. exp Hospital Information Systems/
50. hospital information system$.mp. [mp=title, abstract, registry number word, mesh subject heading]
51. (hospital$ adj3 informat$ adj3 system$).mp. [mp=title, abstract, registry number word, mesh subject heading]
52. 49 or 50 or 51
53. exp ALGORITHMS/
54. algorithm$.mp. [mp=title, abstract, registry number word, mesh subject heading]
55. 53 or 54
56. 34 or 38 or 47 or 48 or 52 or 55
57. 29 and 56
58. limit 57 to (human and all infant )

The reference list of selected articles was reviewed. The authors were approached and asked about ongoing trials and unpublished studies. Hand searching of the relevant journals was also undertaken.

2. Published abstracts.
The proceedings of international meetings were comprehensively searched to identify relevant published abstracts from Proceedings of the Annual American Medical Informatics Symposium, MEDINFO and IEEE in Biomedical Engineering.

3. Database of the Cochrane Central Register of Controlled Trials (CENTRAL)
The Cochrane Central Register of Controlled Trials (CENTRAL, The Cochrane Library) Disk Issue 2, 2007 was searched with the above strategy.

4. Databases of the Neonatal Cochrane Review Group (CRG) of the Cochrane Collaboration.
The Neonatal CRG database was searched with the strategy as described above.

5. Databases of the Effective Practice and Organisation of Care (EPOC) Review Group of the Cochrane Collaboration. The EPOC database was searched with the strategy as described above.

6. Internet based resources
Internet based resources were looked at by consulting the web site (www.neonatology.org) and a search for the relevant web pages was done using search engines.

7. Selection process
The level of evidence was assessed after the Oxford Centre for Evidence-Base Medicine Criteria (Sackett 1986)
Clinical decision support systems were sorted into three categories:
i. CDSS not evaluated in the clinical environment, or prototypes
ii. CDSS evaluated in a non-clinical environment
iii. CDSS evaluated in the neonatal intensive care unit, including being evaluated using the randomised controlled trial design. Each investigator sorted the papers independently and any disagreement was resolved by discussion.

Methods of the review


The standard method for the Cochrane Collaboration as described in the Cochrane Collaboration handbook were used.

1. Eligible studies were independently selected by the investigators by applying inclusion and exclusion criteria. Any disagreement was resolved by discussion.

2. Data extraction and entry:
Data was extracted, assessed and coded separately for each study using a form that was designed specifically for this review. For continuous data, any standard error of the mean was replaced by the corresponding standard deviation.

3. Planned subgroup analyses:
Subgroup analyses according to the setting of newborn care were planned: neonatal intensive care unit, other in-hospital setting, or community setting.

4. Criteria for assessing the methodological quality of the studies:
Each trial was assessed for:
a) blinding of randomisation
b) blinding of intervention
c) whether there was complete follow-up
d) blinding of outcome measurement
and categorised them into "yes", "no" and "can't tell".

5. Statistical analyses:
Abstracted data were entered into RevMan 4.1 software and the studies were assessed for statistical heterogeneity. Meta-analyses was performed, if appropriate, using the fixed-effect model. The standard methods of the Neonatal Review Group were used to synthesise data using relative risk (RR), risk difference (RD) and number needed to treat (NNT) if there was a statistically significant reduction in RD and number needed to harm (NNH) if there was a statistically significant increase in the RD.

Description of studies


The number of possible eligible studies identified from the search was 29. From these, 26 were excluded after closer reading of the abstracts and text of the paper. The reasons for exclusion are listed in Table of Excluded Studies.

For computer aided prescribing, two studies, (Cade 1997; Balaguer 2001) fulfilled the selection criteria.

Cade 1997



Balaguer 2001
This paper was published in the Spanish language.



For physiological monitoring systems, only one study, Cunningham 1998, fulfilled the selection criteria set out above.

Cunningham 1998 was a randomised control study involving 600 infants receiving intensive care in one neonatal intensive care unit of whom two groups comprising 445 infants were eligible for inclusion in this review. The eligibility criterion of this study was any infant receiving intensive care.




There were no direct assessments of the effects of computerised monitoring on staff performance; however, this study incorporated a questionnaire-based study of staff and parental attitudes towards computerised monitoring

Methodological quality of included studies


In Cade 1997, the method of randomisation was by computer generated random numbers. The method of concealment was by sealed envelopes. There was no blinding of the intervention. Blinding of outcome measurements is uncertain. All the patients were accounted for in the follow-up.

In Balaguer 2001, the method of randomisation was by computer generated random numbers, but concealment was uncertain. There was no blinding of the intervention. All staff participants were accounted for in the follow-up.

There was insufficient information about the method of randomisation in Cunningham 1998. However, there was concealment of randomisation using sealed envelopes. Concealment of intervention was not possible because of the nature of the intervention. It was unclear if there was blinding of the infants' intervention group when measurements of short and medium term outcomes were made. All the patients were accounted for in the short and medium term measurements of outcomes.

Results


The search produced no eligible study that described the development and use of CDSS for diagnostic or prognostic purposes.

COMPUTER AIDED PRESCRIBING
There were two studies (Cade 1997; Balaguer 2001) that fitted the selection criteria.

Data on mortality, whether at 28 days or one year, was not described in the Cade 1997 study. This was not a outcome measure in Balaguer 2001.

There were no data on adverse drug events from Cade 1997. Balaguer 2001 showed significant reduction in errors in calculation of drug doses among resident pediatric staff, pediatricians and, to a lesser extent, in nurses. Errors classed as "serious" was eliminated with the use of the system. Adverse drug events were not applicable to Balaguer 2001 as the study used entirely simulated cases.

There were no data on neonatal staff performance from Cade 1997. However, Balaguer 2001 studied the length of time taken to calculate drug dosages, and this was significantly reduced by the use of the Neodosis spreadsheet program.

In Cade 1997 there were data regarding the actual volume of PN delivered to the infants and wastage of PN; and there was no significant difference in either of the outcomes between the intervention and control groups. Balaguer 2001 did not have cost as an outcome measure.

COMPUTER AIDED PHYSIOLOGICAL MONITORING (Comparison 01):

One study (Cunningham 1998) fitted the selection criteria.

There was no significant difference in the mortality of infants in the control or intervention groups. The relative risk for death was 0.74 (0.42 to 1.31) and the risk difference was -0.03 (-0.08, 0.03).

Volume of colloid administered can be a proxy measure of blood pressure and perfusion stability. There was no significant difference between the two groups in the amount of intravenous colloid administered; the mean difference of the volume of colloid administered (in ml/kg/day) was 0.10 with a 95% confidence -2.45 to 2.65.

With regard to severe (Papile Grade 4) intraventricular haemorrhage, there was no significant difference between the groups; relative risk was 0.73 (0.28 to 1.92) and the risk difference was -0.01 (-0.05, 0.02).

Two outcomes described in the study (that were not included in our analyses) were the length of ventilation and days on supplemental oxygen. The results were reported as median and ranges by Cunningham 1998. The investigators reported that there was no significant difference in the mean number of days on assisted ventilation or on supplemental oxygen.

It was not possible for us to analyse the outcomes from the published data as they were expressed in percentages of patients. However, the report stated that of the surviving infants, 71% that had computerised monitoring were normal while 67% of the control group were normal (95% confidence interval of the difference was -6% to 11%).

Test ordering behaviour as indicated by number of blood gas samples taken per baby per day was not significantly affected by computerised monitoring, MD -0.10 (-0.60 to 0.40).

Discussion


Computer decision support systems (CDSS) comprise a heterogeneous group of interventions. The definition of CDSS in MeSH states that CDSS incorporate information systems that use patient specific data and other clinical data to assist decision-making for an individual patient (NLM 2001). In our search, we found that even though most of the studies identified fall within this definition, there are some systems that are borderline - they have features both of CDSS and of computerised medical equipment where human decisions are not involved. That said, it was useful to begin the process of categorising computer systems for decision support as they will undoubtedly become ubiquitous in the future.

Besides being a heterogeneous group in terms of their underlying software or logic engine, it was also clear that the functions for which the CDSS had been designed are also heterogeneous. Again, it was useful to have a priori definitions on the areas of neonatal medicine where CDSS could be important.

The search strategy was comprehensive. One criticism may be the choice of databases that were searched. The search involved predominantly clinical databases, less so computer science and engineering databases. However, the question to be answered in the protocol was the effectiveness of CDSS vis-a-vis clinical care of newborn infants; hence the choice of databases searched was appropriate.

It is pertinent to consider the issue of outcome measures. While there is ongoing debate regarding the outcomes that should be measured, it is clear that the bottom line is improvement to either patient outcomes or physician performance. Indeed, improvement in these outcomes is precisely the raison d'etre for CDSS. Although we did not assess outcomes such as performance of the CDSS within the organisation, ultimately it is the clinical outcomes such as survival and staff performance that are most important.

Computer-aided prescribing
Cade 1997 was a short-term study investigating the effects on computer-aided PN ordering on short-term measures. It did not address longer-term outcome like survival, growth, and morbidity. There was also no measurement of the effect of CDSS on the performance of the doctors when using the software. In particular, there were no attempts to study the effect on error rates. Although useful for measuring short-term outcomes, this study suffered from small sample size and did not address long-term outcome issues.

Balaguer 2001 was a study that used simulated clinical cases rather than actual patients. The aim of this study was to investigate the performance of both medical and nursing staff in the neonatal setting in calculating drug dosages and drug dilutions, respectively. It did study the important issue of drug errors, particularly in dosaging errors. Although the cases were simulated, they did mirror actual clinical practice on any neonatal unit. Furthermore, in the original protocol for this review, the use of simulated or hypothetical cases was not a criterion for exclusion.

Computer-aided monitoring
Overall, Cunningham 1998 was a well-conducted large, single-centre randomised controlled trial. However, there were some methodological issues in this study. Approximately one quarter (23.6%) of the infants were randomised into two groups where computerised trend monitoring was used on alternate days (i.e. on odd and even days, respectively). Although the aim of this design was to test if just monitoring itself had any short-term effects (and it did not), the net result was that there were fewer infants available for analysis for the short and medium-term outcomes. It was not possible to include data on a few outcomes such as days ventilated, days on oxygen supplementation in the meta-analysis as they were not published in a suitable format. There were problems with the outcome measures at one and two years even though follow-up was available in 93% of the cases. Twenty-two percent of the infants were assessed by pediatricians who were not involved in the study, and who used different measures of neurodevelopment from the research group who had used the Denver II scale. These important data was not included in the meta-analysis.

It was not possible to analyse the effect on staff performance of the computerised monitoring as the study only set out to assess staff (and the infants' parents) attitudes and acceptance of the system through structured questionnaires. The frequency of blood gas samples was used as a proxy measure of impact on staff behaviour, i.e. in the ordering of blood gas analysis. A study (Griffith 1997) had shown that less experienced medical staff ordered more blood gas analysis.

The search strategy also revealed some reports of computerised monitoring of newborn infants in intensive care, but none of them was a randomised controlled study. These reports were mainly of systems in the prototype stage of development (Wolf 1996) or clinical experience with computerised monitoring (Dugdale 1983). It is often difficult to conduct a randomised control trial with enough power (by implication, usually requiring large numbers of subjects) with computerised systems because software and hardware upgrades and modifications occur constantly.

Finally, the conclusions from this review are probably limited in applicability. For example, Cunningham 1998 used technology that was developed some years ago. Information technology is constantly evolving. In the future, there will be more advanced and sophisticated systems (using genetic algorithms, fuzzy logic and neural networks) that would have greater capability. These systems would be expected to have different impact on the clinical care of the child. However, they should still be evaluated in randomised control studies.

Reviewers' conclusions



Implications for practice


There are insufficient data from randomised trials to determine the benefits or harms of CDSS in neonatal care.

Implications for research


Before CDSS are introduced the effects of the technology should be systematically studied using the randomised controlled study design (with either cluster randomisation or randomisation at the individual patient level).

Acknowledgements


The authors are grateful to Pat Spoor, Faculty Team Librarian for Healthcare Studies from the University of Leeds for her advice on literature search strategy.

Potential conflict of interest


None

Characteristics of included studies

StudyMethodsParticipantsInterventionsOutcomesNotesAllocation concealment
Balaguer 2001Blinding of randomisation: Can't tell.
Blinding of intervention: No
Completeness of followup: Yes. Masking of outcome measurement: Can't tell
27 health staff (9 nurses, 10 paediatric residents, 8 paediatricians) involved in the care of infants in 3 different NICUNeodosis, which was a database created within Excel versus manual calculation as control intervention (54 simulated cases, randomised crossover).Time taken for calculation, errors in calculation and major errors54 simulated cases were presented to the test participants. The medical staff were tested on drug dosage calculation, whereas the nursing staff were tested on drug dilution calculations.B
Cade 1997Blinding of randomisation: Yes (Sealed envelopes).
Blinding of intervention: No
Completeness of followup: Yes. Masking of outcome measurement: Can't tell
52 infants receiving parenteral nutrition on a regional neonatal unitComputer aided prescription of parenteral nutrition (27 infants randomised) versus control intervention of "standard bag" regimen where there was a fixed ratio of the volumes of amino acids and dextrose (25 infants randomised)Daily weights, Plasma electrolytes values, Actual volume of PN received, Wastage of PNOnly short term outcome measure. No long-term outcomes.A
Cunningham 1998Blinding of randomisation: Yes (sealed envelopes).
Blinding of intervention: No
Completeness of followup: Yes. Masking of outcome measurement: Can't tell
445 newborn infants that require intensive care in one regional neonatal unit. Inclusion criteria: all infants <=32 weeks gestation or if =>32 weeks - infants receiving ventilation or suffering from birth asphyxia.Computerised physiological trend monitoring displayed at the cot side. Computerised trend monitoring with continuous cot side display (n=230). Control group received monitoring but no display (n=215).Short-term: number of blood gases taken, volume of colloid support, daily cranial ultrasound.
Medium-term: death, time to death, time to other discharge, cranial ultrasound at discharge, days ventilated and days given supplemental oxygen.
Long-term: 1-year and 2-year follow-up - neurodevelopmental assessment, Denver II
This study randomised 600 infants into 4 groups: continuous display of trend data (n=230), no display of trend data (n=215), display on odd days only (n=76), display on even days only (n=79). Only the first 2 groups, comprising 445 infants in total, were eligible for inclusion in this review.A

Characteristics of excluded studies

StudyReason for exclusion
Ambalavanan 2001Part of the database from 810 ELBW infants (23 variables) was used to develop a regression model and a neural network model of ELBW mortality. The other part of the database was then used to validate the model. This was a study describing development of a system, not testing it in clinical setting.
Ambalavanan 2005This was a paper detailing the development of a statistical model for predicting outcomes in ELBW infants. This was developed from a (retrospective) data set from a group of NICUs. This was not a clinical trial.
Ambalavanan 2006This was a secondary analysis of data from a RCT on hypothermia for infants with HIE. This study aimed to identify variables that predict outcomes of HIE. The paper did not describe a system tested in a clinical trial for guiding decision-making.
Arroe 1991A computer program for providing advice on infants ventilated on a volume controlled ventilator. The ventilator treatment of 30 premature infants were retrospectively studied - a retrospective study. This was not a randomised control trial.
Baumgartner 2004This study was on the development of a computerised technique (data-mining) from dataset of a regional newborn screening program. This was based on data from records and was not a randomised control trial
Carlo 1986Study of a computerised algorithm for managing ventilated infants according to arterial blood gas values. Infants in the study were divided into 3 groups: 1) treated before the introduction of this system 2) managed by pediatric residents with aid of the system 3) treated after the introduction of the system, but without the system's output taken into consideration for management. Study design essentially of before-after type.
Claure 2001A study of a computerised algorithm for closed-loop control of the inspired oxygen concentration of mechanically ventilated neonates. The system was tested on 14 newborn infants. The intervention of computer control closed loop oxygen delivery was compared to manual control of oxygen concentration (by altering ventilator setting). Each infant was tested on one intervention for 2 hours and then switched to the other intervention for 2 hours also. The starting intervention was randomly assigned. Essentially a randomised control trial where each infants was its own control. However, the system described was really medical equipment and not a CDSS (which integrates clinical and patient information).
de Haan 1999A system developed for proposing dosage of heparin and for the detection of clotting disturbances in infants on ECMO. The system was tested on historical data from 44 neonates receiving ECMO. Not a randomised control trial.
Desch 1991A computer-assisted instructional program was developed for teaching medical students about care of the newborn infant. The students were randomly divided into 3 groups of 26 each; the first being the control (no additional task) group, second having specific educational material and the third group assigned to the computer aided instruction. The outcome measures were marks from pre-test and post-test multiple choice questions, and acceptability to user. This randomised control study investigated an educational intervention.
Dugdale 1988Closed-loop control system for controlling the inspired oxygen concentration of newborn infants with RDS in head box oxygen. Seven (n=7) preterm infants were studied. Study design was of an observational nature. The system is not strictly a CDSS and hence did not meet selection criteria.
Garibaldi 1999Expert system for interpreting umbilical cord blood gases to predict neonatal outcome. Not a randomised control study
Hatzakis 2002Report of a system that used fuzzy logic to propose changes to SIMV levels in weaning infants. Post-operative weaning data from 10 newborn infants were collected and then the proposed SIMV changes from the system were compared to actual changes. This is a validation study using archival data; not a randomised control trial.
Heerman 1997"The STABLE Assistant" is a rule-based CDSS for stabilising newborn infants being transported from community hospitals to acute care facilities. This study evaluated this system using retrospective data from 19 charts from infants requiring transport; not a control trial.
Jirapaet 2001A computer expert system, ES-MVN, was developed to support nursing decision-making in the management of ventilated newborn infants. This study did not meet selection criteria as it used case simulations in measuring the effect of ES-MVN on the self-perceived performance of neonatal nurses (n=16). Furthermore, it used a pretest-posttest (before-after) design, not a randomised control study.
Lehmann 2004Evaluation study of an online parenteral nutrition order system where utilization rate, error rate and user satisfaction were studied over 3 time periods; 1) immediately before, 2) immediately after and 3) 2 years after the introduction of the system; not a randomised control study.
Mahieu 2000This paper describes NOSEP, a computerised cot side scoring system, for predicting nosocomial infection from clinical and laboratory variables. Prospective data from 80 neonates were used to develop the system. The system was then validated on retrospective data from 39 other infants.
Mani 1997Description of MENTOR, a system that uses Bayesian probability techniques to predict mental retardation in newborn infants. The system was validated using generated cases and data from a national database. It was not tested in a randomised control study.
Michnikowski 1997A study which evaluated the performance of an expert system, AVES-N, against the historical performance of actual physicians. Data from 320 ventilated newborns were used. This was not a randomised control study.
Miksch 1996Descriptive study of VIE-VENT, as an open-loop system for monitoring and managing ventilated newborn infants; not a randomised control study.
Moorman 2006This study was on a predictive algorithm for detection of neonatal sepsis, based on heart rate variability. This was not a randomised control trial
Morozoff 1992A descriptive study of a closed-loop control system for the delivery of inspired oxygen to neonates targeting a set oxygen saturation. Not a randomised control trial.
Onofrey 2001This paper describes a program, RM-ROP, that calculated the risk of progression to threshold retinopathy of prematurity and risk of poor retina structural outcomes in preterm infants. The study was essentially an observational study ("non-interventional case-series") that used the actual retinal outcomes from preterm infants screened for ROP to validate the computer model.
Puangco 1997Before-after study of automation in parenteral nutrition ordering and delivery. Total of 81 infants were studied. Not a randomised control trial.
Shimomura 1994A study of AFES (Apgar expert system) a program using fuzzy logic, to assign Apgar scores for newborn infants. Three versions of AFES was devised, from the knowledge of inexperienced obstetrician, experienced obstetrician and experienced neonatologist. This study described 267 infants who were assessed by an obstetrician 1 minute after birth, and given their Apgar scores. The performance of the AFES in giving the Apgar scores was compared to that of the obstetrician. The performance of the AFES derived from the knowledge of neonatologists performed significantly better than the other AFES. This was not a randomised control study.
Snowden 1997A study of ESNIC, a rule-based expert system for neonatal intermittent positive pressure ventilation. An observational study that involved 63 ventilated newborns investigating how the advice given by the system was followed.
Sun 1997This paper described a fuzzy logic system for an open loop control system for the delivery of inspired oxygen concentration in mechanically ventilated infants. The system recommended changes to be made to FiO2 in order for the SpO2 to stay within a target range. Nursing staff would enact the changes. 16 infants were studied with a 2-hour period of no computer assistance, then 2-hour period where the system was on followed by a final 2-hour period of unassisted FiO2 control. The main outcome measure was time spent by the infants within the target SpO2. This was not a randomised control trial.
Wallace 2003A computer program was developed that could analyse digital images from a video-indirect ophthalmoscope to determined if plus disease in retinopathy of prematurity was present. Digital retinal images from 20 preterm infants were obtained from a database and the performance of the program was compared against that of 2 human examiners. This was not a randomised control trial.
Zernikow 1998Retrospective data from 890 preterm infants were used to generate both a logistic regression model and an artificial neural network to predict the risk of neonatal mortality. The infants were born over a 6 year period. Just under half these infants born around the first three years of this time period were assigned into the training group while the other half born in the latter 3 years formed the validation set. This study was a description of development and validation of a system that was not tested in a randomised control study.
Zernikow 1998bRetrospective data from 890 preterm infants were used to generate both a multiple logistic regression model and an artificial neural network to predict the occurrence of severe intraventricular haemorrhage. Half of these infants were randomly assigned into a training group where their data were used to generate the models and the other half were assigned into a validation where their data were used to validate the models. This was not a randomised control study investigating the performance of this system.
Zernikow 1999Retrospective data from 2144 preterm neonates were used to generate a multiple regression model and an artificial neural network to predict the length of stay. About 75% of these infants data were randomly assigned to provide data to generate the models; the remaining 25% of these infants fell into the validation group.

References to studies

References to included studies

Balaguer 2001 {published data only}

Balaguer Santamaria JA, Fernandez Ballart JD, Escribano Subias J. Utilidad de una aplicacion informatica para disminuir errores de tratamiento en neonatologia [Usefulness of a software package to reduce medication errors in neonatal care]. Anales Espanoles de Pediatria 2001;55:541-5.

Cade 1997 {published data only}

Cade A, Thorp H, Puntis JWL. Does the computer improve the nutritional support of the newborn? Clinical Nutrition 1997;16:19-23.

Cunningham 1998 {published data only}

Cunningham S, Deere S, Symon A, Elton RA, McIntosh N. A randomized, controlled trial of computerized physiologic trend monitoring in an intensive care unit. Critical Care Medicine 1998;26:2053-60.

References to excluded studies

Ambalavanan 2001 {published data only}

Ambalavanan N, Carlo WA. Comparison of the prediction of extremely low birth weight neonatal mortality by regression analysis and by neural networks. Early Human Development 2001;65:123-37.

Ambalavanan 2005 {published data only}

Ambalavanan N, Carlo WA, Bobashev G, Mathias E, Liu B, Poole K et al. Prediction of death for extremely low birth weight neonates. Pediatrics 2005;116:1367-73.

Ambalavanan 2006 {published data only}

Ambalavanan N, Carlo WA, Shankaran S, Bann CM, Emrich SL, Higgins RD et al. Predicting outcomes of neonates diagnosed with hypoxemic-ischemic encephalopathy. Pediatrics 2006;118:2084-93.

Arroe 1991 {published data only}

Arroe M. A computerized aid in ventilating neonates. Computers in Biology and Medicine 1991;21:15-21.

Baumgartner 2004 {published data only}

Baumgartner C, Bohm C, Baumgartner D, Marini G, Weinberger K, Olgemoller B et al. Supervised machine learning techniques for the classification of metabolic disorders in newborns. Bioinformatics 2004;20:2985-96.

Carlo 1986 {published data only}

Carlo WA, Pacifico L, Chatburn RL, Fanaroff AA. Efficacy of computer-assisted management of respiratory failure in neonates. Pediatrics 1986;78:139-43.

Claure 2001 {published data only}

Claure N, Gerhardt T, Everett R, Musante G, Herrera C, Bancalari E. Closed-loop controlled inspired oxygen concentration for mechanically ventilated very low birth weight infants with frequent episodes of hypoxemia. Pediatrics 2001;107:1120-4.

de Haan 1999 {published data only}

de Haan AF, van der Velde J, Geven WB, Festen C, Verbeek AL. A multistate Kalman filter for neonatal clotting time prediction and early detection of coagulation disturbances. Methods of Information in Medicine 1999;38:113-8.

Desch 1991 {published data only}

Desch LW, Esquivel MT, Anderson SK. Comparison of a computer tutorial with other methods for teaching well-newborn care. American Journal of Diseases of Children 1991;145:1255-8.

Dugdale 1988 {published data only}

Dugdale RE, Cameron RG, Lealman GT. Closed-loop control of the partial pressure of arterial oxygen in neonates. Clinical Physics and Physiological Measurement 1988;9:291-305.

Garibaldi 1999 {published data only}

Garibaldi JM, Westgate JA, Ifeachor EC. The evaluation of an expert system for the analysis of umbilical cord blood. Artificial Intelligence in Medicine 1999;17:109-30.

Hatzakis 2002 {published data only}

Hatzakis GE, Davis GM. Fuzzy logic controller for weaning neonates from mechanical ventilation. In: Proceedings / AMIA Annual Symposium. 2002:315-9.

Heerman 1997 {published data only}

Heermann LK, Thompson CB. Prototype expert system to assist with the stabilization of neonates prior to transport. In: Proceedings/AMIA Annual Fall Symposium. 1997:213-7.

Jirapaet 2001 {published data only}

Jirapaet V. A computer expert system prototype for mechanically ventilated neonates development and impact on clinical judgment and information access capability of nurses. Computers in Nursing 2001;19:194-203.

Lehmann 2004 {published data only}

Lehmann CU, Conner KG, Cox JM. Preventing provider errors: online total parenteral nutrition calculator. Pediatrics 2004;113:748-53.

Mahieu 2000 {published data only}

Mahieu LM, De Muynck AO, De Dooy JJ, Laroche SM, Van Acker KJ. Prediction of nosocomial sepsis in neonates by means of a computer-weighted bedside scoring system (NOSEP score). Critical Care Medicine 2000;28:2026-33.

Mani 1997 {published data only}

Mani S, McDermott S, Valtorta M. MENTOR: a Bayesian Model for prediction of mental retardation in newborns. Research in Developmental Disabilities 1997;18:303-18.

Michnikowski 1997 {published data only}

Michnikowski M, Rudowski R, Siugocki P, Grabowski J, Rondio Z, Lindstrom DP. Evaluation of the expert system for respiratory therapy of newborns on archival data. International Journal of Artificial Organs 1997;20:678-80.

Miksch 1996 {published data only}

Miksch S, Horn W, Popow C, Paky F. Utilizing temporal data abstraction for data validation and therapy planning for artificially ventilated newborn infants. Artificial Intelligence in Medicine 1996;8:543-76.

Moorman 2006 {published data only}

Moorman JR, Lake DE, Griffin MP. Heart rate characteristics monitoring for neonatal sepsis. IEEE Transactions on Biomedical Engineering 2006;53:126-32.

Morozoff 1992 {published data only}

Morozoff PE, Evans RW. Closed-loop control of SaO2 in the neonate. Biomedical Instrumentation & Technology 1992;26:117-23.

Onofrey 2001 {published data only}

Onofrey CB, Feuer WJ, Flynn JT. The outcome of retinopathy of prematurity: screening for retinopathy of prematurity using an outcome predictive program. Ophthalmology 2001;108:27-34.

Puangco 1997 {published data only}

Puangco MA, Nguyen HL, Sheridan MJ. Computerized PN ordering optimizes timely nutrition therapy in a neonatal intensive care unit. Journal of the American Dietetic Association 1997;97:258-61.

Shimomura 1994 {published data only}

Shimomura K, Shono H, Kohara M, Uchiyama A, Ito Y, Sugimori H. Neonatal assessment using the Apgar fuzzy expert system. Computers in Biology and Medicine 1994;24:171-178.

Snowden 1997 {published data only}

Snowden S, Brownlee KG, Dear PR. An expert system to assist neonatal intensive care. Journal of Medical Engineering and Technology 1997;21:67-73.

Sun 1997 {published data only}

Sun Y, Kohane IS, Stark AR. Computer-assisted adjustment of inspired oxygen concentration improves control of oxygen saturation in newborn infants requiring mechanical ventilation. Journal of Pediatrics 1997;131:754-6.

Wallace 2003 {published data only}

Wallace DK, Jomier J, Aylward SR, Landers MB 3rd. Computer-automated quantification of plus disease in retinopathy of prematurity. Journal of AAPOS: American Association for Pediatric Ophthalmology and Strabismus 2003;7:126-30.

Zernikow 1998 {published data only}

Zernikow B, Holtmannspoetter K, Michel E, Pielemeier W, Hornschuh F, Westermann A, Hennecke KH. Artificial neural network for risk assessment in preterm neonates. Archives of Disease in Childhood Fetal & Neonatal Edition 1998;79:F129-34.

Zernikow 1998b {published data only}

Zernikow B, Holtmannspoetter K, Michel E, Theilhaber M, Pielemeier W, Hennecke KH. Artificial neural network for predicting intracranial haemorrhage in preterm neonates. Acta Paediatrica 1998;87:969-75.

Zernikow 1999 {published data only}

Zernikow B, Holtmannspotter K, Michel E, Hornschuh F, Groote K, Hennecke KH. Predicting length-of-stay in preterm neonates. European Journal of Pediatrics 1999;158:59-62.

* indicates the primary reference for the study

Other references

Additional references

Adams 1986

Adams ID, Chan M, Clifford PC, Cooke WM, Dallos V, de Dombal FT, Edwards MH, Hancock DM, Hewett DJ, McIntyre N, et al. Computer aided diagnosis of acute abdominal pain: a multicentre study. British Medical Journal (Clinical research ed.) 1986;293:800-4.

Ball 1985

Ball PA, Candy DC, Puntis JW, McNeish AS. Portable bedside microcomputer system for management of parenteral nutrition in all age groups. Archives of Disease in Childhood 1985;60:435-9.

Coiera 1997

Coiera E. Artificial Intelligence, Chap 19 in: Guide to Medical Informatics, The Internet and Telemedicine. In: Guide to Medical Informatics, The Internet and Telemedicine. London: Chapman & Hall Medical, 1997:200-219.

Delaney 1999

Delaney BC, Fitzmaurice DA, Riaz A, Hobbs FDR. Can computerised decision support systems deliver improved quality in primary care? British Medical Journal 1999;319:1281.

Dugdale 1983

Dugdale RE. Lealman GT. A cotside microcomputer monitoring system for use in the neonatal intensive care unit. Clinical Physics & Physiological Measurement 1983;4:373-80.

Evans 1998

Evans RS, Pestotnik SL, Classen DC, Clemmer TP, Weaver LK, Orme JF, Lloyd JF, Burke JP. A computer-assisted management program for antibiotics and other antiinfective agents. New England Journal of Medicine 1998;338:232-8.

Fitzmaurice 1998

Fitzmaurice DA, Hobbs FD, Delaney BC, Wilson S, McManus R. Review of computerised decision support systems for oral anticoagulation management. British Journal of Haematology 1998;102:907-9.

Griffith 1997

Griffith CH3rd, Wilson JF, Desai NS, Rich EC. Does pediatric housestaff experience influence tests ordered for infants in the neonatal intensive care unit? Critical Care Medicine 1997;25:704-9.

Hanson 2001

Hanson CW3rd, Marshall BE. Artificial intelligence applications in the intensive care unit. Critical Care Medicine 2001;29:427-35.

Hermansen 1987

Hermansen MC, Hasan S. An evaluation of a computer program to predict the outcome of hyaline membrane disease. American Journal of Perinatology 1987;4:249-52.

Hunt 1998

Hunt D L, Haynes RB, Hanna SE, Smith K. Effects of computer-based clinical decision support systems on physician performance and patient outcomes: a systematic review. Journal of the American Medical Association 1998;280:1339-46.

Johnson 1995

Johnson KB, Feldman MJ. Medical informatics and pediatrics. Decision-support systems. Archives of Pediatric and Adolescent Medicine 1995;149:1371-80.

Kaushal 2001

Kaushal R, Barker KN, Bates W. How can information technology improve patient safety and reduce medication errors in children's health care? Archives of Pediatric and Adolescent Medicine 2001;155:1002-7.

Mowatt 1997

Mowatt G, Bower DJ, Brebner JA, Cairns JA, Grant ASM, McKee L. When and how to assess fast changing technologies: a comparative study of medical applications of four generic technologies. Health Technology Assessment 1997;1:1-149.

NLM 2001

National Library of Medicine. MeSH, 2001 Supplement to Index MedicusĀ®.

Perlstein 1976

Perlstein PH, Edwards NK, Atherton HD, Sutherland JM. Computer-assisted newborn intensive care. Pediatrics 1976;57:494-501.

Randolph 1999

Randolph AG, Haynes RB, Wyatt JC, Cook DJ, Guyatt, GH. Users' Guides to the Medical Literature: XVIII. How to use an article evaluating the clinical impact of a computer-based clinical decision support system. Journal of the American Medical Association 1999;282:67-74.

Sackett 1986

Sackett DL. Rules of evidence and clinical recommendations on use of antithrombotic agents. Chest 1986;89:2S-3S.

Warner 1961

Warner HR, Toronto AF, Veasey LG, Stephenson R. A mathematical approach to medical diagnosis: application to congenital heart disease. Journal of the American Medical Association 1961;177:75-81.

Wolf 1996

Wolf M, Keel M, von Siebenthal K, Bucher H, Geering K, Lehareinger Y, Niederer P. Improved monitoring of preterm infants by Fuzzy Logic. Technology and Health Care 1996;4:193-201.

Wraith 1976

Wraith SM, Aikins JS, Buchanan BG, Clancey WJ, Davis R, Fagan LM et al. Computerized consultation system for selection of antimicrobial therapy. American Journal of Hospital Pharmacy 1976;33:1304-8.

Wyatt 1990

Wyatt J, Spiegelhalter D. Evaluating medical expert systems: what to test and how? Medical Informatics 1990;15:205-17.

Wyatt 2000

Wyatt JC. Decision support systems. Journal of the Royal Society of Medicine 2000;93:629-33.

Other published versions of this review

Tan 2005

Tan K, Dear PRF, Hewell SJ. Clinical decision support systems for neonatal care. Cochrane Database of Systematic Reviews 2005, Issue 2.

Comparisons and data

Comparison or outcome
Studies
Participants
Statistical method
Effect size
01 Computer aided physiological monitoring versus control
01 Mortality
1
445
RR (fixed), 95% CI
0.74 [0.42, 1.31]
02 Volume of intravenous colloid used (mls/kg/day)
1
445
WMD (fixed), 95% CI
0.10 [-2.45, 2.65]
03 Grade 4 IVH (Papile)
1
445
RR (fixed), 95% CI
0.73 [0.28, 1.92]
04 Number of blood gas samples taken /baby/day
1
445
WMD (fixed), 95% CI
-0.10 [-0.60, 0.40]

 

01 Computer aided physiological monitoring versus control

01.01 Mortality

01.02 Volume of intravenous colloid used (mls/kg/day)

01.03 Grade 4 IVH (Papile)

01.04 Number of blood gas samples taken /baby/day


Contact details for co-reviewers

Dr Peter RF Dear
Consultant Neonatologist
Department of Paediatrics
St. James's University Hospital
UK
E-mail: prfd@doctors.org.uk

Dr Simon J Newell
Consultant Neonatologist and Paediatric Gastroenterologist
Department of Paediatrics
St. James's University Hospital
Beckett Street
Leeds
West Yorkshire UK
LS9 7TF
Telephone 1: 01132066959
E-mail: simon.newell@leedsth.nhs.uk

 
This review is published as a Cochrane review in The Cochrane Library, Issue 4, 2007 (see http://www.thecochranelibrary.com for information). Cochrane reviews are regularly updated as new evidence emerges and in response to feedback. The Cochrane Library should be consulted for the most recent version of the review.