ECG-based Signal Analysis Technologies

1

General comment

The technology assessment for “ECG based signal analysis technologies” requested by CMS and performed by AHRQ and the Duke Evidence Based Practice Center is an extremely well written and thoroughly researched document.

Thank you.

1

General comment

I am not aware of any additional studies that should be included in the document. However, I did not perform a separate literature search.

Acknowledged.

1

General comment

I am impressed that one study of the PRIME ECG which compared the performance characteristics with the 12-lead ECG showed neither clinically nor statistically significant differences. There is no clinic evidence indicating that this technology impacts the diagnostic decision making or patient outcomes.

Acknowledged.

2

General comment

I was initially confused with the decision to not include stress testing without imaging as a reference standard for CAD diagnosis. The stress ECG has been a long term diagnostic test that has stood the test of time and has been shown to be extremely helpful in providing prognostically important information associated with patient outcomes and whether or not CAD is present. However, further reading of the document indicated that your gold standard or reference standard for CAD diagnosis would include coronary angiography and stress imaging as stress testing without imaging did not provide high enough diagnostic accuracy to be included as a reference standard. After further thinking through this approach, I agree with the final decision but would recommend that more focus be given to emphasize that even though there is a wealth of information indicating the usefulness of stress ECG, it still does not serve as an adequate reference standard to be used when comparing alternative technologies such as the ECG based signal analysis.

In the introduction to KQ1b (pg 18), we have clarified that we are focusing on reference standards for research purposes. We also modified the report to describe how stress testing in combination with longitudinal followup may be an adequate reference standard for CAD and CAD-related events (pg 22).

1

General comment

I do not have sufficient expertise to critique the statistical methods used in the technical assessment.

Acknowledged.

3

General comment

This whole report seems slightly off base to me. The goal of analyzing enhanced ECG technology to diagnose CAD seems farfetched. Why would we believe that such an analysis would provide a diagnosis of CAD in the absence of stress testing unless there was rest ischemia (AMI or Unstable Angina)? I don't think anyone obtains a resting ECG to screen for CAD. The screen for CAD, after exclusion of acute ischemia, is non-invasive stress testing, CT coronary angiography or cath in the highest of risk patients.

We appreciate the reviewer's comments. The report does not address screening for CAD. Rather it addresses patients with chest pain and low to intermediate probability of CAD. ECGs are often used in this clinical context to evaluate for acute myocardial ischemia. In addition, some device manufactures with devices evaluated in this report make claims about the accuracy for diagnosing CAD — making this an outcome of interest for our stakeholders.

3

General comment

An ECG would be obtained in the office setting primarily to 1) provide a baseline for future comparisons, 2) look for markers of hypertrophy, 3) examine for evidence of previous infarction, 4) exclude the unlikely event that there is currently active ischemia. An ECG would be obtained in the ED setting to exclude active ischemia and direct reperfusion therapy. They are related to, but very different from, screening for CAD. A more appropriate goal, therefore, would seem to be assessment of the effectiveness of enhanced ECG technology to detect acute ischemia (ACS).

We agree that there are multiple potential uses for an ECG in the office setting, including the evaluation of chest pain. Our report does not address screening for CAD and we have revised the report to clarify this issue (pg 10).

Our report focuses on CAD as the outcome (see response above). However, we acknowledge that myocardial ischemia is a relevant outcome and we discuss the importance of CAD, myocardial ischemia, and their relationship in the report's Introduction (pgs 7-8). In addition, when no eligible studies were available, we evaluated studies with higher risk samples that often reported data on myocardial infarction as an outcome.

3

General comment

Additionally I would change the focus from patients with “chest pain” to those with “symptoms of ACS.” On average, 1/3 of patients who present with AMI present without chest pain (Canto JG, et al Prevalence, clinical characteristics, and mortality among patients with myocardial infarction presenting without chest pain. JAMA 2000;283:3223-3229.) In some settings, this is as high as 50% of patients with AMI [Gupta M. et al. Presenting complaint among patients with myocardial infarction who present to an urban, public hospital emergency department. Ann Emerg Med. August 2002;40:180-186.] Although the authors mention a significant component of patients who receive and ECG in the acute setting.

Our mandate was to evaluate these devices for the detection of CAD in patients with chest pain (see key questions). We agree that another application of these devices might be to evaluate patients with symptoms of acute coronary syndrome (ACS) and that many patients with ACS present without typical chest pain. In the report's description of objectives, we have clarified that we are not focusing on ACS (pg 10).

3

General comment

I am concerned about interpretation of some of the literature. For example, they state, “However, a resting ECG has limited sensitivity (approximately 50%) for detecting CAD.” More correctly, this is the sensitivity for detecting ACD (50%). Sensitivity for detecting CAD is probably way lower! I reviewed the 3 references they cite to support this premise — none support this statement. #5 analyzes rates of previously unrecognized MI on ECG, #6 makes recommendations for competency in SCG reading, and #22 does not address this issue directly.

We appreciate the reviewer's comment. We've revised both the text and the references to reflect simply that ECG has low sensitivity for the detection of CAD (pgs 1, 10). The ECG is a complex diagnostic tool, with each segment or waveform having its own sensitivity and specificity for detection of ischemia, infarction, or CAD.

3

General comment

Additional concerns include one of their methodologic approaches. They report that there is statistical heterogeneity for the results of PRIME ECG and some of the results of 3DMP, but they still report a summary estimate. My understanding of meta-analysis methodology is that it is not statistically acceptable to calculate a summary estimate if there is heterogeneity — should only report the range of results.

We first evaluated the studies for conceptual homogeneity (sample, device, outcome measure) and judged sufficient design similarities to justify a combined estimate. We did find statistically significant heterogeneity, but used a random-effects model which does not assume homogeneity. Therefore, we think a summary estimate is justified and the random-effects model appropriate. However, we've added a comment about the unexplained heterogeneity in the discussion (pg 32).

3

General comment

Also, I do not completely agree with their interpretation of the results. Having calculated a —LR of 0.09 for 3DMP testing, they conclude that it would not be useful. My understanding is that a test with a —LR less than 0.1 or +LR greater than 10 is diagnostically useful. Therefore, if further testing supports this very low —LR for this test, it could, in fact, be useful diagnostically.

The reviewer is correct that the low LR- (0.09) would be useful in patients scheduled for angiography. However, by studying patients selected for angiography, the test performance is likely biased towards greater sensitivity and a more robust LR-. We think our caution that “..it is uncertain how the device would perform in outpatients with undifferentiated chest pain…” (pg 32) is warranted.

3

General comment

Finally 2 new articles on this topic have been published after this review was written. Not sure if possible, but they may want to include these.
Ornato JP, Menown IB, Peberdy MA, Kontos MC, Riddell JQ, Higgins GL 3rd, Maynard SJ, Adgey J. Body surface mapping vs 12-lead electrocardiography to detect ST-elevation myocardial infarction. Am J Emerg Med. 2009 Sep; 27(7):779-84.
Hoekstra JW, O'Neill BJ, Pride YB, Lefebvre C, Diercks DB, Peacock WF, Fermann GJ, Gibson CM, Pinto D, Giglio J, Chandra A, Cairns CB, Konstam MA, Massaro J, Krucoff M. Acute detection of ST-elevation myocardial infarction missed on standard 12-Lead ECG with a novel 80-lead real time digital body surface map: primary results from the multicenter OCCULT MI trial. Ann Emerg Med. 2009 Dec;54(6):779-788.el.

The article by Hoekstra has been added, as it is relevant to potential effects on patient outcomes (pg 33).

About Ornato et al.: This study evaluated patients presenting to an ED with suspected acute coronary syndrome and reported the outcome of ST elevation myocardial infarction. Thus it did not meet our full eligibility criteria and focuses on a very narrow outcome that was not a focus of this report.

2

Pg 12

“Diagnostic Testing and Risk Stratification of CAD”: it is not mentioned how patients are divided into low, intermediate and high risk groups. There are different clinical risk validated in different settings (CCU vs Emergency department vs general population), though Authors seem also to stratify patients according to their site of enrolment (whether outpatient or inpatient). It should be specified how the population of interest was stratified.

This discussion was intended to introduce the decisional approach to evaluating patients with chest pain, which bases the decision to test on pretest probabilities that are informed by the history and physical examination. We have added a note that validated risk prediction scores have been developed for a variety of clinical settings (pg 8).

1

Pg 13, Ln 9

The word “at” should be changed to “a” to read “As having a low to intermediate risk.”

Thank you. The typo has been corrected (pg 9).

2

Pg 17

“Evaluating Emerging ECG-based Technologies”. Only “one-shot” ECG-based technologies are reviewed. There is no mention about ECG telemetry/Holter technologies. Automatic algorithms for ST segment deviation have been developed by different Companies, along with linear and non linear analysis of heart rate variability measures. These indexes had been investigated in clinical setting with regard of CAD. Although an “over-time analysis” of ECG might be out of the scope of this review, this issue should be addressed and clearly stated.

We clarified the objectives to state that studies of heart rate variability were not evaluated in this report (pg 11). We did not exclude ECG telemetry/Holter technologies but did not identify any studies that used devices of this sort in the population of interest in the report.

2

Pg 38, Ln 21

Literature search and study inclusion criteria were appropriate, using May 2009 as deadline. It is worthwhile to mention a study on 80-Lead Surface Map recently published (Hoekstra JW, Acute Detection of ST-Elevation Myocardial Infarction Missed on Standard 12-Lead ECG With a Novel 80-Lead Real-Time Digital Body Surface Map: Primary Results From the Multicenter OCCULT MI Trial, Annals of Emerg Med Dec 2009) which included 1830 patients. This trial did not consider ambulatory patients rather than large ED-based population. If eligible for this review, it would count alone over one-half of patients considered for metanalysis.

Thank you for the citation. The study was conducted in a higher risk sample than the population of interest and did not report sensitivity or specificity. However, it did report some clinical outcomes, and we have now included these in the results for question 2d (pg 33).

2

Conclusion

Overall, the review is well conducted. Findings, limitations and conclusions are pertinent.

Thank you.

1

Conclusion

I agree with the final conclusion of the document that there is little available evidence that describes the utility of ECG based signal analysis technology as a diagnostic test used in patients with low to intermediate risk of coronary artery disease presenting in the outpatient setting with a history of chest pain. This is the group of patients in whim a screening test such as ECG based signal analysis would be most useful clinically and yet there is little data to support the utility of the test.

Acknowledged.