[This Transcript is Unedited]

NATIONAL COMMITTEE ON VITAL AND HEALTH STATISTICS

SUBCOMMITTEE ON STANDARDS AND SECURITY

HEARING ON PMRI MESSAGE STANDARDS

October 10, 2001

Hubert H. Humphrey Building
Washington, DC 20201

Proceedings By:
CASET Associates, Ltd.
10201 Lee Highway, Suite 160
Fairfax, VA 22030
(703) 352-0091

Participants


TABLE OF CONTENTS


P R O C E E D I N G S (8:30am)

Agenda Item: Call to Order

DR. COHN: The Committee is the main public advisory body to the US Department of Health and Human Services on national health information policy.

I am Simon Cohn, Chairman of the subcommittee. I'm the national director for health information policy for Kaiser-Permanente and practicing physician. I want to welcome subcommittee members, HH Staff, and other here in person. I also want to welcome those on the Internet and remind everyone to speak clearly and into the microphone.

The focus of the hearings today which is really a continuation of yesterday's hearings is to identify next steps for the Committee for Patient Medical Record Information Standards. Once again, I want to thank Jeff Blair and Mike Fitzmaurice for their work in terms of putting these hearings together. I think we made exceptional progress today and I'm looking forward to the panel this morning before move into broader issues of PMRI.

With that let's have introductions around the table and then around the room. As before, I would ask members of the subcommittees, if there are any issues coming forward before the subcommittee today for which they need to recuse themselves, if they would so indicate in their introduction. With that, Kepa, would you like to go next?

DR. ZUBELDIA: I'm Kepa Zubeldia with Claredi Corporation. I am also a member of NCPEP and X12.

DR. HORRI: I'm Steven Horri a radiologist at the University of Pennsylvania. I represent the American College of Radiology on the Digital Imaging and Communications in Medicine (DICOM) standards committee. So I'm both a standards developer as well as a standards user.

MS. LORENZI: I'm Virginia Lorenzi. I work at New York Presbyterian in the interface team.

MR. TSIOLIS: Good morning. My name is Thanos Tsiolis. I work for EPIC Systems Corporation and I work with interfaces mainly.

DR. FITZMAURICE: My name is Michael Fitzmaurice. I'm senior science advisor for information technology to the Agency for Healthcare Research and Quality. I'm liaison to the National Committee on Vital and Health Statistics and staff to the Subcommittee on Standards and Security.

MR. BLAIR: I'm Jeff Blair with the Medical Records Institute. I'm vice chair of the Subcommittee on Standards and Security. I'm a member of ANSI/HISP, HL7 and ASTM.

MS. ADLER: Jackie Adler and I've from NCHS.

MR. CLARKE: My name is Howard Clarke. I work for the National Electrical Manufacture's Association and provide the secretariat for the DICOM Standards Committee.

MS. HAWK: I'm Susan Hawk of ASPE.

MS. WILLIAMSON: Michelle Williamson, CDC, National Center for Health Statistics.

DR. COHN: Before we start, I do want to warn those in the room, those giving testimony and those on the Internet that there may be a fire drill at some point this morning. Especially for those in the Internet, if we suddenly cut out and there is loud blaring in the background, you will, of course understand. Jeff, do you have any introductory remarks to make for this panel?

MR. BLAIR: I'll try to make this a little shorter than yesterday. For those on the Internet and for those of you who are here to be our guests. This is part of the administration simplification provisions of HIPAA in the context of these hearings, specifically, the mandate to the NCVHS that we should study issues related to the adoption of uniformed standards for patient medical record information and the electronic exchange of that information.

Our direction was set forth by and NCVHS report that we submitted to the Secretary a year ago in August. Within that framework, one of the recommendations indicated that we would focus on the selection of specific PMRI standards in phases and we indicated a target for ourselves of February this next year to make recommendations on message format standards for PMRI.

And your invitation to testify is within the context of us already hearing from the standard development organizations by both testimony and questionnaires earlier this year. So, now, this is a panel of vendors and users of the PMRI message format standards.

We're very interested in your specific experience with respect to the specific standards both positive and negative so that the Committee can understand either what needs to be improved in these standards or how they related to each other or what's the right version that would be appropriate for specific standards. I think that that would conclude my remarks. Should we go ahead and have them introduce themselves, Simon?

DR. COHN: I think they already have.

MR. BLAIR: Okay. Then, Virginia, are you poised to be the first one to testify to us?

MS. LORENZI: Sure. I'm ready.

MR. BLAIR: If you didn't know, I'm blind so if I do funny little things like not look directly at you, that's the reason.

MS. LORENZI: I've heard stories of presentations that you've done for HL7 that were amazing.

MR. BLAIR: Well, thank you.

MS. LORENZI: I have more than 10 minutes worth of slides. So I'm going to skip some.

DR. COHN: Virginia, if you could move the microphone to you. These microphones are not very good in terms of capturing you at a distance.

Agenda Item: Fourth Panel

MS. LORENZI: Is that better? The standards we employ in our hospital are the HL7 V2.X suite, patient administration, observation reporting, order entry, financial management, and also a little taste of scheduling now that we're getting out to physician organizations. A pretty even split between patient administration. We have most on interfaces, but also orders and observations a close second and then charge interfaces.

We chose HL7 because we were trying to implement interfaces without standards and that wasn't working very well. It covered pretty much all of our needs at the time and it was the only widely implemented standard and that's still true today. So I think we made the right choice.

I wanted to give you my definition of a healthcare interface which is the integration of systems and applications and people and function and the hardest thing for us is this implication of different departments and the integration of different people. Those give us the non-trivial interface problems beyond stuff that's addressed by the standard.

The reason why I love this business is it's not just field comparison, it's hidden complexities. Who would have thought that on a patient administration system the admitting people would go ahead and cancel discharge and transfers on a dead person just to fix the admit dates and transfer trail? Certainly not the vendors who coded the interfaces.

So, all of a sudden, people came back to life in our ICU system and we call that the "ghost in the ICU" problem. Our doctors were really finicky. We gave them a list on our physician order entry system. We gave them a list of the orders from the pharmacy formulary.

And they got annoyed when we made them choose between 20 different entries for ampicillin oral. Do you want the 250mg size or the 500mg size; capsule or do you want a caplet or a tablet? HL7 helps a lot because it provides us a starting point for integration.

HL7 exceed expectations and it's very comprehensive. Everything is in there. Our vendors usually can't find where it is, but it's in there. It's compatible. It advertises that it's backwards compatible. You can take a 2.1 interface and send it to a 2.3 system and it shouldn't break.

What's really cool about HL7 is it's forwards compatible. We converted one of our main patient administration interfaces to 2.3, sent that to 30 downstream systems that were v2.1 and v2.2. And half of them worked. The reason why the other half broke because it was version incompatibility.

But that wasn't true. It was because the vendors who implemented those 2.1 and 2.2 interfaces didn't bother to follow what was written in the second chapter of HL7. The first implementable chapter of HL7 has receiving rules. It's real basic. It's written in a lot of jargon there, but basically ignore what you don't expect. If they ignored those new repetitions, new fields, new segments, everything would have worked. That was in v2.1 those rules, in there since the beginning.

The other thing that exceed expectations is the HL7 community. Imagine this: it's the middle of the night. I can't figure out some complicated HL7 and maybe it's even a stupid problem in my mind you know, and I'm not really very shy. I go onto HL7 and type in my question. All of a sudden some expert in the field writes me back with this detailed e-mail explaining every possible nuance of what I asked. In the morning, there are 20 more e-mails. Next thing you know, there's an argument about some point that I may have brought up.

This is how I started contributing to HL7. I started asking some questions, dumb questions and they said I got three different answers. I said, maybe I can contribute here. In addition to this unlimited support behind the scenes, I look brilliant because I've got all these people working for me. I work I could call my help desk and get that frontline of support from a vendor.

Whenever you have a problem, you see that someone in HL7 is already working on it and they want to hear it and they always work to fix everything. You ask them you question, how long did it take to implement HL7, which I enjoyed that question. I thought it was kind of funny.

One particularly HL7 interface doing straight HL7 to HL7, I would say the quickest we do it is in a month and that's because it takes time to do design, development, to really do it right, to go through the whole project life cycle and it takes time to communicate. A lot of people have to chit chat to do an interface.

Some projects just take a lot longer. ADTM financial systems usually go together, it's one system, but I guess for fun, we severed an ADTM financial system in half and pasted one onto the other. I got to implement every single field in v as a result in patient management and finance.

You get those problems like six insurance plans. You can put six insurance plans into one system, but the other six only supports four. What do you do? One system has the same kind of account number for inpatients and outpatients, one account number generator. The other system has two account number generators. What do I do when I convert from inpatient to outpatient? Account numbers change on the destination system.

Doing that physician's order entry interface to a pharmacist and medication orders, that was probably the most interesting challenge I've done recently and I had to become a pharmacist and a doctor to do them becuase you had to really understand the vendor applications and what the user wanted to find out, there's a big problem here. Both vendors have a different concept of how to implement IVs. We need to do some trick in between, and vocabulary is always an issue.

I would say HL7 is not easy to use. In the interface group, you bring up some person, they're out of school, couple years experience doing program development. They've got UNIX experience so they can use our interface engine. They're lost because our job everyday is to take the HL7 specification which is lengthy, and sending systems specifications and a receiving system specification and we've got to compare them.

The specifications are not the same because there's no rule that says how you write your specification. I've got to sit there and compare, does this vendor deviate from HL7 and will these vendors communicate, field by field? The burden of proof is on us, the users. By the time I finish comparing all this, I'm bleary eyed and I can't see the ghost in the ICUY or the real non-trivial problem.

MR. BLAIR: Virginia, with respect to your comment of that level of frustration, what version of HL7 were you talking about right now?

MS. LORENZI: It doesn't matter. V plus are much better defined. It's much clearer. They have much better definition, but at doesn't mean that my vendors really read it clearly. I still have to read it no matter what. It's still up to me no matter what version it is. I still have to read every single field and every single event and say, okay, does everything match?

I take pride in the fact that I'm very good at HL7 messages. They've all this gobbledy gook in it like vertical bars and I take real pride in that and how to make acknowledgements, but when you come into this world of HL7 stuff, it's not obvious.

I would say that over time, we became educated consumers of HL7. It becomes easier to use. With education and experience, we can do interfaces very rapidly, but that's because we're tough consumers. A vendor comes into our shop and if they don't implement HL7 the way everybody implements HL7, we get mad and we say go back and fix it. The tell us we can just fix it with the interface, and we say, no, you guys fix it. You know there's flexibility in HL7 and you figure it out.

You asked how much it costs and that was funny too. HL7 is big business for us. We spend about $2 million per year. Most of it's for staff because we've got all these people comparing the specs. We produce about 15 new interfaces a year. We've already got 400 in production.

Remember, some of those 400 were developed before we were smart about HL7 so they're high-maintenance interfaces. Without HL7, who knows how much it would cost. I'm an HL7 evangelist, but any HL7 fanatic is the first to say that HL7 is full of weaknesses, but every weakness that there is, v3 will make it better and I do believe that.

But there is too much optionality. It's very ambiguous and lacking definition. In HL7 v2.2 in patient administration for example especially, there are trigger events with no definition at all, field definitions that are totally inadequate, an English teacher would flunk the entire chapter. Starting from 2.3 on is when you get the good definitions of what already existed.

All these artifacts left in make it unreadable and every once in a while you get a vendor who implements something that was marked backward compatible six years ago. My husband always tells me you've got to throw out the old stuff every once in a while and I just think HL7 needs to do that every once in a while.

The biggest problem for me is that I can't test conformance. Vendors can't even specify conformance to me. Those esoteric encoding rules, those complicated encoding rules that nobody understands except maybe me, and when you get into vocabulary and you see a code table, you know you're in trouble. You're out of your standard land.

Like I say, I really think that v3 will fix everything, but while we are waiting, Santa Claus what I would like for Christmas is I would like my vendors to start using a tool that's already been developed, that's supported by the special interest group for conformance, was written by Peter Ronte in the Veteran's Administration, and is called Messaging Workbench.

This tool is holds a vendors hand and helps them implement their specification. They can implement 2.1, 2.2 or 2.3 and it asks them, do you require this field? HL7 doesn't require data first, but most people do. So you get to mark it's required. What's your actual length of this field? What are your business rules? You can put all sorts of commentary of how you really use stuff.

It walks you through that, produces this beautiful spec because it's an XML, but because it's an XML, you can do stuff on it programmatically which means that I can sit there and push a button on this tool and say tell me how this is different from HL7 and it shows me.

I can take two vendor specs that were written with this tool and say let me say what's different and it shows me a list. So at least I have a starting point. I don't have to do all that stuff, comparing each thing back and forth. Then, what's even better, it's in the prototype phase now, but on the HL7 Web page, you can start sharing these profiles.

You can start developing a set of profiles that we really want people to use. The only other thing is I'm very big on people upgrading within v2, but I would say even if you want to give me a 2.1 or 2.2 interface, when I evaluate it, I want to read the latest version of HL7 so I can understand what those 2.1 and 2.2 fields really should have meant.

I think the 2.3 plus versions are much better than the older versions. I'm a big believer in v3. My biggest concern about v3 is it's being developed by all these fanatical technologists, but I want those vendors who weren't able to understand Chapter 2 in v2 to be able to actually implement this successfully.

I would like us poor folks in the hospitals to be able to actually understand what the standard says. I want to make sure it's understandable. If it doesn't work as advertised, I don't want it. HL7 put out a ballot, major milestone. I did my part, I reviewed it, and I voted no, no, no. That's what I'm supposed to do. I think it will take a while for it really to be right. I don/t want it too soon. I don't want half-baked chicken. I don't want to get sick.

MR. BLAIR: Thank you, Virginia. Who is our next testifier?

DR. COHN: Thanos probably should be our next and then he'll be talking more about HL7 I presume and then Steve will talk about DICOM.

MR. TSIOLIS: Thank you very much for the opportunity of being here today. I represent EPIC Systems Corporation and recognized as innovative technology company with client-center values, EPIC Systems Corporation is a leader in developing and supporting state-of-the-art, futuristic software that helps organizations change the way healthcare is delivered.

Its comprehensive portfolio, clinical access, and financial systems centers around the single based record delivering data level integration across all environments and providing the information management that organizations need to enhance services, trim costs, and delight patients.

Every day EPIC Systems help many of the country's leading healthcare organization take care of patients and their business. We're based in Madison, Wisconsin, and EPIC was founded in 1979 by a group of healthcare and IT professionals. Today, EPIC remains of the largest and most successful privately owned vendors on healthcare information systems.

Now, I'm going to jump to the answers to the specific testimony questions and starting with the question, if any of the standards listed on original questionnaire address the highest priority for exchanging PMRI information and our answer is, yes, for the most of the practical purposes.

Priorities that are not met by these standards were concerned with a lack of overall PMRI structure among the discussed standards. It is our belief that the list of standards provide the means for exchanging pieces of the PMRI structure, but they do not provide consistent, overall framework.

We believe that HL7 v3 provides such a global framework. The specific standards I'm going to address have to do with HL7. I'm going to talk about HL7 v2.x, order and transaction sets, observation reporting, patient administration financial management, patient administration, scheduling, medical record and payment management, clinical document and detection.

I'm starting with the order and transaction set and observation reporting, Chapter 4 and 7 respectively on the current HL7 specifications. They provide a rich order entering results of functionality that has been crucial for the success of our clinical applications since their introduction in 1994.

The two transaction sets discussed here have allowed us to provide this functionality to customers such as Kaiser-Permanente Northwest, Harvard, Vanguard Medical Associates and other large groups. These sets have facilitated a successful exchange of clinical information between our applications and other external entities.

These particular standards were not as popular or not as accepted in 1994 as they are now. Never, we show their potential and understood that direct participation on our part was crucial to developing a promising standard. We selected these specific transaction sets based on relative completeness of data elements included in the standard, representative work for all customers, glowing acceptance of the transaction sets among the rest of the vendors, and data presentation in abstract application neutral format.

About our experience during implementation of the standard. Our answers to the following questions cover our view and efforts in developing the general framework needed to implement the HL7 standard as part of our application functionality.

Our answers do not, however, reflect the possible effort that our customers needed to utilize the standard for actual data exchange among the various vendors. Along with incorporating data elements included in the standard within clinical applications, we implemented the general framework, the model of HL7 messaging. Based on this generic framework and implementation of the various HL7 transaction set were relatively simple.

We didn't find any aspect of the implementing the standard that exceed our expectation. The initial interface development following the standard took us approximately six months for both interfaces, the transmittal and the result receiving.

However, the interface development is an ongoing process as the standard evolves and as you add more options and functionality in the interfaces and applications. When we implement these interfaces with our customers today, the user implementation time which includes analysis, installation of the interface, application set up, programming if needed, and testing of course, is between 3-6 months depending on the complexity of the project.

The implementation cost for our customers consists of the license fees, staffing from the implementation period, and the maintenance staffing once interface is active in a real data environment. These costs vary based on the complexity of the customer systems and their specific interface needs.

This implementation period can be considered to be relatively long. There are reasons behind it. The long implementation and testing time are that not all vendors follow the standard and very often we have to compensate for that.

Ambiguity and optionality in the standard specifications and it's our belief that the main reason for the ambiguity and optionality in the specifications comes from the fact that the individual transactions we address were not designed to be a part of the global PMRI framework.

For example, the standard suggests the preferred way to submit microbiology results in HL7 v2.4 Chapter 7. The suggested format is very precise and contains all the required information for a complete microbiology transmission. However, the fact that this part of the specifications was not made required, it resulted in many vendors implementing microbiology results reporting in an ad hoc and less precise way.

Out of the writing here, this may be the microbiology results, the most time consuming part of the process of installing an interface that has to do with other microbiology results and sensitivities. What can we do to improve the standard?

The standard could be based on assorted foundation using advanced methods and tools in modern and systems design. This will ensure internal consistency among transaction sets and will greatly improve their ability.

HL7 has realized this requirement and has developed a reference information model as the basis of our newer v3 efforts. The version of the HL7 transaction sets is no the ballot and we believe that this new version should be considered by the Committee for inclusion in any recommendations here.

The next response has to do with our evaluation of one of the most current versions of HL7 which is v3. In the past three years, we have participated in the HL7 demos at the theme expositions. We have developed prototypes for v3 of these transaction sets, each time adapting our prototypes to the current form and specifications.

We found it relatively easy to implement because of the features of the new standard. The formula is based on the extensive market up language, XML and you can use freely available tools in our prototyping implementation. The underlying reference information model facilitates the use of computational objects, source code, programming libraries, configuration settings, and provides a consistent framework for implementing these interfaces. We are currently, actively participating in the balloting of the new v3 and we plan to implement the new version as soon as it is approved.

The expected time frame for completing the balloting process is the middle of 2002. I believe this is our timing at this point. I don't want to bother you again and read the same general ideas for the rest of the actions. In my written testimony, please note that under the part that has to do about the easiness of use of the standard, some specific examples that have to do with the optionality and the ambiguity in some of the specific set of transactions so they are clearly marked there.

I'm not going to bother you now reading the specific examples. I would like to jump to the v3, clinical documentation framework, which is quite important in our opinion. It defines a mark up of clinical documents based on the artifacts of existing healthcare environment.

The standard provides an important uniformity for a desperate document format existing today while allowing for expression to be preserved. Then so that it was not left out since I've already taken a good amount of time here, I would like to jump to the last question, Number 7.

Are there any other standards that support exchange for PMRI that NCVHS would be considering? In short, HL7 v3. I am in agreement here with Virginia that v3 is the future and we should make sure that we consider it very carefully.

There are several reasons EPIC believes that v3 should be considered as part of the standard here. V3 is based on a comprehensive abstract reference information model. The development of this model has been done over a significant period of time and with the participation of almost every major player in the healthcare information technology field, all the major healthcare vendors, leading healthcrare providers like Kaiser-Permanente and the Mayo Clinic and certain companies like Apgemini(?), Heaston Young and the Gardner Group, government agents like the VA and DOD.

The methodology employed in the development of v3, reference information model will bring healthcare information technology into the 21st century. Because of the comprehensive nature of the model, issues like security can be addressed as part of a standard rather than being treated as add-ons at a later point.

HL7 is also international organization with affiliates around the world, the results of taking into account and then sure incompatibility with existing work in other parts of the world like the CENF14 and the government electronic health record in Australia.

The standards methodology requires four months description os application roles and interactions for every transaction set. This proceeded a certain level of conformance requirement which is not available in previous versions. When he HL7 v2.x transactions were made part of the claims attachment provision of HIPAA, it was a significant effort needed to strictly define the constraints under which the HL7 transactions are deemed as satisfying the requirements.

The form that is used by HL7 v3 is XML. This allows implementers of the standard to freely existing fields to process a different transaction. This results in increased interoperability, quicker adaption, and lower cost of implementation. In addition, HL7 v3 provides another very important feature, format independence.

Because of the reference information model underlying the standard the semantics represented by the transaction sets are independent from the format which apparently XML provides the best format for these transactions, if a better one is invented, it can be easily adopted by the standard without changing the meaning also expressed in existing transaction sets.

There are several possible arguments against considering HL7 v3 for the exchange of patient medical information. We would like to address three of them. The development of the new version has not been completed yet. We would like to point out that the development of this version started in 1996 and the current reference information model as well as the development methodology are rather mature.

The new version is on the ballot now and is expected to be approved by the middle of 2002. Around the standing of the scope and goals of adopting standards for the exchange of PMRI, the new version will be available well in advance of the adoption of any specific requirements.

V3 has no installed base while v2.x is widely adopted. It's very educational to look at the proceed os including the HL7 v2.3.1 transaction sets as part of the requirements for claims attachments under HIPAA.

Even though the transaction sets have existed for several years and they were widely in specificity needed for that particular purpose requires significant effort in removing ambiguity and optionality from the standard.

The result has been a transaction set that requires just as much effort in implementation as a totally new transaction set. In contrast, v3 of the standard incorporates features within its development methodology that will enable a specification close to if not identical to the one used by organization in cases falling outside of the realm of exchanging PMRI.

Finally, there is an argument against v3 since v3 uses a different format and methodology from v2.x, making it a requirement will significantly increase implementation costs. As pointed out in a previous paragraph, implementing a new specification for exchanging PMRI will require significant effort regardless of whether v2.x or v3.

We believe that using v3, we'll have a lower cost in the longterm. As the specificity of the standard itself, we enable implementers to use development efforts with greater ease that is currently possible with v2.x.

Based on our experience, we also believe that properly engineered applications will have very little difficulty switching from v2.x to v3 over a period of several years. Indirectly, adopting v3 transaction sets in standards specification will improve competition in the marketplace as better engineered application will have an appropriate advantage.

In conclusion, we'd like to thank NCVHS for giving us the opportunity to present our work and ideas on an industry standard for PMRI structure. We hope that stat standards will be finalized in the near future and that vendors, like us, contribute to the success of this effort with steadfast and full implementation of the standard. Thank you very much for your time.

MR. BLAIR: Steve, if it's possible to keep your testimony within a 10-minute guideline, that would help as well.

DR. COHN: Do you have a question of clarification?

MR. YASNOFF: I just want to ask a quick question to clarify: You had said that you were going to implement v3 as soon as it's available?

MR. TSIOLIS: Correct.

MR. YASNOFF: What are you going to do for your customers who are using 2.x and either need to continue to use it? How are you going to support 2.x after you implement it?

MR. BLAIR: That's more than a clarification question really.

MR. YASNOFF: Really. I thought it would be better to talk about that before we get into DICOM, but it's up to you.

MR. TSIOLIS: A very quick answer, we'll continue supporting v2.x and we're not going to force our customers to jump to v3. We'll have it available though.

DR. HORRI: Thank you all for invitation to talk a little bit about DICOM this morning. I think you've probably seen some material on DICOM. I don't need to go into extensive discussion of what the standard represents or what it is, just tell you that it started life with the American College of Radiology and the National Electrical Manufacturer's Association in 1984 with the first meeting actually starting in 1983.

So the standard has been around for awhile. At the time, the standard only had two sponsors, but now is a multi-member, international standard with representation from Europe, Asia, and multiple medical specialties. One of the nice things about working with DICOM group is that the manufacturers set aside their competitive strategies and the physicians set aside their turf issues and work together to produce a standard.

You'll hear me mention a couple of acronyms. You'll hear me talk about Picture Archiving and Communication Systems (PACS) and also Radiology Information Systems (RIS), and Integrating Healthcare Enterprise (IHE). I should also point out that a great deal more information about DICOM is available on the Internet as www.NEMA.org. If you look at the medical section and navigate to the DICOM under those pages, you'll find everything up to and including the full text of the standard available on line.

I've tried to address the specific questions that were asked. One was, do the message format centers, DICOM, on the attached list that we were given, address our highest priorities. Radiology departments are largely digital. Rather than film based, we've gone to digital detectors of various sorts.

In assembling the information for radiologists to use clinically, we have these systems called PACS. The primary role of DICOM is the interface standard to connect equipment to that system, the PACS and DICOM has certainly addressed those needs for us.

The high priority of PACS and the support of the imaging needs of the medical facilities and DICOM has served well in that regard certainly as far as we're concerned and reports from other institutions would tend to support that. The specific standard that I'll be discussing is DICOM 2000.

Now, there is some terminology that needs to be explained. People will talk about DICOM3 and there's no DICOM2or 4. We differentiate the versions of the DICOM standard by the year. The 3 comes from the fact that it started life as DICOM 3.0, but since we have a multi-part standard, it consists of 15 parts presently.

We differentiate those parts by saying 3.1, 3.2 up to 3.15. That doesn't mean it's v3.15 of the standard. It means it's part 15 of the standard and the version is given by the year.

In our institution, we've implemented DICOM all the way from the very first version of it which was the 1993 version all the way up to the present 2000 version. The third question was what functions or characteristics caused us to select this particular standard?

The major factor influencing our decision to use DICOM was the fact that it's widely available for multiple imaging equipment vendors. Virtually all of the vendors we use now can supply us with DICOM interfaces for our equipment.

I would say that most other radiology practices and departments that I'm familiar with have found a similar thing to be true. There are still the occasional vendors, you ask them about DICOM and they sort of look at you with a puzzled look. That's becoming much less frequent.

The DICOM implementation by manufacturers support the functions that we need to do, that is, get data out of imaging equipment into PACs and more recently, they've allowed us to automate some of the steps that we have in radiology, things like entering PMRI.

Well, our hospital and radiology information systems already have all the patient identifying information and demographics and having the technologies enter it a second time just opens it up for errors. With automated connections between our PACS and radiology information systems using HL7 interfaces, for example, has really made this a lot simpler and a lot more robust.

About our experience in implementing the standard, there was a question about whether there was anything that exceeded our expectations and actually there was. We were quite pleasantly surprised. My clinical subspeciality is ultrasound.

We had several occasions in ultrasound where using DICOM turned out to be plug and play application. Following the manufacturer's instructions, we entered the appropriate network addresses and host names, plugged the machines in, rebooted them, and were talking DICOM within half an hour.

This worked for both imaging equipment and printers and we were quite pleasantly surprised by that. It was much better than we thought. Two of those installations were ones that I did myself. I will admit to being a little more computer savvy than most radiologists, but it was that straightforward that it didn't require my entire informatics team to help me with that.

How long did it take us to implement the standard? In a lot of senses, we don't implement the standard. We use it. The standard is built and implemented by vendors. In our research divisions, we have had people write code to do DICOM, to do things like strip the patient identifiers out and substitute anonymous ones.

That turns out to be a straightforward exercise because of the way DICOM is structured with its information models. Aside from the implementations that we've seen, the real driver for us in radiology in installing new equipment tends to be the other aspects of hardware installation and we tend not to be throttled back by the DICOM piece. That tends to be much shorter than installing the cables and power lines for the equipment itself.

Our total implementation cost is difficult to determine. What I can tell you is that most manufacturers now include DICOM as either a standard feature on their equipment or an option. In the early days of DICOM implementation, when it wasn't that common, a typical cost for interface boxes, for Legacy equipment, for the vendors who would come in and retrofit things, or as an option when it was a separate option ranged between about $10,000-$25,000(US).

That's just a ballpark figure. There were some that were much less and some that were much more than that. The cost also has to be examined in comparison to the alternative which would be to do custom interfaces for the equipment and we've actually done some of that.

In terms of development of custom interfaces for imaging equipment, we're talking about 4-6 person/months of computer programmer and engineering work. It's a fair bit of work. After that, you also don't have support from the vendor for it. You have to have your own staff support the equipment and that tends to be fairly expensive.

We think from a cost standpoint that using DICOM is cost neutral if not cost savings over customer interfaces. After we completed the implementation that it proved easy to you use, or for the most part, once we did the initial connection to DICOM, conforming equipment to our PACS, it's a very low maintenance implementation.

DICOM uses off the shelf communications hardware and protocols, CDPIP and IEEE802.3 as communications. Because of the wide use in information technology sector, this stuff is inexpensive and it's easy to maintain. The hardware pieces of it are fairly straightforward. The only real problems that we've seen with the implementation side are when vendors to things like upgrade their software.

One vendor comes in and updates software on a machine that does something to the DICOM piece of the interface and now it no longer communicates with the PACS. In general, those instances have been generally few and far between, but they do occur. When they occur, it's not usually a great problem to solve them. It requires getting both vendors in to find out where the problem is.

If I talk about major weaknesses or limitations of the standard, on weakness has been a function of its voluntary nature. There's no enforcement mechanism for DICOM conformance. There is a conformance requirement that those claiming they support DICOM are required to submit a conformance statement which is done in a standard fashion so that it's very readily possible to put together two conformance statements side by side and determine very quickly that two implementations will not work together.

It's a little bit more difficult to prove using conformance statements that they will work together, but it's very easy to prove that they won't work together. One of the problems of being a voluntary standard is that we have a mechanism in there that allows manufacturers to include proprietary information in a DICOM message.

It's not a violation of the DICOM protocol to do that, but some manufacturers do and the side of that is that the voluntary nature of the standard also we think requires a lot less bureaucracy than if you have standard that's required by regulatory agency. So we think there is an advantage to being a voluntary standard.

The second weakness has to do with the fact that there is no standard database query mechanism built into DICOM. We define a query as a true mechanism, but it's not SQL based. This goes back to days when a number of us lobbied to have SQL as a query mechanism.

At the time, not enough of the vendors were familiar with implementing it to go along with that. So that's one of the disadvantages currently of the standard. It does not employ a database standard query mechanism.

A third weakness is one that's not universally agreed upon to be a weakness and that is that DICOM specifies all the way down through network hardware, but some would say that our real value was in the information models and that we should cut it off there and just let it run over whatever network hardware exists.

On the other side, there are those that say the network hardware that we specified tends to grow up as the requirements do. We are running DICOM over Ethernet all equally well so that as the requirements in imaging have increased and the network speeds have increased, we have been able to just continue to use the existing DICOM protocol with no problem.

What could or should be done to strengthen the standard? There is a specific working group 10 whose task it is to examine what's happening in other standardization efforts in healthcare and what technology advancements might benefit DICOM. An example is security mechanisms that now are part of DICOM and continuing to grow.

Improving the database aspects of the standard are one area that I think the strategic group has been looking more long term than short term. Have we evaluated more current versions of the standards and, if so, what is the evaluation of them?

Since we're heavily manufacturer depended, it's really difficult to say that we have personally evaluated those, but our vendors supply us the current versions of DICOM and we're required in our request for proposal and contracts for purchase that manufacturers support the most currently available versions of the DICOM standard.

We've also started specifying more recently functionality that derives from the integrating healthcare enterprise the IHE effort. It makes it a lot easier for example to specify what we want between PACS and Radiology Information System if we say the vendors must conform to this IHE year 2-3 profile.

The reason we use those is that IHE profiles are heavily built on DICOM and HL7. So they are built on existing standards. That organization does not define new standards. Along those lines, the question about are there other standards that support the exchange of patient medical record information that the NCVHS should be considering.

I think looking very carefully at the work that the IHE has done would be very useful because that's a group that has taken existing standards and done profiles through them rather than defining new standards, just saying what is that folks need to do when actually operating in a clinical scenario, things like getting patient registration information from radiology and hospital information systems and scheduling information into our PACS.

If the PACS knows that a patient is coming in for a chest X-ray or another type of study, it can go ahead and pre-fetch from long term storage all of the previous studies on that patient. So when the radiologist sits down to work, the previous information is already there. They don't have to wait 10-15 minutes to pull that stuff up.

There are a number of things like that like how we deal with the unconscious patient who comes into the emergency department who is a John Doe. We later find out who that patient is and we have to go back and the medical records folks know well about this. How do you go back and propagate the correct name to the record.

In radiology, we have the same problem. In the days of film, it was fairly easy to put a little sticker over the John Doe and write the correct name and medical record number on. How do you do that in PACS environment where everything is digital. That's where we needed the interaction between different information systems and that's when the strength of the IHE has come from.

More information is available on the IHE effort. It's a Web site. It's www.RSNA.org/IHE. That will get you a number of different information bits about IHE and I think that's a fairly useful thing to look at.

I will summarize to say that DICOM has become the predominant standard for radiological imaging and it's rapidly being adopted by another specialities that generate digital images. They all have representatives on the DICOM standards committee. It's wide adoption and implementation are a testament to how well it works. At the introduction of DICOM during the 1992 RS&A meeting, there were about 25 or so vendors who were demonstrating the standard.

Currently, among the thousands of vendors who exhibit at the RS&A, it's actually now very rare to find an imaging equipment vendor that does not support DICOM. It moved from what was a works in progress area of the Radiological Society of North America meeting to the show floor where people buy it as a product.

The work of the DICOM standards committee is done by volunteers from industry, medical professions, and standards organizations. I think it's an excellent example of how a standard can be built through collaborative efforts. We currently have a joint working group with HL7. It's one of the things that's unique in the standards industry. It is both an HL7 group and a DICOM working group. We have representatives from both sides that. That's to help us line up with what's happening in HL7. Thanks very much.

DR. COHN: Questions from the group. First of all, thank you for a wonderful set of testimony.

DR. FITZMAURICE: I want to echo Simon. It continues the excellence of the presentations yesterday. This is really a top-notch group. I want to ask Steve Horri, does DICOM have a patient or imaging information model? Does it fit in with something like HL7's rim? Is there a workgroup working on this? You answered part of that question saying you're working with HL7. Are you working with how the imaging fits in with the total patient record?

DR. HORRI: Yes, that's one of the major tasks of that working group. There is a thing that just says patient in our information system. There are identifiers and attributes that go with that, but it's not nearly as robust or as complete as what's in HL7. So there has been an effort to make sure that those two models, the information models don't conflict with each other. Rather that we employ what's used in HL7 where possible.

DR. FITZMAURICE: I'd like to ask Thanos a question that probably is Bill's question. Is there a problem with existing installed base that uses HL7 v2.x? Is this a matter of building another interface when you go to v3 or converting the software that's in place already which is a very difficult task. How do you move from v2.x to v3 over time?

MR. TSIOLIS: We beleive that the major issues of information that is exchanged with the interface is not going to change. We're still going to deal with the patient name, we're still going to deal with the patient identification. What changes is the format, the standards and the way it's built in v3. It's independent, but the major gain of going to v3, I would think is first to organizations that just start their interface. They don't have an existing 2.x interface and they want to start a new interface.

The clarity and the lack of optionality that we see and it will stay in v3, will help them a lot in implementation and maintenance cost. Organizations that have a v2.x interface, if they wish to switch to v3, again, the clarity of the information exchange is going to be better because the applications that are behind these interfaces change, evolve.

We get new pieces of information and new pieces of data in there that will eventually need to be exchanged. HL7 does -- on these changes incorporated in their standard. Our belief is that it's going to be a lot easier to make these changes to existing bases with v3.

DR. FITZMAURICE: Would you also say that v3 will be less technology dependent?

MR. TSIOLIS: Correct.

MR. BLAIR: Could I follow up on that question? Do you envision that there will be some software vendors, systems integrators that will offer interface engine services which would allow your systems, EPIC Systems, for example, it might be sending a v3 message, to have that converted to a 2.x capability of receiving it from a lab system or from a pharmacy system or from others? Do you envision that that will happen?

I'm still trying to understand whether it will require that the system that you're talking to will have to convert to v3 or whether they will be able to still be on v2.x? What do you think is going to happen?

MR. TSIOLIS: I do not see this ability on the interface as something that is not going to happen. Let me rephrase. It is possible that we will have an interface engine or a conversion that will go from v3-v2.x. On the bottom line, v3 is XML. So there are very nice tools that convert from one format to the other.

What I'm concerned about is the extra functionality and the extra pieces, well, the presentation of some of the extra pieces of information that may be incorporated in v3 and they don't have a clear counterpart in v2.x. The problem with v2.x at this point again, is optionality and ambiguity. How are we going to get a model that is very specific?

The reason that this model was created in the first place was to eliminate the problems that v2.x had and get it back to 2.x, I don't know exactly how this is going to happen.

MR. BLAIR: To clarify it, you made the statement that you really felt that the primary opportunity would be when you a new install to make that a v3 implementation, not to retrofit to a legacy system.

MR. TSIOLIS: Correct. V3 implementation we believe it's going to be a lot easier in the long term for the organization to implement. In the previous statement I also said that it may be worth it for organizations that have a base of 2.x interface to upgrade to v3 once they start adding functionality in their applications that will be better represented in v3.

MR. BLAIR: Can we have Virginia's observations on this same topic?

MS. LORENZI: Good. I was getting excited here. I work in an interface engine group. I'm all ready to convert from v3 to v2. If I've got one of my big patient administration vendors comes and says, we're going to do v3 and I've got just one other innocent vendor who's willing to give us v3 ADT receiving, maybe we're willing to do that challenge.

The problem, of course, is that that's an interim thing and I really want to be able to throw that code away eventually. Sometimes that's not what happens. Sometimes vendors say, well, you have an interface engine, right? So you can deal with all the differences between the v2.2.

I see that that definitely is a possibility and in reviewing the ballots especially for the big heating applications like results and patient administration, I will be looking that there be some version conversion we can do for the essential stuff.

But there are things that Thanos pointed out that the events are very specific and you're going to get maybe 10 events out of v3 which are always represented in the same event in v2 and maybe in some of those questions, I can send the event 10 times and maybe you can't in some cases and then I would be stuck.

The pharmacy and lab HL7 v3 work is very sophisticated and I'm hoping that in some of those cases I really get both vendors implementing v3 so I can get closer to plug and play there earlier on.

MR. TSIOLIS: Could I add one more thing? Actually, it's a little more expansion to an earlier question about -- I can talk only about the vendor that I represent -- if we're going to still continue to support v2.x or join to v3 and stop all the support that we have?

From what Virginia is saying, this seems to be on the minds, and since we got the question, it probably this question is on the minds of a lot of big users of current software. Again, speaking from the EPIC perspective, I'm going to continue supporting v2.x. As far as I know, there is going to be a v2.5 coming out as well. We will continue supporting this version. We will also offer the alternative of v3.

It takes two to play. If the application that we're talking to or receiving data from is not ready or does not want to exchange data with v3, we're not going to make anybody, but it's very important to point out the gains in the long term that the end users are going to have by v2 to v3 on a new installation or an existing installation.

DR. HORRI: I can point out that there is actually an example from the DICOM days. We started out with the AC standard that was not object oriented, used a 50-pin parallel interface that was defined by the standard that no other segment of the information technology market used.

And yet we continued to have that as a part of DICOM for backward compatibility and there were vendors who made boxes that could transition from DICOM to the ACRMINA(?) standard and gradually what happened is those got phased out. As new equipment was acquired and replaced, the older stuff just gradually got phased out, but that was a market that was created. There were several vendors who stepped in and said, we will build you a box that will convert from DICOM to the old ACRNIMS so you can continue to use your old CT scanner for a while.

DR. COHN: Kepa and then Bill.

DR. ZUBELDIA: How long would a process like that take? We heard yesterday that most of the HL7 implementations today are maybe v2.2 and then if you were to implement something today, you would probably implement v2.4 and if you're going to implement something next year, maybe v.3. How long would those legacy versions last?

MS. LORENZI: The only thing I want to say is I know there is a lot of this v2.2. Remember how I said we upgraded to v2.3 and half our system still worked? In 2.2, most of what you really need has already been defined. It just doesn't have good definitions and there's lot of ambiguity in the spec written.

V2.3 and 2.4 write the good definitions that explain what was already defined before. So if someone just actually implemented 2.2 correctly by the book and actually followed what was written or not written, but people should have just understood, then I don't v 2.2 is that far from v2.4.

DR. ZUBELDIA: So if you implement v2.2, you're saying that's 2.4 specifications?

MS. LORENZI: Yes, you can do that really though.

MR. TSIOLIS: To follow up on this, following the model if it's not broken, don't touch it, don't fix it, I believe that old versions are going to stay around until there is a need for extra information, extra capabilities that these versions don't provide.

For example, v2.2 does not have the transactions that were introduced with v2.3 for transcription, the MDM transactions and so on. There was a need for them and they provided some extra functionality there. An organization that is using v2.2 and is a need for the extra functionality that these transactions offer will have to move the new version.

So I don't have a definite answer in time line. I can just say, they're going to stay around, the actual users find something that they cannot achieve with the existing version. How long is this going to be, with the kind of technology, I wouldn't expect it to be too long. The applications are changing rapidly.

DR. YASNOFF: I have several questions about DICOM. The first question is: Is DICOM used extensively for exchange of images between institutions as opposed to in tact systems within institutions?

DR. HORRI: Yes. In a couple of different ways. There is a part of the standard that addresses images recorded on removable media, CD, optical disk and I've engaged in interchanges with other institutions where we just basically sent each other CDs. That is certainly a thing that's used. There is also electronic transfer that folks do transfers over the Internet using DICOM.

DR. YASNOFF: You touched on conformance issues just briefly. I inferred from your testimony, but I want to ask you explicitly, is there an actual conformance testing tool that is available?

DR. HORRI: There is no gold standard. Rather, what tends to happen in the DICOM community is that it's driven by market forces. So people say, well, I've got a manufacturer X CT machine and a manufacturer Y PACS and they're talking fine over DICOM.

Another user may say, well, I've got the same PACS, but I've got another manufacturer CT and it doesn't work and we found this is the reason why, this is the problem with DICOM and they're fixing it. There is no, there's been a lot of interest in having a gold standard, but at present, there is not one. There is no DICOM police.

DR. YASNOFF: I'm not talking about enforcement, but would it be helpful to have a conformance tool where you could send an image and it would say, yes, this conforms, no this does not conform. Or you could test two systems and see if they really --

DR. HORRI: The other model that's been used, the model that we use is at the Radiological Society of North America. They actually establish a DICOM network for the entire show. If you can hook into it and you can send images, you're DICOM conformant. Again, it's a large demonstration in a public forum. So you really don't want to fail. It really puts a lot of pressure on the manufacturers to conform and that's probably one of the more stringent tests.

There is also a lot of offline testing that goes on. Vendors will talk to each other and say, this is my new DICOM software for this new equipment. Let's make sure it works with your version of DICOM. There is a lot of that that goes on.

There are some third-party vendors that build boxes that will output standard things like test patterns and things in DICOM format, but there is no seal of approval that says that is the DICOM standard.

DR. YASNOFF: So there is a testing activity that occurs at RSNA, but there isn't anyone to essentially certify that the test has been done?

DR. HORRI: There's a meeting called INTEROP that's network oriented. This is the same kind of idea.

DR. YASNOFF: Finally, you mentioned that there are three major versions of DICOM, 1,2,and 3. What are the use patterns of those versions in terms of percentage of user community using each one?

DR. HORRI: Basically, there are currently only one version of DICOM. The major transition was from ACRNMA standard v1 and v2. When DICOM came out, we just called it 3. Really, the differentiation is by year. So anywhere from 1993 to 2001 will be coming out relatively soon, our versions of DICOM, and I would say that the majority of the users are probably on the 1997-8 versions of DICOM in terms of the number of things that are supported in it.

The versions are backward compatible and we don't take away attributes and things that are already there. There are things that become obsolete and no longer supported, but it doesn't mean you can't include them in your messages. It just means that you're not obligated to deal with them.

DR. COHN: I was going to follow up. Thank you for clarifying the backward compatibility issue because I think that's a key piece. Tell me a little more about the RSNA process of public demonstration of conformity. I wasn't sure you couldn't actually even show anything at an RSNA meeting unless it was DICOM compliant.

DR. HORRI: The RSNA has some standards for exhibitors, but the way this all started was it was decided to contract with the forum to promote DICOM and to get some implementations off the ground. So the ACR and NMA and the RSNA together funded an effort. It was eventually won by the Mellon-Carn (?) Institute of Radiology at Washington University to develop software which was then made publicly available.

You can still download all of that code to do DICOM. They set up a central test node or central server that acts as a repository for images. The vendors received all of that code after it was developed by Mellon-Carn and they went and implemented against it.

That really became the standard implementation. The other thing that happened that was very interesting is the very next year, they let a contract to a group in Europe to develop code independently of the work that was done at Mellon-Carn so they did not use the same code.

The very nice thing that came out of that was that the two sets of software were compatible. They operated together. They were two servers, but they could freely exchange information along with all the vendor implementation. That's the way it goes.

In about this time of year actually, about September of every year, there's actually a time when the vendors get together with the RSNA and they test some of their equipment against the DICOM implementation. So they know ahead of time and they're not going to just show up in November at the end of the month and find out that their stuff doesn't work.

Although there are some fascinating things that have happened. There was one of the user exhibits one year from USC or UCLA. They actually had no idea that their exhibit would work until they put it together on the exhibit floor. They came in. They literally bought commodity parts. They bought PCs and they bought displays from a bunch of different vendors.

The stuff showed up at McCormick Place. They plugged it into DICOM network using the software they had written and they got images displayed. There have been some fascinating tests of this thing and it really does work.

The big hurdles to get over were the ability, for example, GE CT images on a Philips display system or on a Seimen's display system and they can do that now. You can go to different booths and say, I want to see test image number so and so, and it comes up on their workstation.

DR. KOLODNER: Steve, thank you for the presentation and clarity of the information you provided. Two questions. First of all, DICOM grew out of radiology and that's the area where it's implemented the most. Can you comment on the use of DICOM for all types of medical images as opposed to just radiology images?

DR. HORRI: The other big user currently is cardiology. The cardiology folks have a working group that's cardiac and cardiovascular information. As you probably know, cardiologists and radiologists have this natural turf battles all the time, but we set those aside and came up with a standard that we could both agree on.

The cardiology folks realized they didn't want to reinvent the entire standards wheel to do their stuff. So they just applied the appropriate things to DICOM and came up with an information object that they could use for cardiology information.

It's similar to the one that our vascular radiologists use. Cardiology is probably the second big user. The ones that are coming along have been a little slower, but ophthalmology, there are a number of ophthalmology, fluorescing and fundoscopic images, they're all digital now.

Those folks have some DICOM implementation. Gastronuerology for endoscopic images. Most endoscopes now are no longer fibroptic. They have a little CCD camera out at the end. A lot of them are digital. I was surprised to get invited down there, walked down and found out they basically had a PACS for several years and their stuff is all filmless and stored electronically just like we do.

Now, the manufacturers are starting to supply the equipment, the output DICOM. Pathology is another area. They've actually been more helpful to us in terms of SNOMED and standard nomenclature rather than actually image piece because they're still working out how you represent color images with Hi-Fidelity and also very, very large image sizes.

There are some things that pathologists are working on, but I have heard that there are a couple of digital microscope cameras that you can get DICOM data out of. There have also been some completely non-medical, people found the DICOM standard useful enough that there's a thing called Digital Imaging and Communicating for Non-Destructive Evaluation (DICNDE).

The folks who x-ray jet engine parts and things. They're using DICOM. It just doesn't have any of the medical stuff in it. They defined it on information objects, but if I were a workstation vendor, I could actually take one of those images of some x-ray of an engine part and I could display it on my workstation if I could figure out their bitmaps and things.

It's turned out to be a fairly nice framework for a number of people to use. The other specialties are signed on. Cardiology, ophthalmology, dentistry are probably the three biggest non-radiology users of the DICOM standard.

DR. KOLODNER: The second question and this is an area where maybe some areas can elaborate the details. Yesterday, one of the people who was presenting talked about DICOM as being a wrapper around several different types of image objects. They were advocating that there be a Web interface or a way of showing the images through a Web browser. Can you provide some insight into that?

DR. HORRI: You can e-mail DICOM objects. There is a DICOM mind type now so you can actually e-mail DICOM objects. There are several vendors who have taken the DICOM standard and they've sniffed out the communications layers. In that sense, they are not fully DICOM conformant, but they take the information model.

They take the DICOM information piece and right now they're proprietary codings, but they basically have implemented wavelet based progressive transmission mechanisms to allow rapid sending of images across the Internet for Web viewers and there are a number of commercial vendors of products like that and they are proving at a lot of institutions to be a very favored way of getting images distributed widely across a campus.

Security issues aside, once you can set up your fire walls appropriately. There are a number of vendors that supply product like that. They are very effective. They start out with DICOM, but they have their own transfer syntaxes or communications protocols. They don't use the TCPIP ones or they use encodings that are different from the DICOM encodings.

They take the information model. They can take a DICOM image coming out of a piece of equipment. They just take off the bitmap and all the patient information, encode that in a non-DICOM way for transmission across the Internet. It does work, but right now, you can't call it DICOM conformant.

DR. ZUBELDIA: Following up on that, you mentioned if you can figure out the bitmaps. How much of a problem is that, the fact that each vendor may have a proprietary bitmap? I assume from what you're saying that it is absolutely no problem in radiology because you all understand everybody else's bitmap. But if you move to clinicians, how difficult is it to represent the DICOM images quality?

DR. HORRI: One of the problems that we have is we move away from film. One advantage that film has is you acquire your image on it. Once it's on there, it's the image that you've got. No matter who looks at it, your only variable is the individual's perception and whatever quality of light box you're using and those of you who work in medical facilities have seen physicians in the hallways holding films up to lights in the ceiling or windows or things like.

But one of the things we discovered is that suppose we do distribute images by the Web. I have a widespread PACS. How do I know that the image that's out there, that is the same as what I'm seeing and particularly if clinical decisions are being made on those images, whether my report goes with it or not, some clinician in the ICU going to look at the film and say, the radiologist says that there's a pneumonia here.

They look at the film and say, well, we got to start treating this. How do I know that the image they see is as good as the one I see on film. I didn't have that problem. My assumption was the film is the film. The image is on it. It's fixed on there, but with different display devices, it could be anything from a relatively inexpensive personal computer up to a high-end workstation with a high-resolution display.

DICOM has considered that and there are two things that they've done to help alleviate that problem. One is a thing called a grey scale display function standard that allows you to calibrate your displays basically using the visual mathematical model.

You can get two different widely different display devices that will display the DICOM images at least in the same perceptual way that your perceptual response to that image will be the same no matter which device you look at it on.

The second thing they've done is something called presentation state context. A radiologist looks at an image and I have it available on a workstation a number of controls for that image. There are a number of image processing steps that I can do adjusting the grey scale and so on.

What DICOM will have in it, it's not been widely implemented yet, but it has a feature that allows you to capture what it is the radiologist did to that image so that when it's displayed for the referring physician, it can be displayed exactly in the way the radiologist looked at it, including any annotations and things they put on there.

There are ways of trying to solve this problem of different display devices in different lighting environments. There are a number of things we're all concerned about, but the first part of the DICOMs that did not specify hardware was the display function standard.

That was first demonstrated at a radiology meeting in Europe and subsequently at the RSNA. It's been shown to work that you can get printers and CRTs of various kinds, flat panel displays all perceptually looking the same.

DR. FITZMAURICE: I'm very much impressed wit the grasp of the detail and the view of the overall picture that all of you have. I want to ask a question that probes in both of those areas. Are there gaps in standards for the patient medical record information that exists now? For example, Virginia U had to pull together different things.

Dennis, you had to pull a lot of different things. Steve, you're looking at how DICOM and images relate to all these other different things. Are there gaps in patient medical record information standards that exist? What would you like to see the government do about it, keep hands off, participate in the development, fund something? It's a question of what gaps are there and what's the government do?

MR. TSIOLIS: There must be some gaps. That's why we're here.

DR. FITZMAURICE: For example, can you code radiology results reporting? That may be one of the gaps.

MR. TSIOLIS: Could you elaborate on this a little more? What exactly do you mean code radiology labs??

DR. FITZMAURICE: In some part of clinical medicine produces information. You need to get that information in a uniform way over to somebody else who is looking at it. If two physicians stood side by side or face to face, in 20 seconds, they could communicate most of the important information about a patient.

If they're 2,000 miles away and they have to write it out, the communication becomes something different. I'm asking you are there areas where this information is being generated and needs to be in another place and yet there are no good standards for communicating that information?

MR. TSIOLIS: For clinical data at least, I believe the current HL7 standard and the v3 have a very good way of getting the data from one place to the other. I don't see any gaps in information that exists on the standard.

The problems that we have seen is the ambiguity and it takes time to communicate and make sure we're both saying the same thing about communicating. Once this is done, we will never want bad or missed data to go from one vendor to the other because there are medical decisions made. No, I don't see anything missing.

DR. FITZMAURICE: Is there something the government can do to make the vocabulary the same in one place or another or to encourage common medical terminology?

MR. TSIOLIS: That's one of the efforts in my understanding of v3. There is a vocabulary. It's part of the whole model. Where there is coded entry in there should be part of the vocabulary and we don't really have the option of picking whatever we want. It's going to be there and be clear. When we talk for a specific code, we all talk the same thing.

MS. LORENZI: I agree with a lot what Thanos said. It's amazing the vocabulary group in HL7 is a group of fanatic technologists that have produced so much in a couple of years. It's very impressive and vocabulary is one of our weaknesses. Once you see a code table, you know you're in trouble.

I believe there are functional holes in HL7 that we haven't seen yet. In the middle of the night I was thinking we could code or fix for the ghost in the ICU problem. HL7 always comes up with new and better ways of doing things.

The ways you could help me is I would love are vendors to give us a little more information on conformance. If I had to two conformance statements like you said in DICOM that I could put side by side and say it's not going to work and do that in less than a month or something, that would be good.

If we had some sort of conference, they have HL7 vendor conference, but they don't attract a lot of vendors to that. We get a lot of people who go and do some demo, but if you actually had some conference where people were shamed into it, that would be great.

I'm hoping that part of v3, part of the answer I skirted before is some of the big vendors will go ahead and take responsibility and put out v3 and then we're going to have some new vendors that are going to just do v3 and they're going to have some nice niche systems and hoping the combination of the two of them will make v3 be the main standard in the industry and start killing of v2.

DR. HORRI: I think there's a wealth of data and information out there. I think there are plenty of standards around. I don't think the problem is gaps in standards. I think the problem is gaps and how you integrate those things. I've got imaging information and we're just now beginning to get to the stage where I can integrate that with stuff from the healthcare information systems and radiology information system side of the world both ways.

For example, some of the new work that DICOM and IHE have done with the HL7 group is I get a notification from the radiology information system that the patient is registered, I can then pre-fetch that data, load that data into my imaging equipment.

The technologist does the exam. When that examination is done, a message goes back to the information system that says this examination is completed. So now the family can be told, yes, your father is back on the way up from radiology.

That's radiology and the information systems associated with radiology. I think, overall, if we're going to have patient medical record systems, you need an integration effort like that that spans all of those information sources that we deal with.

How the government can influence that, I guess that forums like this are certainly useful and some incentives in term of research funding and so on to support efforts that look at that sort of thing.

DR. COHN: I want to thank our panelists. It's been a tremendous panel. You've reconfirmed a lot of our initial thoughts. HL7 must dream of Steve's vision of taking a half an hour to plunk things in and turn them on and run. Obviously we have ways to go for that.

The issue of conformance testing, the directions that you pointed out are very useful. I also want to thank you for directing us towards the IHE effort and the HL7/DICOM working group effort which I think is one we'll be following up on.

What we heard yesterday was that the standards area is very crowded and as much as there are holes in standards around vocabulary, there are concerns about things not integrating well around the edges. I think that's an area that we will need to investigate in December.

As I said, I want to thank all of you. I think it's been a tremendous panel. I want to take a quick break at this point knowing that at some point, we may have a fire alarm. Thankfully, we made it thought this panel without one. We'll take a vie-minute break and then move on to our next issue. Once again, thank you all very much for a wonderful panel.

(Short break.)

DR. COHN: We'll have an overview by Jim Scanlon of the Assistant Secretary's Office for Planing and Evaluation on previous recommendations regarding the PMRI standards and the status. We serve under this threat of fire alarm. Depending on where we are, if there is, indeed, a fire alarm in the midst of all this, we may decide to adjourn the meeting. Hopefully, we'll be able to continue this on and we'll keep our fingers crossed if there is a fire alarm, it happens late in this presentation.

Agenda Item: HHS Agency Responses to Recommendations of NCVHS's PMRI Report

MR. SCANLON: Thanks, Simon. Good morning, everyone. This is Fire Prevention Week. So we really have a fire drill on the Wednesday morning of Fire Prevention Week. It's a secret drill just so everyone is prepared.

Let me bring everyone up to date on the Departmental activities related to the NCVHS report on PMRI standards. I'm going to give you a little bit of background of the report and the time line and go through some of the activities, but you have the document in your packet that describes the detail and it probably was not fruitful for the full group to go through all the detail.

I'll characterize some of the activities and answer questions about where some of the activities are. I think in our conference call with the subcommittee on September 25th. We probably made it through halfway through 3 or 4. We'll pick up there, but let me give you an overview just to remind everybody of the overall process.

You'll remember that HIPAA has a provision that requires the Committee to study the issues related to the adoption of uniform data standards for patient medical record information and the electronic exchange of such information and report to the Secretary of HHS by April 2000 on recommendations and legislative proposals for such standards.

The Committee did, indeed, submit a report in April 2000. That report was presented by Jeff to the full HHS Data Council and it was circulated to everyone in HHS at that time. Everyone was asked to review the report and identify any issues or concerns and describe any activities the agency was taking or planning to take to address the recommendations and the basic framework.

In a way it's viewed as a limiteing factor so that the PMRI standards would have to go in hand in hand with progress on privacy and confidentiality. We have asked the Agency a couple of times since then to update their activities relating to the PMRI standards and the report you have before you is up to date as of September 20th.

We can conclude basically that there are actually a considerable number activities relating to the PMRI standards underway. You'll see from the report that the nature of the recommendations were characterized. There was one set that laid out a process for the Committee to identify and recommend PMRI standards to the Department.

There were corollary recommendations relating to what should be done in connection with those recommendations relating to funding, development, promulgation and so on. The Department and the Data Council is receptive to when the recommendations would come in. The Council designated the Data Standard Committee to serve as the focal point when the recommendations from the Committee are submitted.

That will be the focal point for review. Now, it's possible that the framework for PMRI standards that HHS might consider would not necessarily take the same path as the regulatory framework for HIPAA. There may be other ways. You talked about some of these this morning about one way or moving forward is through a regulatory framework like HIPAA.

Another way is through incentives. There are a number of other steps that federal agencies and others could take less than a national regulation like HIPAA. It may be so that with the PMRI standards, it may be a better way. We just don't know.

We'll have to see what the nature of the recommendation is, what the particular standard is and how it might be best implemented. For that whole set of recommendations, it's probably the first 2-3 recommendations and then throughout the report the Department is quite receptive to recommendations for PMRI standards.

The nature of the actual new assessment and implementation would depend on what exactly the standard was and what kind of implementation you were recommending and we were considering. There are a couple of broader issues I want to bring up in terms of the overall framework for PMRI standards in HHS.

I think it's fair to say that in the IT area, the greatest priority currently probably is on basically internal improvements in IT within HHS and within the federal government. It's basically relating to things like standards, interoperability, security.

We have an HHS initiative called Enterprise Information, Infrastructure Management which is run by our CIO, the focus on e-government and other federal government initiatives in IT.

These have an internal focus although there is an external relationship as well with HHS partners. I think clearly, that's a big part of the efforts within HHS and a big part of the spending within HHS.

The second major initiative in this area really is HIPAA. I think in HHS, HIPAA is viewed as the framework though not necessarily a regulatory framework, but it's used as the framework from which improvements in PMRI standards and related NHII activities will take place.

To the extant that HIPAA is successful, that all goes well for progress in these area to the extent that there are problems with HIPAA, delays in HIPAA, other problems with HIPAA, I think it augurs less well for advances here as well.

In addition, within HHS, HIPAA is viewed as the major initiative leading to PMRI standards and NHII progress as well. In fact, we are beginning to have our agencies, when they think of new data systems communication systems, new standards and vocabularies and so on, I think we really are having them look at the HIPAA framework before they look anywhere else.

We have a couple of examples. My own office has a project with CDC where we're looking at the utility of the standard clinical vocabularies for infectious disease reporting in public health surveillance. We have our patient safety test activities, we have a group looking at the existing state reporting systems for patient safety occurrences.

Again, we have the HIPAA as well in national data standards architecture and a look at existing standards rather than the development of new ones. We have a similar activity underway in long term care data area where rather than how can you bring the various requirements for information in the long term care area into a more integrated framework.

Again, it's the HIPAA framework. Within HHS, there is a look at this type of framework for future work and again a look at existing standards or developing standards rather than creating new ones as a future oriented framework.

There are always a number of activities. The National Library of Medicine has always supported research in informatics and applications. CDC has supported public health informatics training and you probably know about the network and the health awareness network which were invoked during September 11 and since then.

NIH, AHRQ have long traditions of supporting informatics research. Telemedicine has research efforts. These are the mission-oriented agencies of HHS. In terms of policy, work continues on the security regulation, on privacy, modifications to the privacy regulation.

In terms of the recommendations and the PMRI report to deal with national policy such as security and assuring confidentiality and so on, HHS is in the middle of this as well. Some of us were puzzled in the informatics community, a distinction between what was HIPAA and what was PMRI and what NHII because we tended to view one as a step leading to the effort.

When people would say that HHS isn't doing anything related to PMRI or NHII, it was a little puzzling to us. Now I can see that people are beginning to see the HIPAA framework as the platform and I use that in a very loose sense. If we can make progress there, that is a positive effort. If we can't, it's not a good predictive other than the way it works now.

With that kind of a background, are there any questions and then let me pick up on some of the specific actions in the report. There were 10 recommendations and each had 10 subparts. At our conference call on the 25th, we got to page 3(c)(4) on page 8 in your handout.

I think it's fruitful to read every one of these, but let me give you a flavor. Recommendation 3 dealt with provide immediate funding to accelerate the development and promote early adoption of PMRI standards. This should take the form of support and there are a number of recommendations that we went through.

I venture to say there is some HHS activity underway in virtually every one of those area. Some of the areas we really raised some legal questions about what exactly a federal can or can't do. There we just have to have our lawyers looking at things.

When we arrive here at 3(c)(4), it's to promote development and testing in appropriate multi-agency projects such as the GCPR project. Here HHS, the Indian Health Service and other parts of HHS are working on the GCPR project with the VA.

The emphasis here is on multi-agency projects. I'm thinking that probably the GCPr is the major inter-agency project of this nature currently within the federal government. The other project is the US Health Information Knowledge Base. Our agencies within HHS are participating and supporting the US Health Information Knowledge Base.

The 3D Woman deals with the recommendation relating to the support for the development and maintenance of an open medidata registry. Since then, the precis has begun for the US Health Information Knowledge Base. If the Committee feels that there are additional things here, we would welcome them, but that seems to be the large medidata registry project currently.

FDA was the subject of a number of the recommendations in the PMRI report. They provided some updated information here as well. They have been involved in international harmonization efforts and standards. Sometimes they don't arrive at the same place as the domestic standards. I think there some harmonization needed between the two. This may be an issue for standards as well.

I don't know that I'm going to go through each of these. You can see from the list that in each one of the subrecommendations, there is some activity. I reported already on the assessment we have underway relating to the appropriateness of the standard clinical terminologies, IDC, MEDRS(?), SNOMED for our infectious disease surveillance. Chris Chute is doing that project. We may have Chris make a report to the Committee.

The next set of recommendations relates to FDA making publicly available its NDC database registry information. It looks like FDA is actually beginning to do this, but from the recommendations we cite, it looks like there is a way to go. They're working with NLM and with our data standards committee. They're now moving forward on these recommendations as well.

The whole set of recommendations within 3(f), I think I'll just have you ask question if you have these. These are very specific. FDA is currently developing standards for safety data collected during clinical trials. We'll look at the HIPAA standards and PMRI standards.

They're working with our data standards committee within HHS in this regard, but there are a lot of very specific activities like that. CDC relates two examples of where they're looking at PMRI standards for some of the work in their mission and it includes the emergency department called DEEDP where they are looking at HL7 and standards there as well.

They're looking at the basic HIPAA framework and through the NEDS, there's been considerable growth on that. All 50 states have received some funding for NEDS and planning efforts and implementation efforts. So it seems to be moving well along. I think it's been treated favorably in the upcoming budgets as well.

Some reports from IHS. They're implementing HL7 standards in their clinical information systems and their administrative information systems. They're part of the GCPR initiative. We can probably arrange a briefing on it at a later date rather than go through the actions here.

Recommendation 4 relates to the standards that would be recommended by the Committee and for each standard that the Committee would recommend, the recommendation to submit funding for a list of activities and I think our basic position on HHS was that as the recommendations are actually considered, the appropriate way of funding and support and implementation would be considered.

It relates to a series of other activities that would support a standard like development of an implementation guide, development of conformance testing procedures, licensure or other arrangements for those standards to make them available for use at little or no cost. That whole set of recommendations relating to 4 would await the recommendations from the NCVHS.

Five relates to support demonstrations of the benefits and measurement of the cost of using uniform data standards for PMRI and then there are areas suggested which include the ability of clinicians to go from patient's clinical performance measurement, use of practice guidelines, reduction in medical adverse events and public health surveillance and intervention.

Here, we have begun to overall support some literature reviews of the benefits and costs of healthcare and public health. We actually had an informatics intern this summer and we had her try to compile a literature review on systematic studies of the benefits and costs of EDI in healthcare setting.

There was less literature than we thought. There are certainly testimonials and that sort of thing, but in terms of actually demonstrating systematically, there wasn't a whole lot of literature, but we started along and we did identify some literature and we may support an update of the cost benefit analysis as well.

Six relates to supporting increases in funding for research demonstration and evaluation studies. Here with CDC, AHRQ and NOM, we have funded basic and applied research in health informatics and the areas identified. CDC was planning to increase the public health informatics, investigator initiated research program. These things depend on funding.

DR. YASNOFF: If funding is available, then that will be done. I think what's reported here is accurate that there is a plan to do that, but there is no funding at the present time.

MR. SCANLON: I should mention that federal agencies are now in FY02 as of October 1st and we are funded on a continuing resolution. No agency is going to make a big commitment at this point. Work is underway on the FY03 budget which is now with OMB and the President will submit that later in the winter.

I would have prefaced which and this is the danger for an agency coming forward with plans. All of these are premised on the idea that funding would be available. It's the nature of planning.

FDA has begun to look at its adverse event reporting system. These are largely the after post market surveillance where manufacturers, doctors, and providers are supposed to report problems with medical devices, medicines and so on.

FDA has begun to look at that and they're coming forward with a fairly large proposal for revamping those systems which you'll see described here. This is dependent on budget resources as well, an attempt to integrate the biological with the device report and upgrade this tremendously.

Health Alert Network reports to the state health directors and NEDS. You have a pretty good idea there and all 50 states have gotten some funding for NEDS and additional grants were award to 35 states to develop systems. There are some examples from HCFA and SMS relating to these areas as well. From AHRQ, there was some work.

Let me go to 9 and 10 and then we can discuss this more, 9 relates to the equitable distribution of the cost for using PMRI standards among all major beneficiaries. This may take the form of incentives for sufficient data using standards including quality improvement.

Again, here incentives may be a way as opposed to regulations that we may want to consider in the next generation of PMRI standards, but again, we'll have to see what the nature of the standards might be.

Ten deals with encouraging the policy level and legislation for electronic PMRI including the following. The first is comprehensive federal privacy and confidentiality legislation. Here there is no other legislation that I am aware of. We have the HIPAA privacy and confidentiality regulation undergoing some --, but we have some framework in place.

In terms of security, we have the proposed rule and we will have the final rule later this year or next year, but all the major policy issues have been resolved wit the security rule. Let me stop there. I don't want to read the actions. You can do that better than I.

I think the basic flavor is that for the actions recommended by the Committee there is actually an activity underway. Some depend on funding and some depend on authorization and a bunch of them depend on recommendations that we would want to send to HHS. The nature and form of the support activities would depend on them. The framework of HIPAA is serving as a framework for a lot of the data activity within HHS and within our partners though not in a regulatory framework.

DR. COHN: I guess we should add that we have received a letter from Claude Allen in response to a letter from the Committee in June that indicates that the security and the attachments he expects to have out by the end of this year. The national provider identifier will likely not be released until FY03.

MR. SCANLON: Progress. Our Deputy Secretary has responded to two earlier letters and brings the Committee up to date on where things are and shows his commitment as well. I think on the more immediate standards, I think the change in the transaction, there are a couple of modifications to the transaction recommendations that the Committee recommended.

Those are aimed for later this year. The change in the requirement for what code to use in the pharmacies and the fast track set of changes. Those were projected for the end of this year. Security is working its way through.

On the provider ID, there is more of an issue of not being in a position to issue s regulation until we have the wherewithal to assign or arrange for the provider ID. We're in the middle of the budget process now. Depending on how the '03 budget works out, we may actual have a way to proceed.

I think the problem there is it's budget. In a way, HIPAA set up something almost akin to an FDA responsibility for standards without really any money or staff and unfortunately everyone has done a marvelous job of getting everything to here, but for certain things there is no alternative but to do it right. We couldn't issue a reg requiring that something be done when we don't have the ability to make it happen either.

DR. YASNOFF: Would it be possible to take this list and put it on the Web and keep it up to date?

MR. SCANLON: We might be able to do that. A number of these are plans. We'd have to caveat saying that subject to funding. Other than that, I don't think there's anything here that would be an issue.

DR. YASNOFF: I think it would be useful to show how the Department is working on these things and also to keep this information up to date. It's a single snapshot and these activities are all changing regularly and if all the folks who submitted this information would take the responsibility to keep it up to date, then we can keep track of how the Department is responding.

There is one thing that's missing in here that I think might be of interest to the Committee, the effort at the National Cancer Institute to build the cancer information infrastructure. They have good evidence now that some years of work in this area are producing real benefits in terms of lower costs for collecting and exchanging patient medical record information. One of the things we might want to consider is to invite John Silver to come talk to us about that cancer information infrastructure.

MR. BLAIR: Bill, are you referring to the limited license CDC got for SNOMED cancer registries across the nation?

MR. SCANLON: No, this is work that's been done in the National Cancer Institute to build what's been called a cancer information infrastructure that allows for electronic collection of clinical trial information. It's been several years in development and now being used. They have some very impressive cost benefit data that would be useful to us in terms of a real application of what you might call a subset of national health information infrastructure and how standards are important and vocabulary is important the problems that he's had.

DR. COHN: Comments or questions?

MR. BLAIR: I would just like to wind up giving my sincere thank you to Jim Scanlon for pulling together this information from all the HHS departments and agencies on the actions they've been taking on the NCVHS recommendations on PMRI standards.

It's very gratifying as you look through the report. We do understand that many of these things are subject to funding. As we look through, we understand that some of these things are projects or plans that had been independently projected by departments and agencies.

Even though you wind up looking through this and you see a mixed bag like you do with feedback, overall the impression that you get is that HHS has really taken the recommendations very seriously and given the great support and I would like to express my appreciation for the effort that's been made.

MR. SCANLON: Thank you, Jeff, and let me express my appreciation too. Susan Hawk who was in our office who actually did a lot of work and pestering the agencies to bring this up to date. Our plans are to update this quarterly and we'll see if we can disseminate it more broadly and have it as a working document in progress. Thank you very much.

DR. COHN: I also want to share Jeff's admiration and pleasure with seeing this document. I think as we reflect on it, certainly it reminds us that we can weave a lot of what our original recommendations are into the next document. For example, recommendations around conformance testing and such.

They're not actionable until we actually have standards that we can propose that there be implementation guides and conformance around. This will fit nicely with the next phase. I was reflecting about the work that's been going on over the also year and year and a half and how the work we've done on the HIPAA standards and the fact that this seems to be moving forward. Not that we are responsible for that, but I thin we can be thankful that it is working well.

MR. SCANLON: The Committee provided the impetus for looking at this and in a future oriented way. It's a way of having the agencies work together towards the future.

Agenda Item: Discussion - Next Steps

DR. COHN: Are there other comments on this before we wrap up. I want to think the subcommittee and staff for their support and activities. The next meeting will be a brief one at the meeting of the full national committee on November 15 and 16. It's likely that on the morning of the 16rh, we'll have a brief meeting of the subcommittee at that point.

The next meeting of the subcommittee are on December 13 and 14. It's currently planned that the first day will be devoted to issues of PMRI. Hopefully, the next morning until we adjourn on the 14th will be devoted to where we are with the overall administrative simplification, trying to collate information on electronic signature work, implementation based on views of those that have been looking at the overall issue. Preparation for our annual report to Congress will have to be started in the end of this year. There is a lot that's been happening this year. Questions or comments?

MS. GREENBERG: I probably should just mention that it's not finalized, but the workgroup on quality is trying to organize a session related to patient safety issues and they're trying to organize it for the afternoon of December 12. I informed them that 1-3 conflicts with the Data Council which impacts some staff. The idea is to coordinate it with your 13-14 meeting if people on the workgroup want to come in for that session. Everyone on the subcommittee for population is also invited.

DR. COHN: There has been an 3-mail querying for dates. I didn't realize it had become definite.

MS. GREENBERG: Staff are working on lining up speakers.

DR. ZUBELDIA: Is this going to be a conference call?

MS. GREENBERG: This is actually a meeting here at the Humphrey Building on the afternoon of the 12th.

MR. BLAIR: We did have on our agenda that we would take a look at some of the PMRI recommendations that Margaret Omtiokin(?) had given us back in August to follow up with SDOs for verification and clarification. With your permission, we deferred that to this morning. We'd like to be able to close so that a number of folks wind up getting some travel arrangement needs met.

If it's okay with the rest of the subcommittee, I will just send an e-mail to subcommittee members for your comments and suggestions with respect to some of these areas of clarification of the report that Margaret gave to us. Is that okay?

DR. COHN: That's fine. With that, I want to thank you all for a day and a half of very useful panel hearings and sessions. We will adjourn this meeting.

(The meeting was adjourned at 11:15am.)