Protecting People and the EnvironmentUNITED STATES NUCLEAR REGULATORY COMMISSION
UNITED STATES OF AMERICA
NUCLEAR REGULATORY COMMISSION
ADVISORY COMMITTEE ON REACTOR SAFEGUARDS
***
MEETING: PLANT OPERATIONS AND RELIABILITY AND
PROBABILISTIC RISK ASSESSMENT
***
U.S. Nuclear Regulatory Commission
11545 Rockville Pike
Room T-2B3
Rockville, Maryland
Tuesday, January 26, 1999
The subcommittee met, pursuant to notice, at 8:30 a.m.
MEMBERS PRESENT:
GEORGE APOSTOLAKIS, Chairman, ACRS
JOHN BARTON, Member, ACRS
MARIO FONTANA, Member, ACRS
THOMAS KRESS, Member, ACRS
DON MILLER, Member, ACRS
DANA POWERS, Member, ACRS
ROBERT SEALE, Member, ACRS
WILLIAM SHACK, Member, ACRS
GRAHAM WALLIS, Member, ACRS. P R O C E E D I N G S
[8:30 a.m.]
DR. BARTON: The meeting will now come to order. This is a
joint meeting of the ACRS Subcommittee on Plant Operations and
Reliability and Probabilistic Risk Assessment. I am John Barton,
chairman of the Subcommittee on Plant Operations. Dr. George
Apostolakis is chairman of the Subcommittee on Reliability and PRA.
ACRS Members in attendance are Mario Fontana, Thomas Kress,
Don Miller, Dana Powers, Robert Seale, William Shack, Robert Urich, and
Graham Wallis.
Also in attendance is Mario Bonaca, who has been selected as
a new Member. He is expected to be appointed to the ACRS in the near
future.
The purpose of this meeting is to continue the subcommittee
reviews of proposed improvements to the NRC inspection and assessment
programs, including initiatives related to development of a risk-based
inspection program and performance indicators. Subcommittees will
gather information, analyze relevant issues and facts, and formulate
proposed positions and actions as appropriate for deliberations by the
full Committee.
Michael T. Markley is the cognizant ACRS staff engineer for
this meeting.
The rules for participation in today's meeting have been
announced as part of the notice of this meeting previously published in
the Federal Register on December 21, 1998. A transcript of the meeting
is being kept and will be made available as stated in the Federal
Register notice.
It is requested that speakers first identify themselves and
speak with sufficient clarity and volume so that they can be readily
heard.
We have received no written comments or requests for time to
make oral statements from members of the public.
I note that we've got Bruce Mallett on video from Atlanta.
Bruce, can you hear us?
MR. MALLETT: I can hear you fine. Can you hear me?
DR. BARTON: Yes, we can. And I also understand that you
have another commitment, and we lose you at what time?
MR. MALLETT: I have to be out of here by ten o'clock.
DR. BARTON: All right. So at this point we'll proceed with
the meeting, call upon Frank Gillespie, Mark Cunningham -- Mark's not
here -- who's got the lead here this morning?
MR. JOHNSON: Frank.
MR. GILLESPIE: I do.
DR. BARTON: All right. Do you intend, since we're going to
lose Bruce, we're going to kind of get his inspection stuff tied in
early?
MR. GILLESPIE: Yes.
DR. BARTON: Since we did meet in December, and sent a
letter up, the things that we were interested in that you were still
working on at that point I believe was the work on the assessment
process. So if we can spend some time on that today, the transition
plan, and hopefully can answer some questions on the package that you're
asking the Commission to proceed with has got some, you know, future
work needed, some holes in it, how you plan to address those issues.
We'd be interested in that also.
So at this point, Frank, you've got the show.
MR. GILLESPIE: In order to get Bruce on pretty quickly, and
we're not going to repeat everything we said the last time, we've got
that fat package out, and it pretty much lays out all the details. So
rather than go through all the viewgraphs we have here, which are kind
of a subset of the Commission viewgraphs we use, there is one on the
concepts. We're going to I think just get right into it.
Our one big hole we identified with the Commission just to
highlight it, the Commission said that they're interested in, is a
risk-informed screening measure for dealing with inspection results that
aren't really conducive to quantification. And we're still working
that, and Alan Madison's going to be here a little bit later to go over
the details of that.
DR. BARTON: All right.
MR. GILLESPIE: And that was probably the one biggest policy
hole that we had I think when we addressed this with the Commission that
they were looking forward to seeing in March. And probably one of the
more -- I don't want to say intellectually, academically challenging
things to do is to come up with something that the inspector in the
field, who's basically only had a two-week risk course, can apply and
will get a meaningful screening process out of. So Alan will come.
We've made some progress on that. And he'll be explaining that a little
later.
So with that, rather than go through all of these, the
schedule is there, our transition plan is there, we'll hit that at the
end. We're on kind of a tight schedule in that we're going to try to
have eight pilot plants started in June.
DR. BARTON: These are six-month pilots?
MR. GILLESPIE: Yes, we're going for six-month pilots. We
feel we need at least a half a year to just exercise the system and work
the bugs out of it, that just doing a three-month pilot isn't going to
give us a chance really to observe it.
DR. POWERS: I guess the question that came to my mind was
the opposite. Why isn't it a full cycle, a full-year pilot?
MR. GILLESPIE: There's a certain amount of impatience on
people's parts --
DR. POWERS: I understand that.
MR. GILLESPIE: To get on with it.
DR. POWERS: But it seems to me you don't get through the
whole shooting match in one consistent pilot by going with six months.
You're guessing a lot.
MR. GILLESPIE: Probably the -- it's not guessing. Six
months actually gets us through the basic inspection cycles, gets us
through a planning cycle, and because the reporting is going to be on a
quarterly basis with a one-year running average, once you've reported
one or two quarters' worth of material, you've basically exercised it
twice. The first time you've made a lot of mistakes, the second time
hopefully you've made fewer, and now you can kind of clean it up. So we
would have exercised reporting through two cycles. We would have
exercised inspection planning.
What we are likely going to be missing will be those things
that might only be done once a year or once every two years, like the
engineering inspections, but the engineering inspection is, quite
honestly, because there's no PIs in engineering, is fundamentally what
we're doing today, because we're picking high-risk systems. So what
we're not testing in the six-month period is actually the things that
might be very similar to what we're already doing.
With that, I'm going to turn it over to Bruce right away,
because he's got an interesting section and one that's got a lot of
implementation questions still left on how we're going to actually
organize it to get it done. So Bruce --
MR. MALLETT: Thank you, Frank. Mike, were you able to
receive the two slides that I sent?
MR. BARANOWSKY: No, I did not yet. I'll get someone to run
for them.
MR. MALLETT: Mike Johnson.
MR. JOHNSON: Yes, Bruce.
DR. BARTON: George, do you have a question at this point?
DR. APOSTOLAKIS: Yes.
DR. BARTON: Bruce, can you hold a second? We've got a
question here at the table.
MR. MALLETT: All right.
DR. APOSTOLAKIS: Well, I want to ask, as I was reading this
report, the question of the objectives of the whole exercise came to
mind, and I'm wondering when it would be a good time to raise that
question. Shall we let Bruce go on first, or --
MR. GILLESPIE: No, I think that's probably a good time,
because there's a good lead-in to one of the criticisms we got of the
inspection section was it did not -- and, Bruce, chime in if I get this
wrong, but it did not clearly state the objectives we were trying to
meet when you read the inspection section by itself. So, no, this is a
good time to ask --
DR. APOSTOLAKIS: Okay. Let me tell you then what the
concern is. On page 2 of the document it says this paper presents
recommendations for improving the NRC's reactor oversight processes
including inspection, assessment, and enforcement, and includes a
transition plan.
Now it doesn't really say what the objective of the
oversight process is, but then on page 3 it says, the first full
paragraph, the objective of the inspection task group was to develop
recommendations for a baseline inspection program blah blah blah blah
blah to determine whether plant performance is at an acceptable level.
Now I am not sure that this is what the objective of an
oversight process should be. I see this as equivalent to what people
call quality control in manufacturing. You have a process which is
acceptable. I mean, the plants have already been licensed, right.
They're operating.
MR. GILLESPIE: Right.
DR. APOSTOLAKIS: It seems to me that the purpose of the
oversight exercise it to make sure that what you have licensed, what you
think you have licensed is indeed the truth.
So, now, then we go to Attachment H. If that premise is
correct, then if I look at the figures H1 through 6, which are pages
H-18 on, these are the actual data that I believe NEI supplied you or
some of it is -- yes, okay. So, for example, figure H1 on page H-18 is
the number of unplanned scrams per year for a number of units. It's not
all the units, right? For a number of units. And the same thing, the
next one is the BWR high-pressure injection system unavailability and so
on. This is real life.
In the text it says that the way to define one of the goals,
I think it's the -- I don't remember the colors now, green to --
MR. GILLESPIE: Green to white.
DR. APOSTOLAKIS: Green to white. Is to consider that
number of unplanned scrams that about 95 percent of the plants meet or
have a smaller number of scrams. And then we make that part of the
assessment process and say this is a threshold.
It seems to me that this runs counter to the notion that the
purpose of the oversight process is to make sure that what the plants
have been doing, what we think they have been doing, is indeed correct.
In other words, you are forcing now this way the five percent of the
plants that are above to actually come below the threshold. In other
words, you are regulating them instead of just overseeing them. Isn't
there a difference?
You are telling the -- in fact, if you go to one of the --
yes, for example, figure H4, BWR RHR system unavailability, the
threshold is set at .015, and there are 1, 2, 3, 4, 5, 6, 7, 8 plants
that in fact have an RHR unavailability greater than .015. So aren't
you sending them a message that they better make sure that their RHR
unavailability comes below .015, which may be a good message, but it's
not the job of the assessment process to do that.
In other words, the way I would do it, I would say okay, Mr.
Plant 70, you have an RHR system unavailability .025, it's the highest,
this is what you have in your IPE, and your whole IPE was accepted by
the staff or it was reviewed by the staff, and you have a license. So
all I have to do now from now on is make sure that you don't become
worse than .025, not that you reduce it down to .015. That's a separate
regulation. The oversight process' job is to make sure that what we
have licensed and the assumptions and all that is still the way we
thought it was. It's quality control, in other words, not regulation.
Which brings up now another. If that is the case, that's
the second point now, the first point is we are in fact intervening with
this process, it's not oversight anymore. The second part I think is
also a conceptual difficulty. Let's say you agree with me. I don't
expect you to agree with me right away, but let's say you agree with me.
DR. SEALE: You're going to give him a little time.
DR. APOSTOLAKIS: So you say okay, fine, so even for this
plant where the RHR unavailability is kind of high, they probably have
other things that are much better on the average, so the total core
damage frequency came out to be some reasonable number. So now we make
sure that the oversight process will monitor the plan to make sure that
the IPE -- the IPE's assumptions and the numbers and so on remain the
way they were submitted.
But the plant has not been licensed on the basis of the IPE.
Right? It's the whole --
MR. GILLESPIE: Right.
DR. APOSTOLAKIS: Deterministic. So can we really do that?
Can we regulate, can we have an oversight process that is based on a
study that is not part of the licensing basis? Is that allowed?
Which then creates the question, should we again go back to
Part 50 and make sure we change things there before we develop this?
MR. GILLESPIE: Two things, George.
DR. APOSTOLAKIS: Yes.
MR. GILLESPIE: Yes, we're out of sync, because inspection
is ahead of Part 50 -- risk-informing Part 50.
DR. APOSTOLAKIS: That's correct; yes.
MR. GILLESPIE: So that's true. The other piece is we don't
have the sophistication to do this plant by plant by plant right away.
The third piece is what's probably missing in clarity, and
we -- I tried to make this clear at the Commission briefing -- is that
this piece and the way these are used and the way they're used in
inspection isn't intended to be a backwards ratchet.
The baseline program is an indication program. Even the
inspection program is an indication program. It is not diagnostic. It
is not intended to force someone to be better than they can be.
DR. APOSTOLAKIS: Aren't we de facto doing this though?
MR. GILLESPIE: Could that happen?
DR. APOSTOLAKIS: Yes.
MR. GILLESPIE: Yes, and if you set up a standard and say,
you know, pass this standard it means be go into more of a diagnostic
mode to understand why and that is the real point here. That is the
objective of the thresholds is if someone breaks the threshold it is to
engage them to ensure that we as regulators understand why you might say
in this case that they are different from the expected norm.
DR. APOSTOLAKIS: So it becomes plant-specific at that
level.
MR. GILLESPIE: And at that point you become plant-specific
and engaged doesn't mean send out a 20-man team. It means first you'd
probably end up asking a question -- why? If there is an explanation,
we understand it -- then there's an explanation. We understand it, and
that is when it becomes plant-specific.
DR. APOSTOLAKIS: I understand that but again there is a
fundamental concern here that -- and there is a third point -- but first
of all, I don't think it's such a monumental job to do it on a
plant-specific basis, because we don't have to do it. We can ask the
utilities to do it. We can give them guidelines how to do it but we
don't have to do it on a plant by plant basis.
For example, if I go to Figure H-2, look at Plant Number 62
on page H-19. It is a BWR. High pressure injection system
unavailability is the lowest of all the plants that are listed here.
It's less than .01 and you have already set the threshold at .04, right?
It's really low. Now that plant probably wants to get some credit for
it. Maybe somewhere else -- they are not named, right? -- somewhere
else they have an unavailability for something else that is higher than
the threshold, so they may come back or they should come back and say,
look, on balance our risk is acceptable and so on and so on, but if we
look at each one of these individually, which is the third point I
wanted to make and I forgot, we are regulating -- I mean the idea of the
cornerstones was not really to continue to be as intrusive as we have
been.
We don't want to regulate initiating events and the pressure
boundary and inspect the same way we were doing it before and I think
that is the path that this is going down, because now we have thresholds
for initiating events, thresholds for unavailabilities of various
systems, and so on, so it will be as intrusive as the regulatory system
and inspection process, as it was.
How can we give them more flexibility, which is the whole
idea of performance-based regulation? Remember, these plants have
already been licensed. Even if the core damage frequency is greater
than the goal, there's nothing we can do about it, unless it is 'way up
there, which is it not. Right? Because that was not part of the
licensing basis.
So can we relax a little bit this and instead of looking at
the systems and the initiating event frequencies and so on propose a
methodology perhaps that the utilities can implement for them to come up
with individual -- not goals. Thresholds? I don't know -- using some
sort of systematic approach.
MR. GILLESPIE: What we have generated here is kind of a
generic profile of certain operating parameters --
DR. APOSTOLAKIS: Yes.
MR. GILLESPIE: And could you generate a plant-specific
profile for each site, given site differences and engineering
differences? In the future the answer is probably yes, you could.
We started from the general and now we are going to the
specific.
DR. APOSTOLAKIS: I understand.
MR. GILLESPIE: And right now we are at the generic. I
don't disagree as a future goal it would be very nice to have a
plant-specific profile but we don't have it --
DR. APOSTOLAKIS: But aren't we over-stepping the bounds
though of the oversight process -- telling those 5 percent of the plants
that they better shape up.
MR. GILLESPIE: We are not saying that.
DR. BARTON: George, I don't think that is what they are
trying to do here. I think they are looking for trends. They are
looking for plants that all of a sudden are above a norm -- and the
industry is using one of the performance indicators which looks an awful
lot like this kind of stuff so the industry is already familiar with
this.
I think what they are looking for is a plant that is below
this .04 and all of a sudden it shoots up to, you know, .07, and you
say, well, why?
DR. APOSTOLAKIS: If I owned a plant that already has at BWR
HP injection system unavailability of .07 -- Plant 75 -- and the NRC as
part of its oversight process says the threshold is .04, what do I do?
MR. GILLESPIE: Could this system be abused to that point?
I guess the answer would have to be yes, but that is clearly not the
intention. The intention is it's an indication system. At that point
as the regulator we would want to understand why they are different and
what offsets it. It is not clear.
Now maybe if we had a detailed, plant-specific profile that
was completely justified, it would be clear, but we don't have it and
right now the industry hasn't really proposed to do a plant-specific
kind of profile.
MR. MALLETT: But Frank, let's be careful here. Let's make
sure we understand the process here is that we have a band of operation
for plants and those plants in that band of operation might be, as
George said, from .04 to .07 unavailability of that system and therefore
when you look at them you may need nothing more than a baseline
inspection program if they are in that band of performance.
It is only supposed to be an indicator of whether we would
increase our inspection above a minimum level, so the concept is every
plant will get a baseline minimum inspection and if the indicators show
they are outside those bands or thresholds we would increase at least a
part of that. They could very well operate within that band and we'll
leave them alone.
DR. APOSTOLAKIS: But Bruce, you are bringing up another
issue now. You are saying that what counts is the band not the
threshold, and if I didn't speak to you and listen to you, I wouldn't
know that. The document does not say that. In fact, that may be a way
out since I fully appreciate, by the way, that you had to do it in a
generic way. Perhaps instead of a threshold give a band because what
you don't want to do is introduce new requirements through a document
that is supposed to be an oversight and assessment document.
MR. GILLESPIE: Yes. This is fundamentally an internal tool
for us to know when we have to do and understand more.
DR. BONACA: I would like to make one comment.
MR. GILLESPIE: Go ahead.
DR. BONACA: Simply it seems to me that, yes, I can
understand it is an internal tool but then it introduces again
subjectivity and one goal is go away from subjectivity.
One thing that concerns me, looking at these thresholds --
that is the same point that George is making -- is they may not be
equally achievable for all plants at the same level, but again there is
compensation from other systems that may not -- what I mean is designs
of plants vary very much for the BWR high pressure injection system.
Some plants have just one train but they have other things
that compensate for that, and simply the threshold is not equally
achievable for all plants, but the reason is there are reasons why,
however, core damage frequency is equally low because there are other
systems and so that is what I am trying to understand.
Is that the point you were making, George?
DR. APOSTOLAKIS: Yes. One of the points.
MR. GILLESPIE: We are not disagreeing with that.
DR. BONACA: But my only concern is the issue of
subjectivity, because the question is how is it going to be interpreted
and used in the field.
MR. GILLESPIE: Well, the intention is, just as Bruce said,
that at that point we have to develop a better understanding, so you go
from an indication mode into an understanding or diagnostic mode.
Once we understand it, then it is a done deal.
Also, this is a starting point, not an ending point and here
is something I am kind of cautious about and one of the reasons we went
from generic in that work in the future to specific and that's on the
other side is all those people who talk about the fidelity of individual
PRAs and overdependence on them and how can you have two plants, how can
you have two plants that appear to be virtually identical with two
different PRA results and different sets of important systems, and how
do you deal with that uncertainty.
At this starting point we were not prepared to have
individual plant by plant specific profiles, so this is our starting
point. To do that in the future is a good option, but I can't deal with
it between now and June very well.
Is this a de facto different set of performance standards?
It could be abused to that extent. I can't say it couldn't. It is
certainly not our intent and that is why we have got it out for comment.
DR. APOSTOLAKIS: Well, I appreciate the difficulties you
have. I mean don't think that I don't.
On the other hand, we have to evaluate these things and
scrutinize them because this is the first time that we are really doing
such things in a big way.
First of all, I think that the document, the written
document, should reflect some of the thoughts that you just gave us,
Frank. For example, we had a similar -- not quite the same but similar
discussion in the development of the Regulatory Guide 1.174 whether the
lines were bright to different shades of gray and so on, and what Bruce
said about maybe the band really being more meaningful -- now I don't
know if you already have several bands of regulatory action and now you
are making the boundaries between the bands themselves -- what all that
means -- but also maybe you should give the option to a utility to come
back and argue on the basis of their IPE or PRA that for them BWR high
pressure injection system unavailability of .07 makes sense and they
will maintain that, but they have other things that compensate for that.
Right now, there is no option like that here. It is generic
thresholds and the truth also is that very few people really understand
the subtleties of all this.
I mean you are saying there is a danger for abuse. I think
the probability is pretty high that well-intentioned people will abuse
this because they don't understand the fundamental idea of overseeing
probabilistically -- not that I understand it. It took me a while to
understand this and make the points.
MR. GILLESPIE: Well, one of the criticisms of the package
was that we didn't have a clarity of purpose in there, and that is
something we're going to fix -- the idea that this is strictly an
indication process and which you then go into a process of understanding
when the indication says more understanding is needed.
I can't fix it here today --
DR. APOSTOLAKIS: I know. I am just raising the issue. I
know.
MR. GILLESPIE: The ideal would be plant-specific profiles.
DR. APOSTOLAKIS: Yes.
MR. GILLESPIE: Because just by the nature of the
statistics, we are agreeing that 5 percent on any given indicator could
be outside the band and that's perfectly acceptable.
DR. APOSTOLAKIS: Right.
MR. GILLESPIE: There's tails on every distribution but
being in the tail doesn't mean you are bad. It means we need to
understand why so we know you are not. We do need to put that clarity
in it.
How we get there from here and how fast we can do it I don't
know, and the other piece we need to be concerned with as we need to
part from generic and go to plant-specific is how do we communicate that
to the other stakeholders. We may understand it and the industry with a
PRA group at each facility may understand it, and it's kind of a
compromise in there in a somewhat more simplistic view in trying to have
something that we can also communicate to the public.
There is some simplicity driven into this.
We also came up with another point of confusion which we are
starting to get clarified with people, and that is that there are two
very important matrices, one that Pat is going to go through which apply
to individual indicators and will then -- also a parallel process to
apply to individual inspection results. That is an individual
indicator. It is one out of 20 indicators.
Then there is a matrix that Mike has in the assessment
process which is the matrix that covers expected agency actions or
regulatory response.
Initially when people thought about the concept they talked
about green plants. We said but there's no green plant. There is a
green indicator.
Also clarifying the difference between a threshold and a
band -- well, thresholds establish bands but the context we use it in is
really a licensee response band, regulatory response band, and the
terminology if you eliminate the colors, the terminology in there is
actually focusing on the bands.
Now the interesting point here is that the industry has -- I
don't want to say -- I guess I could say has tentatively agreed that the
licensee response band as a first cut on this is reasonable. They are
not saying it is perfect, but they are saying it is reasonable as a
first start, that people should be able to actually have some things go
wrong at a facility and they should be able to correct them before they
pass out of the band.
And that's a broad expectation of establishing the width of
the band, be it three scrams or the unavailability numbers. What we
don't want is an indicator system that trips on a failure, on a single
failure, or on a single thing. So, 95 percent performance over the last
several years as in industry set, that is kind of how we picked it.
Then looked back and said, well, is this low risk? And the answer to
both of those was it looks achievable, in general, recognizing there is
a 5 percent tail on every indicator, and it is low risk.
I don't know that we can do much more at this point, but
this is a starting point. That's the other emphasis here. This is a
starting point. We have got a list of about six different topics from
research on longer-term on how to mature this and make it better. So --
DR. APOSTOLAKIS: I think that by putting these thoughts in
the document and starting out by saying, you know, ideally, one would
like to have a plant-specific PRA base on this. If a utility wants to
do it, we are open to suggestions, you are very welcome. Now, we have
to do it in a generic way, so we have to select a way for doing it.
Now, one way might be to say, well, I will take the highest
value in every chart and go with that. Well, we don't want to do that
for some reason. So we want to be generic. We have this requirement.
We go with the 95th percentile, but we will -- this will be an
indication, blah-blah, everything you just told us.
MR. GILLESPIE: Yeah.
DR. APOSTOLAKIS: And maybe soften the impact. Now, let me
tell you what really bothers me at a philosophical level without
necessarily attacking this document. We can work on a PRA space and be
as prescriptive as we are now in the deterministic space. If we ever
produce a document that will say, okay, it will be risk-informed, so now
we are going to tell you what a threshold, a regulatory threshold is for
scrams per year, a regulatory threshold is for unavailability of this
system and that system, I would really object to that, because now we
are being very prescriptive in PRA space. It is none of our business.
If the accident sequences have the frequencies that are acceptable, why
should we regulate the initiating events and so on?
So that is what really struck me, and I am bringing it up,
that, perhaps unintentionally, you do some of that here by saying, look,
95 percent is what we are choosing and the other 5 percent -- now, if I
am the owner of a facility, what do I do? I get a signal from the NRC
that my unavailability is not really good, but that is not a job of the
oversight process, that is somebody else's job.
So, now, for oversight purposes, I think it is all right to
look at each cornerstone, perhaps look at other things below the
cornerstones, because you are overseeing now, you are making sure that
what they told you remains that way in the future, so that is not
intrusive in my mind. But if you demand, though, that the frequency be
below a certain level, that is regulating the frequency of initiating
events and that, I don't think we should do.
MR. BARANOWSKY: There is no demand associated with these
green-white thresholds. They are exactly what you said, and, that is,
they are a point of oversight in which we increase our oversight because
we have indication that we should start to look in those areas.
When you get to the point of unacceptable thresholds, where
we talk about talk about taking actions such as shutting down a plant,
that is a different story. Then maybe that is the point where we are
imposing some regulatory action on a licensee. But this is oversight,
minimal oversight when you are below that line, a little more oversight,
which seems to make sense. Where are you going to spend your resources?
Well, where it seems like the performance is not within the norms, and a
few sigma outside of the norms is the point where you say, is that an
outlier?
DR. APOSTOLAKIS: But you see, Pat, that is exactly my
point. That is why I asked what is the objective of this. Aren't you
bringing now another element in the regulations, namely, how does your
plant perform with respect to the others? When you licensed it, did you
do that? No. When you licensed it, you said here is a set of
regulations, if you meet them, you get your license.
Now, we are saying something else. We are saying but if you
happen to be outside the bound of what everybody else has.
MR. BARANOWSKY: That was just done a convenience because --
I think you are right, we would like to have a more plant-specific
approach to this, where some plants are more susceptible to high or low
reliability depending on system design and operating characteristics.
But as a first cut, we said let's just start with this more generic
treatment.
DR. APOSTOLAKIS: I think we need a good discussion up-front
what the purpose is, that you are fully aware of the fact that there are
certain risks by going this way, but there are some benefits as well,
because we can do it is, it is generic and so on, and if somebody wants
to come in here with a plant-specific PRA and argue that their
thresholds should be different, you are willing to listen.
MR. JOHNSON: George, let's not forget also that we do what
this process is talking about trying to do, we do it today. We look at
the performance of plants, we look at it in some areas that we have set
up for ourselves. We make decisions about events as they occur on a
real time basis. We decide whether we need to engage, to do more
inspection based on the instincts of the inspectors. I am being a
little bit cavalier, but we use our judgment, based on our experience,
based on our understanding of what is important from a risk perspective
today to make decisions about whether we engage, about -- the extent to
which we engage. Do we send out individual inspectors? Do we send out
a team of inspectors? So on and so forth.
And what we need to keep in mind with this process that we
are proposing is that we are simply trying to add a structure and a
framework to do that in a way that is a hierarchial approach, as the
ACRS has pointed out, that we need to go on, we think it is a good
direction to go in. That starts with this notion that, given the fact
that we are working towards this mission, this common mission of public
health and safety, and then that we have had cornerstones which say,
hey, given the fact that we have got this robust framework, regulatory
framework, and all the requirements, we can't possibly look at
everything. And even if we could look at everything, what would we do
with all of that information?
So we have set up this series of cornerstones and PIs and we
have established PI thresholds and inspection thresholds. And the
entire notion is to be able to take a look, from an objective
perspective, to see when plants are beginning to wander from what we
accept as the utility response being -- and this notion that the utility
can respond, and to the point where we need to begin to take some
action.
Now, we are not going to be measuring delta CDFs, we are
going to be looking at numbers of scrams, we are going to be looking at
some very simple things. And the whole reason to do that is to be able
to have some objective way to figure out when we ought to engage or when
the utility ought to engage, and the extent that we ought to engage.
And that is what the action matrix and all those things talk about.
DR. APOSTOLAKIS: I understand.
DR. BARTON: George, I think utilities are used to working
this. You look at the INPO indicator, they have got scrams, they have
safety system actuation, and they have got goals set for each one of
those, and utilities try to operate to stay within the goal. This is
the same kind of thing. If you get enough -- above enough of these
bands, you are probably in trouble. If you don't meet a lot of the INPO
indicators, you get a real good look at those areas when they come in
and do an evaluation as well.
I think it is a trust thing here. If get above a couple of
these targets, is the NRC going to come in and shut you down right away?
I think that gives you some nervousness of how this is going to be used.
But I -- I don't know, I think it is a good process, to try to figure
what is a better process than what they have laid out in looking at, you
know, how are plants performing. You know, where are plants getting out
of whack, so to speak, and what is the reason for that?
If you follow maintenance rule, you have got training,
procedures and human performance, and all those kinds of things are,
quote, "okay" at your plant, you are probably not going to have your
plant above this threshold in these areas. So all this tells me is I
see that, you know, what is going at that plant? What is the
maintenance history of that system? What is the problem with the
maintenance rule at this plant that they are up with this system
unavailable in the RHR system? And that is what I think you use this
for, and I don't have a problem with that.
Now, if you are going to come in here and beat me up over
that and shut me down because of it, I think that is the trust issue
here.
DR. APOSTOLAKIS: Well, no, first of all, I think this is
one of the best risk-informed documents --
DR. BARTON: I thought you believed that until this morning.
I am wondering what is going on here this morning.
DR. APOSTOLAKIS: I do believe it.
DR. BARTON: This is six months revisited or something.
DR. APOSTOLAKIS: And I want it to become better. But,
remember, this is -- no, really, you did a great job, don't get me
wrong. I am just arguing a couple of points that I think -- three
points that I think are very important, and some of them can be remedied
by writing a better background and introduction.
But I think, John, it is different from an industry -- when
I have an industry organization like INPO going to a particular plant
and saying, hey, this is the norm, you are near the tail or you are
outside, and so on, it is very different from having a regulator come
and tell me the same thing.
Let's say that I am a known recover plant, and I tell the
NRC I have legal documents here that you have granted me a license. No
where in the license do I see initiating event frequency and
unavailability and this and that. And now you are going to inspect me
on the basis and make decisions? I am going to have my lawyers after
you.
MR. BARANOWSKY: George, we do that now without this
objective cut. We see something happen at a plant, with our own
personal cult and culture and operation, we go do things.
DR. APOSTOLAKIS: Right.
DR. BARTON: They do it with the SALP process, which is much
more subjective than this.
MR. GILLESPIE: George, one of the --
DR. APOSTOLAKIS: No, no, no, no. Wait, wait. The fact
that we are doing it now, though, let me finish the thought, because
Michael is already -- the fact that we are doing it now doesn't mean
that it is the right thing to do, and, in fact, I thought that was one
of the major complaints at the stakeholder meeting, was the inspection
process, was it not?
MR. GILLESPIE: Yeah, the lack of transparency and the lack
of a standard.
DR. APOSTOLAKIS: And too intrusive.
MR. MALLETT: This is Bruce, let me make a comment. Is that
I think George is right that this is one of the holes that the
Commission emphasized to us that we need to work on, is establishing
these thresholds correctly. And, also, how the inspection findings
interplay with these performance indicators. Let's say somebody drops,
as you said, George, from .04 to .07, I think you were using
unavailability of the HPSI system, then they -- we have to look at the
inspection program and what has it shown us in that area as well.
Because it all fits together to say, do we believe the objectives of
cornerstone were met? That is what we designed the program to do, is to
determine is the licensee meeting the objective of that particular
cornerstone.
DR. APOSTOLAKIS: Bruce, if the four of you gentlemen were
doing this all the time for other plants, I wouldn't have any problem.
Okay. I wouldn't have any problem.
MR. MALLETT: I want to mention a couple of other points,
Frank, that are different than what George was raising. But it is close
to his original question on what was our objective. In the
risk-informed baseline program, we wanted to focus on risk, the planning
inspections and conducting inspections, and we have got that concept in
the process. We also wanted to make it clear why we are inspecting
something from a risk-informed perspective in linkage to our mission.
We have got that in there.
And, last, we wanted to have risk information put into how
we select our samples, which is different than today, and we wanted to
have that in there. So when you look at why we did what we did, those
were some of the original reasons that we put into the program. So I
recognize we have whole lot of thresholds, but we can't forget some of
those objectives on why we wanted to what we wanted to do.
DR. BONACA: One comment I have, if I could. I don't see
why this discussion should not be reflected somewhere in the early
portion of the document to this nation, because you are using a lot of
PRA, risk-informed information coming from PRA concepts. So you have
got to put some kind of insight in how this is going to be used, the way
you are describing here. And I think that that would enhance the
understanding of this approach for inspectors, in fact, and for
everybody else. So it would improve the scrutability of the process
that you are talking about.
DR. APOSTOLAKIS: One of my real concerns here is that we
will send a message to the rest of the NRC that when you develop
risk-informed regulations, it is okay to put thresholds on initiating
events, on unavailability. Well, it is not okay. It is not okay at
all. That is not the idea of performance-based -- risk-informed,
performance-based regulation, that is not the idea. And it is so easy
to fall into that pitfall, because, you know, it is detailed, it is
nice, it is something I can measure, but that is not the intent. For
oversight purposes, it is okay to look at these things, but not to put,
you know, regulatory limits, then you are back to a prescriptive system.
MR. GILLESPIE: Let me go back to one comment that was made
here that we didn't address. And that was, one, this is an indicator
process, including inspection, and part of what a utility gets, if you
would, from participating with the indicators, is the inspection program
won't duplicate the information that that indicator is giving us.
So there is some dependence on the indicator. We have a
reliance that we are not going to duplicate that area. We're going to
do some verification inspection. So there's definitely a backoff, if
you would, on inspection. Because the big question was what information
do we need to make a reasonable assurance finding.
The other thing is, this is not diagnostic or cause. When
an indicator says you need to understand what's happening, the problem
is still going to have to be discussed and identified in the context of
the regulations.
You're right, if we went in and said you've got four scrams
and therefore we're going to take some action based on that, no. The
root cause of the problem is still going to have to have its essence and
any corrective regulatory action taken is still going to be based in
regulation and license.
So when you do get engaged, you tend to fall back on the
traditional system of what was the cause, what's the requirement, was a
requirement violated, and if a requirement wasn't violated, then we have
to say we understand what happened and it's okay, or it's a new
phenomenon, and we have to do something about it and issue an order or a
license condition or something. So it gets us back into kind of the
today environment when you do become more diagnostic.
That was the intent. It's a starting point. It's not a
finishing point, and we recognize that.
The other thing is, it was never our intent to rewrite this
document, by the way. This document goes away by June. This document
is really the description of the concept that we would use as our
guidance in writing I'll call it the real program documents. So this
description of not abusing things would actually go into whatever we
would replace, Inspection Manual 2515, which has the philosophy of the
inspection program in it. One of the cautions I want to do is I don't
want to keep rewriting a document that's really not a functional
document in any sense or any system. This was a descriptive document.
So --
DR. APOSTOLAKIS: Is this a public document, by the way?
MR. GILLESPIE: Oh, yes, yes. It's on the Web. The
Commission gave us permission to put it out before they even saw it so
we could get comments. Comment period has started, and we'll see what
comments we get. I agree that -- the Commission said the same thing,
the context that we were talking to at the meeting was absent by way of
like an introduction or contextual statement in the front of it.
DR. APOSTOLAKIS: Let me ask you another question, Frank --
or all of you. One of the particularly sensitive issues with some of us
on this Committee, and I think with the industry, is that PRA is
perceived as being a tool for adding regulations or adding burden. Is
this document removing any of the burden, the existing burden?
MR. GILLESPIE: Yes.
DR. APOSTOLAKIS: Okay.
MR. GILLESPIE: In fact, if you look in the inspection
section, there are some hours there, and we didn't project it all the
way forward, but I had given this information to Sam, and he used it at
the Commission meeting, so now it's in a public transcript, so I feel
free to use it again.
This probably reflects a 15 to 20 percent reduction in
overall inspection.
DR. APOSTOLAKIS: Okay.
MR. GILLESPIE: In two phases. When Pat ballparked how many
sites would actually meet the entire profile as being in the licensee
control zone, it's in the ballpark of about 50 percent of the utilities.
So about 50 percent of the utilities that are now getting some level of
reactive kind of inspection from us would no longer be getting that
level of reactive inspection. So I think there's going to be kind of an
immediate incremental benefit from being risk-informed and being
focused.
And now the question would be is -- I think we need to be
cautious as regulators, and we're generally conservative -- is as you're
fine-tuning to a plant-specific nature maybe, that as you fine-turn, you
don't lose some sense of checks and balances that you've tried to build
into the system. And if you do lose it, you know what you're losing.
So yes, it looks like about a 20-percent savings. The
immediate piece in the baseline program itself from the core to this
baseline looks like it's approximately 15 to 20 percent. So that's 15
to 20 percent of the core, which represents about half the inspection
program across the board. And then there's about another 20 percent
which is the reactive effort, which might not take place.
Now what's going to happen, George, and you're absolutely
right, is I could say the safety gain here is everyone's going to strive
to be in the best category. It's a possibility. That's a business
decision.
DR. APOSTOLAKIS: Which is not a bad idea.
MR. GILLESPIE: Yes, I mean, you know, I mean, I have to
recognize that. I'm not living in a vacuum, because utilities every day
make the choice do I go along, and this is one of the criticisms. So
I'm not saying this as a fact, but do I go along with the inspector and
just fix it, or do I fight it. I mean, that's the kind of a thing you
hear from the regulatory impact survey going back ten years.
It would be a business decision that they would strive to
stay in the licensee control zone in all of these areas. Now do I feel
that we've kind of agreed if you would in setting these thresholds with
credible thresholds, reasonable thresholds, I think we have. You know,
I'd say the best shot is 95 percent.
DR. BARTON: They are reasonable thresholds.
DR. APOSTOLAKIS: Yes.
MR. GILLESPIE: They're reasonable thresholds. Now that's a
business decision. It could happen. And that could happen without us
doing any arm-twisting. Someone could say you know, if there were
really --
DR. APOSTOLAKIS: All you have to do is show up, Frank.
MR. GILLESPIE: Okay. Okay. All we have to do is --
DR. BARTON: I think we need to get on with Bruce. We're
going to lose him.
MR. GILLESPIE: Let's get to Bruce. Bruce only has about a
half-hour. I think we're in violent agreement. And the Commission was
also in agreement that we need some contextual words in the front of
this to put it in context. I don't want to rewrite the document,
because the document actually will go away within six months.
Bruce.
MR. MALLETT: You should have in front of you -- I think the
easiest way without repeating our last meeting is one slide that I call
a flow diagram that's used -- the scope of the program, planning
inspections and selecting. Do you have that in front of you?
DR. APOSTOLAKIS: Yes.
MR. MALLETT: Okay. Let me walk through that just briefly.
I don't want to bore you, but I want to highlight some of the concepts
we had previously in the program.
First, we've been talking about an oversight framework where
we have performance indicators, a baseline inspection, and an assessment
process. That led us to the cornerstones of safety concept in which we
have seven cornerstones. Those then defined inspectable areas and gave
us inspectable areas out of that. And George Apostolakis asked us
before to put in charts showing that linkage from the inspectable areas
to the mission of the Agency and to the cornerstones of safety, and,
George, we did that there in Appendix 2 to Attachment 3 on the document.
There's a chart for each of the cornerstones that shows inspectable
areas and performance indicators.
DR. APOSTOLAKIS: I've seen that, Bruce. Yes.
MR. MALLETT: Okay. I just wanted to let you know we do
listen to you, George.
We also --
DR. SEALE: Never doubted it.
MR. MALLETT: The document I'm going to describe these
concepts out of for the baseline inspection program are risk-informed is
in Attachment 3 to the Commission paper. There are nine sections to
that document. Each section describes a particular concept of the
program. For example, one that you asked us for, Dr. Seale, you asked
us the last time to make sure we put in resources. We put in a
projection of that in section 8 of that Attachment 3.
Let me continue looking at this chart with you to just
describe a few concepts. First of all, the scope of the program is
defined by the cornerstones and these inspectable areas, but one of the
goals we had up front was to define why we inspect. So we put that in
something called a basis document, and that's why we drew an arrow from
basis documents up to scope.
The next thing you would do in this baseline inspection
program, you would plan inspections. And we developed something called
a risk-informed matrix. We call it RIM No. 1. Those are also an
appendix to Attachment 3 to the document. I believe it's Appendix 3.
If you look at RIM No. 1, it outlines for each inspectable
area what the frequency of inspections are going to be, how many
inspections you might do, and how many samples you might select. We put
ours in there as a way of budgeting. One of the points of contention
for the Commission, you ought to realize, they challenged us on was it's
a great concept, Bruce, but for a lot of the inspectable areas, you
don't have much definition of the sample. And they are right on that.
We need their -- as we go through the pilot and using this program
further develop what samples we'll have and how many.
The next thing where we put risk into the process is in
selecting the sample. Once you've planned on your 12-month cycle and
your plant performance review in each region, you'll take RIM No. 2 then
and select the system that you might want to look at. That's generic.
Plant specific, you'll have to modify that by using the SRAs and any
plant-specific information to determine your sample.
And then the next thing you do in the process is you
inspect, and we've got the procedures concept set up as a procedure by
cornerstone to perform the inspection. And I'm going to -- procedures
are listed in section 4 of that Attachment 3.
And the last thing you'll do is you'll assess those findings
much like today. We envision you'll have a plant information matrix
type summary of the finding, but we envision that you'll have a risk
scale, and this is another hole right now, that this risk scale has to
be developed to set what we're thinking of as maybe three areas. You'll
have a performance that we believe is not risk-important. You'll have
another category we believe that's very risk-important in your finding.
And then one that just says it's a finding, but it's an average
performance.
DR. APOSTOLAKIS: Would you --
MR. MALLETT: We haven't developed this much -- yes, George.
DR. APOSTOLAKIS: Would you put another arrow there under
findings assessment to bring up again the plant-specific nature of these
things? In other words, you do your inspection, and then when you
evaluate what you found, you look again at the plant. Unless that's
understood. Maybe that's understood.
MR. GILLESPIE: Yes. George, let me kind of come out of the
closet a little bit on how we're working this. We actually right now
kind of have a draft which has two scales, two steps right now, and then
there will be a third step.
One is a generic screening out of totally risk-insignificant
anyplace kind of items. Then there's another step which gets you into
is it risk-significant in a sense at that facility. And then there's a
third step that says the inspector isn't in a position to make this
judgment, we need the SRA or we need some more analytic help on the
bottom. So it's kind of a multistep screening process, because we need
something that an inspector can quickly and rapidly use for the majority
of the cases.
DR. APOSTOLAKIS: Okay.
MR. GILLESPIE: So, yes --
DR. APOSTOLAKIS: That makes sense.
MR. GILLESPIE: We're getting to that point, because it's
the risk at the facility that we have to focus on.
DR. APOSTOLAKIS: Okay.
MR. MALLETT: Okay. The last point I wanted to make is that
we put down a block there that says cornerstones objective met, yes or
no. The idea of that is to show that this is a minimum baseline program
to be performed at all plants. If they meet the objectives, they'll
continue to receive the baseline program. If they don't, through as we
talked earlier, through crossing some threshold or from an assessment
finding, meet the objective, we want to increase the inspection above
that for that plant.
So, George, you asked the question of reducing burden. An
implication of this is that currently, you know, we have set out at some
sites rather than inspectors at the N level per number of units, we have
also in some cases set at the N plus 1 level. If that plant is in the
band where we believe they're meeting the cornerstone objectives, the
new program will say they will only receive the baseline inspection
program, you'll have to decide what you do with that other that is there
so that they don't receive more than that. So that's a big implication
of this change. But if we carry it through, that will be certainly a
reduction of unnecessary burden of a licensee's facility.
I can mention more, but what I want to do is refresh you on
those concepts and then leave it to any questions that you might have.
DR. APOSTOLAKIS: Which attachment has the inspection areas?
MR. GILLESPIE: Three.
MR. MALLETT: That's Attachment 3, but the Commission paper,
the SECY paper, the main concepts are just discussed or described on
page 12 of the Commission paper, and the implications for policy I just
mentioned are on page 15 of the Commission paper.
It also listed -- I'll mention one more thing -- in
Attachment 3 in the beginning you'll find an executive summary and in
that executive summary we list the work we believe still needs to be
done prior to conducting a pilot program.
DR. APOSTOLAKIS: Now the inspectable areas, Bruce, like
Attachment 3, I guess arabic 1, page 1, program -- Part 1, inspectable
areas.
MR. MALLETT: That's correct.
DR. APOSTOLAKIS: Attachment, page 1.
First of all, this idea of complementary and verification
inspection I think is very good. You are really trying to discriminate
as much as you can.
MR. MALLETT: Now I have to give credit to Pat Baranowsky
and his group for that --
DR. APOSTOLAKIS: He just stepped out of the room, so let's
hope he reads the transcript.
The inspectable areas by cornerstone -- how did you come up
with these? Was it just your group sitting around the table and saying,
well, what is it that we need to do here or was there some more
systematic way, because this is really what you are actually inspecting,
right?
MR. MALLETT: What we did was originally we sat around the
room or Pat Baranowsky's group sat around the room and we said what all
do we believe we need to inspect to meet these cornerstones objectives?
I think we had a hundred and some areas that we came up with
originally, and then as the framework team developed the performance
indicators and as we met with them and discussed them, we said where do
we have a performance indicator that we don't need to inspect because it
already covers that, and we crossed the list off for those, and then we
said -- we made another cut. What is risk important? Well, we crossed
a number of things off the list that we felt were not risk-important.
I'll give you an example. Under the Mitigating Systems
cornerstone we originally had an inspectable area of observing
maintenance activities. We said we don't need to observe maintenance
activities. The key part of those from a risk perspective is the
post-maintenance testing, we thought, and so we left that one in but
took out observing maintenance activities.
That is how we arrived at inspectable areas, but one thing
I'll also mention, George, in Appendix 1 to that attachment in the basis
documents we added something after we met with you all the last time.
We described in there why we inspect, but we also discussed performance
indicators.
Remember you all suggested we highlight how those interplay.
DR. APOSTOLAKIS: Yes. In fact, let's go to Attach 1, page
5, because that is where my notes are. Attachment 1, page 5 -- where
you also have the inspectable areas by cornerstone.
MR. MALLETT: Right.
DR. APOSTOLAKIS: For example, if I look at initiating event
cornerstones, maintenance rule implementation, maintenance work
prioritization and control -- why do I have to worry about work
prioritization and control if they comply with the maintenance rule?
MR. MALLETT: For initiating events?
DR. APOSTOLAKIS: Yes.
MR. MALLETT: What we felt there was there are some events
where the maintenance caused the problem by not deciding the right -- I
don't want to say sequence, but not looking at one system they are doing
maintenance on versus another system, because they didn't prioritize
their work to do it in the sequence that wouldn't cause that problem
later on -- like if they had an event while they were working on a
particular system and then they were doing maintenance on another system
that that could affect, they need to be careful of the interplay of the
two.
That was the idea was to look to see that the licensee is
looking at the interactions between systems when they do maintenance so
that they don't cause initiating events for one system by working on
another system. Does that make sense? That's what we were trying to do
there.
DR. APOSTOLAKIS: Yes.
DR. BARTON: Understand it.
MR. GILLESPIE: George, I think -- isn't there a change to
the maintenance rule that's been briefed to the committee? I am not
sure where it stands.
DR. BARTON: We haven't heard that yet.
DR. APOSTOLAKIS: We haven't seen it yet.
MR. GILLESPIE: It talks about configuration evaluation.
DR. BARTON: We know it is coming, Frank. We haven't heard
the details of the rule change.
MR. GILLESPIE: That may impact this area if it becomes part
of the rule, but for inspection purposes we're touching it until the
rule is a rule.
DR. APOSTOLAKIS: Now heat sink performance -- we can't have
a PI for that?
MR. MALLETT: We believe you can but we're not going to
develop --
DR. APOSTOLAKIS: Oh, okay, okay, all right.
MR. MALLETT: If you look at heat sink performance, we put
complementary down there --
DR. APOSTOLAKIS: Right.
MR. MALLETT: -- as saying that you need to do an inspection
in that area because there is not a performance indicator.
DR. APOSTOLAKIS: But there could be one.
MR. MALLETT: It's not supplemental to a performance
indicator but it could be. Same way for the -- if you look under
mitigating systems and you look at configuration control, and you follow
that at inspectable areas, one of them we have there is talking about
equipment alignment and so forth. There is nothing for shutdown
conditions right now, performance indicator, but we maybe could develop
one for shutdown conditions, for example.
DR. APOSTOLAKIS: Anyway, yes, I --
DR. POWERS: And that raises a question. In the document
there are PIs that you define now and there are PIs that you think you
might be able to define in the future.
One of the areas where you did not I think define a PI but
rather relied on inspection was fire protection, yet the fire protection
people are off busy trying to create a performance-based code with
performance indicators.
Why did you feel that it was unlikely that they would be
successful?
MR. MALLETT: I am not sure, Dana, that we thought it was
unlikely they would be successful. We felt at this point in time that
there wasn't one developed and so --
DR. POWERS: I mean that is not the criterion on shutdown,
so why should it be the criterion on fire?
MR. MALLETT: Well, the same way in shutdown we felt that
there wasn't one developed yet so we would not rely on that performance
indicator.
DR. POWERS: I understand that, but in some cases you have
said, well, in the future there might be, and you have allowed for that
possibility that in the future there might be --
MR. MALLETT: Oh, I see your point.
DR. POWERS: -- but in fire you apparently had little faith
that this could be accomplished. Now that may be a very, very rational
viewpoint but I just wondered why you had that viewpoint.
MR. MALLETT: I don't think it was having little faith,
Dana, as much as we may have had an oversight inconsistency between the
two writeups and we may have to look at that.
DR. POWERS: I think you should, because if that is a major
initiative that the Staff is undertaking to vote with your feet on it
may not give them the encouragement they need at this critical juncture
in their work.
MR. BARANOWSKY: I think we identified the performance
indicators that we thought we might see some development and acceptance
of in the next six months, and our impression was that the fire program
performance indicator probably was going to take longer than that to
develop and get operable.
DR. SEALE: You are less sanguine about shutdown?
DR. POWERS: In six months we are going to have a
shutdown performance indicator?
MR. BARANOWSKY: Well, there were some proposed by the way,
and what we didn't have was the time to get the data and look into the
details of the indicator to see whether we thought it was a really valid
indicator or not. The feeling was that it probably could be developed.
It was a judgment call. That's all.
DR. POWERS: Well, it may be very sound.
MR. BARANOWSKY: You wouldn't be the first to say that it's
not going to be possible.
MR. GILLESPIE: It wasn't our intention to take a shot at
the fire people.
MR. MALLETT: Yes. We can take Dr. Powers' comment and look
at it during the pilot program and see if there is some way we can
factor that in better, give more credit to them -- good point.
MR. GILLESPIE: I think simplistically when you look at
shutdown, you can ask the question how many water sources are available
and is there power available to pump the water to where it needs to be
and is the heat sink available?
All of a sudden I have got specific things that you can
count at any given time at a shut down facility and say yes, I have got
two water sources available -- yes, I have power from two different
sources ready to go to the pumps. It's actually something we could
count, so we saw some near-term achievability in shutdown.
We were not familiar enough -- in fire protection when you
look a fuel sources and other things, how would you measure fuel
sources? It seemed like a bigger problem, so there was kind of, I
think, a near-term, longer-term perspective when we were doing it.
I literally mean "we" -- I mean just the team working on it
saw shutdown was a place we could make progress and fire just needed a
whole lot more work before you had something that was countable and
easy -- something you would see from actually the control room.
DR. POWERS: You have a tension that runs throughout the
document between defense-in-depth and some of the rest of the document,
the rest of the approaches you have.
One area where you don't have tension, one area that
everything that one would do for inspection and regulation of safety is
lined up with defense-in-depth is in fire protection. The things that
you would count are the things that make up the elements of fire
protection -- of defense-in-depth and fire protection.
Do you prevent fires from occurring? Do you detect and
suppress fires? Do you protect the equipment? So in a counting basis I
think that you will find that you can count and what you count are
things that are explicitly defense-in-depth items. There is not a
tension between defense-in-depth and the things you are inspecting the
way there is throughout the rest of the document -- and that is because
fire protection is the one area where they have explicitly defined
defense-in-depth.
MR. MALLETT: I guess one other comment I would make on a
different note than what Dr. Powers was talking about was the last time
we met NEI said they had two points of contention about the proposed
program.
One was they had thought when we set out there were going to
be more performance indicators than there are now in the program, and we
indicated that we opened the door for that, but we haven't developed all
those performance indicators and that is why they aren't in the program
now.
The second point of contention they were making was remember
when we said we wanted to do an identification and resolution of
problems inspection as a two week inspection once a year and they
indicated they didn't know we needed to do that. They thought it was
repetitive.
We had left it in the program. We changed it to once every
two years but we thought it was important to have in the program to have
an independent look at the routine inspections we're doing in the
baseline by individual inspectors and to also have an across-the-board
look at how they are identifying and resolving problems, but I did want
to point out to you that that was the point of contention on their part
and I believe they still have that view even though they support the
concepts in the program.
If there aren't any more questions I am going to get off the
line. We have an enforcement conference we're going to start.
MR. GILLESPIE: That is risk-informed --
MR. MALLETT: And they are using risk information. All
right?
DR. BARTON: Any other questions of Bruce before he signs
off? Hearing none, thank you very much, Bruce.
MR. MALLETT: Thank you for letting me do it by
videoconference so that was very good. Thank you.
DR. POWERS: Mr. Chairman, I am now fighting with another
windmill, to the same success most people have fighting with windmills.
Have we gone into the details of Appendix H?
DR. APOSTOLAKIS: Yes. Not the details, the overall
structure, but we spent an hour on it. If you --
DR. POWERS: I guess I need to understand a little bit
better the quality of the tools that have been used in defining things
in Appendix H and better to understand the benchmarking that was done in
Appendix I.
I think I understand the approach that was adopted in H, and
I will confess to being confused a little bit over Appendix I.
Within H, I need to understand you utilized the Sapphire
Code plus some PRAs that were submitted by the licensees or by NEI
provided them to you?
MR. BARANOWSKY: We used the Sapphire Code with PRAs that we
obtained from licensees. There were approximately 13 of them that we
used.
DR. POWERS: And these PRAs were in some sense certified for
this particular application?
MR. BARANOWSKY: They were not certified. They had been
reviewed as part of the IPE process.
DR. POWERS: And what we know about the reviews that are
done in the IPE process is that they serve adequately the objectives of
the generic letter that initiated the IPE process, but in no sense are
they what I would call a technical review of how well the IPE or the PRA
reflects the plant, how good the database is, how complete the scenarios
considered are.
MR. BARANOWSKY: Yes. We weren't really trying to do a
plant-specific assessment. What we were trying to do was get some
representative models and our belief is that the results from the models
and the IPEs when one looks at the broad insights are reasonably correct
in terms of what is important and the kind of levels of core damage
frequency that one sees.
In fact, if we look at the results of the action sequence
precursor program, we see approximately the same order of magnitude core
damage frequency comparability between that program and what the IPEs
give us and most of the important sequences are of similar
characteristics but details do different and there are a few different
ones, so our feeling was that in terms of orders of magnitude we have
got a spectrum of plant designs with PRAs that have at least been looked
over somewhat. They don't have obvious flaws in them and therefore they
would be reasonable models to perform sensitivity analyses to understand
how some of these parameters' variation would affect risk calculations.
DR. POWERS: At least when I look at the IPE insights
program, I see an order-of-magnitude difference between plants of
nominally the same type. So you've got an order of magnitude --
uncertainty band? I don't know what you call it in this.
MR. BARANOWSKY: Yes.
DR. POWERS: This context. Derived from a tool that may or
may not be qualified for this kind of job.
MR. BARANOWSKY: Yes. I think the reason that it's a
reasonable tool is because of the way that we used it. We didn't take
any one plant. What we did was we ran a bunch of these calculations and
then we sort of enveloped them. And we have no reason to believe that
the IPEs grossly and in a biased manner underpredict the likelihood of
core damage for this type of a calculation, but that in some cases
they're a little high and in some cases they're a little low. And in
fact most of the work that we've done in AEOD taking actual operating
experience and comparing it to IPEs shows that about 90 to 95 percent of
them have pretty reasonable values when it comes to initiating event
frequencies and system reliabilities, and that there are a few that are
unexplained as to why they are higher or lower than what we would get
from looking at some of the more direct indications from the operating
experience.
So it's our belief that taking a set of these, running the
sensitivities, and more or less enveloping the results probably is
conservative for this cut at setting thresholds. Moreover, I think the
level of risk that we're talking about looking at is, you know, rather
low. We're talking about changes in core damage frequency of 10 to the
minus 5, 10 to the minus 6, for which there's not a significant public
risk associated with it, and that one doesn't need to be precise in
calculating these things because we're not using it in that manner.
We're trying to get into the ballpark of what we think is an area of
increasing risk as we move through the different thresholds and
performance bands.
DR. POWERS: Someone might ask that given the uncertainties
that are associated with risk estimates, is a differential of 10 to the
minus 5 in core damage frequency detectable? That is, maybe you're
looking at just noise.
MR. BARANOWSKY: It probably realistically is the noise, but
it is something that one can calculate. Just like one can sit down and
deterministically say a piece of equipment doesn't meet a certain
regulatory rule, that probably is well within the noise also with regard
to whether the plant is safe or unsafe. So I'm not sure that there's
any difference there. The point is that we are able to identify plant
performance characteristics that seem to show increases of these small
magnitudes, and it's the best tool that we have available for
identifying where we should be focusing our look.
DR. POWERS: Can I ask you about the status of what,
validation verification, peer review of the Saphire code?
MR. BARANOWSKY: Yes, I guess I'd have to ask someone from
the Office of Research to give you an up-to-date statement on that, but
my understanding is that that code has been pretty well developed,
reviewed, and validated by the Office of Research over about 15 or so
years at least. I don't know that anyone here is familiar with that.
I'm looking around the audience. I don't know if John is --
DR. POWERS: What I know is that the Office of Research has
in many cases subjected codes that they have sponsored the development
of to what I think is a rigorous and outstanding peer review process,
very disciplined, very structured, published reports, and allowed
developers of those codes to respond to the peer review comments and
responses from the commenters. But I am not familiar with such a review
for the Saphire code. That doesn't mean it hasn't been done. I'm just
not familiar with it.
MR. FLOOD: This is John Flood from the Office of Research.
We'll have to get back to you on that, Dana.
DR. POWERS: I wish you would, because I think this is
crucial that the Office of Research in developing its PRA tools has
tools that the rest of the Agency can put an enormous amount of
confidence in. Whereas they can have their doubts and qualms about IPE
PRAs, they should not have any doubts about the capabilities and
limitations of their in-house tools.
MR. BARANOWSKY: By the way, I would like to point out that
most of the doubts and questions about IPEs have very little to do with
the specific computer code that was used. They usually have to do with
the assumptions and completeness of the model and the tools generally
give pretty similar results, and that's the way a lot of them have been
tested out, by benchmarking against each other.
DR. POWERS: If you could give me a ten-minute guide on how
I should reread Appendix I to understand what's been done, I would
appreciate it, because I have to admit --
DR. APOSTOLAKIS: Well, let me ask, what's the process here,
Mr. Chairman? We seem to have demolished the presentation.
MR. GILLESPIE: Well, no.
DR. BARTON: I don't think so. I was going to ask Frank,
where are you going from here, because we --
DR. APOSTOLAKIS: We will come back to --
DR. BARTON: We've done some bantering back and forth and
trying to get Bruce's input before he to fall off. I was going to ask
Frank where does he see going next, we take a break now, or you get into
this.
DR. APOSTOLAKIS: Because I had some comments myself, and I
don't know when to raise them.
MR. GILLESPIE: No, right now what we'd like to do is
continue the questioning on performance indicators in the oversight
structure, and then move on to Mike and the assessment process.
DR. APOSTOLAKIS: Okay.
DR. BARTON: Why don't we finish that, and then before we
move on to Mike, we'll take the break.
MR. BARANOWSKY: Okay. Well, I mean, the questions --
DR. BARTON: Any other questions about performance
indicators?
MR. BARANOWSKY: In this area?
DR. BARTON: Do we have any other questions of Pat at --
DR. APOSTOLAKIS: I cannot ask any questions.
DR. BARTON: Okay.
MR. GILLESPIE: I was thinking you were going to continue
on, Dana, beyond -- the question you've asked also applies, and in fact
the Commission asked -- David Lochbaum asked it -- relative to the stuff
Bruce was doing. I'm sorry we don't have Bruce in line, but we've got
John Fleck here. And that was how did you pick the inspectable areas to
be risk-informed given you had to average across PRAs if similar plants
could come up with different analysts with different results.
And John, there was kind of a process of averaging -- it's
kind of gross, but we did this -- on two plants and less than all of
them for picking sequences that led to areas, and could you just in a
few minutes --
MR. FLECK: Yes, I'm John Fleck from the Office of Research.
We looked across IPE the top ten sequences across plants to draw
insights and put those into a matrix called RIM 2. Basically it was to
pull out the information that drives risk out there onto the table, so
when we are choosing inspectable areas we could somehow link the area to
those risk insights.
So we did that -- the insights were generated, as Frank
said, using IPE information that we brought in house through the IPE
data base by scanning across the sequence, the dominant sequences at all
plants, and the insights contained in NUREG-1560. And the process more
or less evolved around -- there was about five of us involved in looking
at the inspectable areas and weighing it with that information in the
IPE as well as basic information like how much effort would it take even
just to go look at that particular area. So there's always some minimal
amount of time that you'd have to spend on it. And that's where we came
down from 150-some-odd areas to something like 26 in the reactor safety
area.
Does that address the question, Frank?
MR. GILLESPIE: Yes. Basically what he's describing is kind
of an averaging process that went across the PWR designs, which meant if
a sequence showed up in the top --
MR. FLACK: Top ten.
MR. GILLESPIE: Top ten at at least two facilities. There
was more detail to it. At a couple of facilities, didn't have to show
up anyplace, then it got to influence an inspectable area, which means
the inspectable areas at one plant might be from a sister plant of
similar design where a different analyst found something more important.
So it's kind of a generic averaging is kind of how I put it in my
lexicon.
Now the next step is when you apply those insights to
plant-specific, and this is a step where I'm trying to develop in the
next phase, is the areas are defined, there will be a minimum look at
these areas, now customize your look to the specific sequences from that
facility.
And it's the kind of a process of attempting, if you would,
in a gross sense, to take the best of the average and say here is the
areas you need to look at, but when you pick your specific sample, you
need to look at your specific plant and make sure you have hit your most
significant sequences at that plant.
It is not a perfect system because we didn't have perfect
IPEs and PRAs, but it was a way of trying to deal with the strengths of
the PRAs and the weaknesses in a process way. And the research reports
are in draft form. They are kind of in for final comment right now.
MR. FLACK: Yeah, we have got them for out for comment. We
should be able to finalize, certainly, within the next two weeks.
MR. GILLESPIE: But the next step is, now, how do you go
from the generic averaging to the specific application and not lose your
generic insights, but, yet, do some successful customization, as you
pick your sample, to the high probability sequences at that facility, or
high risk sequences? It is a challenge. And we are dealing in gross
levels. This isn't a highly -- a high fidelity system we are in.
DR. POWERS: Is this an exercise undertaken with a pad and
paper or undertaken with computer?
MR. FLACK: The exercise of gleaning out the insights was
done basically by --
DR. POWERS: No, I am talking about the step from going --
you have got these generic inspectable areas.
MR. FLACK: Pad and paper.
DR. POWERS: And now you want to customize it. Is it -- is
that reasonable to have it as a pad and paper exercise, or should there
be other kinds of tools in the hands of the people to do this the way
you envision?
MR. GILLESPIE: Again, in kind of the macro sense that we
are dealing in, pad and paper, given we have nailed it down, we have
kind of boiled it down to a matrix kind of approach, and then you can go
to the specific facility and we know to the best we know the dominant
sequences at those facilities. It is now a matter of saying, what
systems are predominant? And if I have to look in the mechanical area
in a mitigation system, what is the system I want to be sure at my
facility will stop a sequence? What is -- and those risk important
things are all really available for that most part right at the site.
So making that match in pad and paper, I am not sure I would
computerize it. That is a subjective match of kind of the generic stuff
that, if you would, that we have put together and the site-specific PRA
results.
Right now we are not thinking about putting any kind of
fancy algorithm together to try to do it other than do it. We also have
to focus on the people doing it. The people doing it are likely going
to be the senior resident inspector and the resident staff, with some
assistance from the SRA. And you are looking at about 20 sites per
region, and a lot of up-front planning to do this right.
So if an SRA spends -- if the senior resident spends about a
week, and of that week, he gets a day of the SRA's time to overview his
recommendations and results, that is probably in the ballpark of the
expertise that is likely to be on most facilities. So that can't
complicate it too much. It has got to be very user-friendly.
Again, not to say in the future we won't try to computerize
it, but to get something done this year, this is the best we
conceptually could approach it with.
DR. POWERS: I was certainly not thinking about this year, I
was thinking --
MR. GILLESPIE: Okay.
DR. POWERS: -- ten years from now.
MR. GILLESPIE: Yeah. I think, again, then you are to the
point of probably having site-specific profiles. And the degree of
sophistication ten years from now, and someone will have settled the
fidelity of the PRA question, I hope, by then. We will either have an
ANSI standard or someone will have a standard that these will be
measured against, and then we can move forward. Lacking that, this is
the approach we have taken to do it.
MR. FLACK: There is one other thing, what we have raised,
we have discussed -- we are discussing with Frank, is developing these
generic matrices to make them plant-specific. So, it would be a matter
of going to the site and finding out what additional sequences would
have to be considered and trying to fold that into a matrix form for
each site, so each site would have its own risk set of insights that
then would be updated and used by the inspector, but these are just
things we are entertaining at this time.
DR. POWERS: I think there is no question that that will be
one of the most interesting aspects of the pilot. How do we go from a
generic kind of appendix that, quite frankly, I don't understand right
now, but that is my fault, to the specific activity will be real
entertaining.
MR. GILLESPIE: Yeah, because then you are going from, as
Bruce said, areas that contain multiple systems to specific systems that
are going to be sampled. And we also view that as a significant
challenge, because the directions on how to do that right now are
lacking
DR. BARTON: Frank, is this the time to take a break? Till
25 after and then we will hear from Michael.
[Recess.]
DR. BARTON: I think we have got a quorum here, so we will
get started. Michael, you are next, with the assessment process. I
don't know if there's any other questions for Pat or not at this point,
I don't think so.
MR. JOHNSON: Okay.
DR. BARTON: We will find out if he leaves, though.
MR. JOHNSON: Okay. I just have a couple of brief slides to
put up, and I don't want to -- or I hadn't intended to go over the
entire assessment process. We did that the last time we met with the
ACRS. I just wanted to remind us that the purpose of the assessment
process is to assemble the information that we get, both from the PIs
that we are collecting and from the inspection, the risk-informed
baseline inspection, plus the additional inspection that we do, assemble
that information to arrive at conclusions regarding plant performance
and then, based on those conclusions, to identify resultant regulatory
actions and then communicate the results of those, those actions, and
the results of the assessment to the public.
And then, finally, we want to provide a means of follow-up
such that we look at future performance, factor that into the assessment
process and make adjustments as appropriate, based on whether the
licensee is or is not responding and addressing the problems that we see
as being important.
The second slide I will put up is this slide I will call key
concepts. I just wanted to remind us that the process is based on
thresholds. We have done a lot of talking this morning about thresholds
for the PIs. Also, keep in mind that we will have thresholds -- a means
to evaluate the significance of findings from the inspection program and
thresholds associated with that in each of the cornerstone areas. And
it is those thresholds that really drive us, or put us in the position
where we consider additional action beyond allowing just the licensee to
take action based on their performance. So it is very much driven by
thresholds.
The assessment process considers the performance for a
12-month rolling window. That is not new, we have been talking about
that for a while. It is a rolling 12-month period of time that we are
looking at, PIs being provided each quarter, and the inspection results
will come in as we accomplish inspections.
The assessment process provides a graded approach. One of
the things that is in Attachment 4 is something called an action matrix.
And in looking at that matrix then, and with the PI results and the
inspection results, we have a graded way of looking at what we
communicate to the licensee based on performance, the actions that we
take, whether we do inspection, so and so forth. All of our actions,
our management involvement are graded such that plants who perform
better get, if you will, the minimal level of engagement by the NRC.
Plants who perform -- whose performance crosses multiple thresholds get
a greater degree of involvement by the NRC.
The assessment process eliminates use of the watch list and
superior performer recognition. That was one of the things that even
the IRAP concept proposed, and we thought it was important then, and we
think it is still important that we do away with things like superior
performer recognition and the watch list, we don't think they serve a
purpose.
And, finally, one of the things that we needed to do with
the assessment process was look at plants that are in an extended
shutdown. Extended shutdown has an impact on the PIs, as you are well
aware, and the inspections, the additional inspections that we do, we
think it is necessary to do, with this assessment process that we are
proposing, exactly what we did today, and that is to set those plants
aside for this assessment process.
And we rely on the Inspection Manual Chapter 0350 process
that has restart approval requirements for us in terms of what we will
look at to decide whether a plant is ready to restart, and we will make
some logical decision about when that plant should rejoin the assessment
process following their restart at the end of that extended shutdown
period.
Let me just put up very briefly the action matrix. I
mentioned this in terms of talking about the fact that the assessment
process provides for sort of a graded range of actions. If you will
look across the top, that gives you the results from anything from a
plant that would receive --would not cross any threshold, any PI
threshold, or any inspection threshold, all the way across to plants who
have crossed multiple thresholds. And then if you look down the left
column, we talk about management meetings, the things that we do in
terms of response, management meetings, licensee actions, NRC inspection
and regulatory actions, and also the communication. And so, for
example, if you look in the left column, you can see that a plant that
hasn't crossed any threshold, a plant with all the indicators and all
the inspections in the green band, simply receives a risk-informed
baseline inspection program. And, again, as you move to the right, the
responses becomes graded in significance, increasing significance.
Let me mention one last thing, and I will shut up and take
whatever questions you have. There's --
DR. APOSTOLAKIS: Mike.
MR. JOHNSON: Yes, sir.
DR. APOSTOLAKIS: Would you put that back on?
MR. JOHNSON: Okay.
DR. APOSTOLAKIS: Where can I find it here? Where is it, do
you remember?
DR. WALLIS: Twenty-five.
DR. APOSTOLAKIS: Twenty-five.
DR. WALLIS: In the handout.
DR. APOSTOLAKIS: No, I know.
MR. JOHNSON: It's Attachment 4.
DR. BARTON: Attachment 4, George. Attachment 4, page 19, I
think.
DR. APOSTOLAKIS: Well, page 9 of the --
MR. JOHNSON: And page 9 of Attachment 1.
DR. APOSTOLAKIS: Attachment 1. Page 9 of Attachment 1. I
am wondering, under Roman II there, one or two inputs white, but the
cornerstone objective is fully met. Why don't we move the regulatory
action of the commenting response and make it a licensee action, and we
do nothing? And I am not even sure that the SRI or the branch chief
should meet with the licensee.
MR. JOHNSON: I think the answer to your question is, when
we cross a threshold, and that is what we are talking about in that
second column, we would do some follow-up inspection in that area,
whether it is a PI or whether it an inspectable area from the
risk-informed baseline inspection program.
And so when we talk about documenting the response in the
graded area and in the inspection report, what we are talking about is
just making sure that that inspection report captures what the inspector
has found in that additional inspection that follows up in that area.
We would do that. It is sort of an obvious outcome of that inspection
that you have done to address that fact that you have crossed that
threshold.
DR. APOSTOLAKIS: Well, yeah, I understand that, but I guess
what I am saying is in the name of granting relief, to both them and us,
why don't we start getting involved when one of the cornerstones is
degraded? In other words, your third column, Roman III, you have one
degraded cornerstone, so now the NRC is beginning to provide increased
oversight, you know, that kind of thing. It is really a matter of
policy, but I am just asking why we can -- can we not do --
MR. JOHNSON: Yes.
MR. GILLESPIE: You're right, George, it is a matter of
where you set the thresholds, and how much margin you are going to have
from you get in a more interdictive mode.
DR. APOSTOLAKIS: Right.
MR. GILLESPIE: And we have basically set the threshold. In
fact, if you look in Reg. Guide 1.174, there is a graph in there that
looks very much like the graph Pat uses but it is upside down. And we
are, in essence -- we are, in essence, adapting those same thresholds.
The bottom band or the least risk significant band in that Reg. Guide,
where things would be turned over to licensees, is very consistent with
the description of what we have got in the licensee response zone, or
the green band. And so what we did was try not to invent new policy,
but adapted that perspective.
DR. APOSTOLAKIS: Yes, but in that guide, though, they are
dealing with core damage frequency in there.
MR. GILLESPIE: Yes. And --
DR. APOSTOLAKIS: Here, you are talking about not even a
cornerstone being degraded.
MR. BARANOWSKY: But the thresholds were based on risk
insights that put us into a core damage frequency level of risk
comparable to that in Reg. Guide 1.174, where the NRC would provide
review and approval of changes in licensee design and operational
features, so --
DR. APOSTOLAKIS: Wait a minute. I mean you can have --
MR. BARANOWSKY: That is what the white zone is meant to
capture.
DR. APOSTOLAKIS: Yeah, but you mean you can have one or two
inputs white, when, in fact, the cornerstone objectives are fully met,
and see a difference in CDF?
MR. BARANOWSKY: No, the -- well, yes.
DR. APOSTOLAKIS: Really?
MR. BARANOWSKY: Well, I mean you can calculate the
difference in CDF, it is small. I mean, for instance, in Reg. Guide
1.174, it says the licensees can make permanent changes to their plant
with a CDF change of up to about 10 to the minus 5. And the white band
of this performance model includes core damage frequency changes up to
10 to the minus 5, so it is meant to be consistent with Reg. Guide
1.174, where there is regulatory oversight of risk-informed applications
that occur with changes up to 10 to the minus 5.
The same thing over here, when you cross any one threshold,
the implications are comparable to being in that kind of a situation
with regard to changes or increases in risk.
DR. APOSTOLAKIS: Well, I would expect that if the
cornerstone objectives are fully met, even if one or two inputs are
white, you wouldn't see much --
MR. BARANOWSKY: Well, you are not going to see much.
DR. APOSTOLAKIS: Unless the plant was already one of the
great performers, way below the threshold, so by increasing it a little
bit, you know, --
MR. BARANOWSKY: This isn't a regulatory hammer. This is
really just the beginning of taking a look so that we don't have things
go from everything is all right to shut them down the next day. This
way the licensee sees it coming.
DR. APOSTOLAKIS: Right.
MR. BARANOWSKY: There is plenty of warning. The NRC sees
it coming. And, in theory, it is a very modest regulatory engagement at
that point.
MR. JOHNSON: Yeah, and it is very much -- it is very much
philosophical. The notion was if the system is set up on cornerstones,
you don't wait till you have got a degraded cornerstone before you
engage. You engage at a point where you are seeing some degradation,
but not degradation significant enough to take down the cornerstone, if
you will.
DR. APOSTOLAKIS: Yes, but if you look at 3, it says
cornerstone objectives met with minimal reduction in safety margin. So
it is not that, you know, the next category is something that already is
pretty bad. You are still meeting the objectives even in Roman III.
MR. JOHNSON: Right, but we are talking about --
DR. APOSTOLAKIS: What I am saying is doing nothing in the
first two columns except being informed and then start doing things 3
and above.
MR. BARANOWSKY: Not meeting a cornerstone objective is
considered pretty significant, it is not the beginning of engagement for
us. I mean that is the equivalent of saying I don't have any
defense-in-depth or something like that.
DR. APOSTOLAKIS: No, but you are still meeting it, though.
Cornerstone objectives met.
MR. BARANOWSKY: No, I understand that.
DR. BARTON: Yes.
MR. BARANOWSKY: But if we wait until we get to the point
where things are really broken, that is one of the concerns that was
raised about not having leading indicators in the current process in
which one day the plant is operating, the next day we go there and we
say, oops, we have to shut you down because this is really broken badly.
We are trying to have a process here that slowly steps into a
recognition of declining performance so that there is plenty of
opportunity to turn that around before we ever get to these level 4 or 5
conditions. I mean we never want to get there.
DR. APOSTOLAKIS: I know, 4 and 5, I agree.
MR. GILLESPIE: Remember, this is an indication system, and
the severity or the pervasiveness of the root cause to the reason the
indication has tripped a threshold is really the question, once you trip
a threshold. And, in fact, NEI's proposal, which we have spun off of
here, up-front said if you meet all the PIs that are proposed, that
should be evidence that your corrective action problem identification
system is working. If you trip a threshold, there is at least some
essence of evidence that your problem identification corrective action
system isn't working. That is all we have got. We have got some
evidence that problem identification corrective action isn't working.
Now, when we get more diagnostic, the question is, how pervasive is it
or isn't it? This is an indication where we ask that question.
So that is kind of one of the fundamental premises that even
the industry proposed, actually, similar levels. If you remember, they
proposed three scrams and we kind of verified that that looked good to
us on the same basis. So that is how we are coming at it.
Could the root cause -- are we good enough to have created
the perfect system? No. So that is how the thresholds were set, and
that is the intent of the indication. The primary indication is there
is a problem with problem correction and identification of problems.
MR. JOHNSON: There is one other thought I ought to mention,
too, George, and it is in Attachment 4, and it mentions the fact that
when we go to apply this action matrix, one of the first things we would
do is to consider what the licensee has already done.
So, for example, if you -- let's suppose you have one or two
of these areas where the licensee has crossed the threshold. The first
thing you would do is to look to see if the licensee had taken some
actin to do some diagnostics, had taken some action to address the
problem, and that would further temper what we would do in terms of the
follow-up inspection that we would plan. So, we really see this second
column as one that is really a very measured response, but necessary as
licensees begin to cross these individual thresholds associated with PIs
and inspections.
DR. APOSTOLAKIS: But look at the first column. All
assessment inputs are green, and there is licensee action, licensee
corrective action. To correct what?
MR. JOHNSON: Well, that simply says if the licensee is in
the green band, all of these indicators are in the green band, where the
inspection would find issues, we would simply put those issues in the
licensee's corrective action program and the licensee would address them
in accordance with that whatever prioritization scheme they set up,
because the licensee has a good success at doing that. This is the
routine dealing with issues as they arise.
DR. APOSTOLAKIS: So there will be issues even though all
assessment inputs are green?
DR. BARTON: Oh, yeah.
MR. JOHNSON: Oh, sure.
DR. BARTON: The world is not perfect, they are saying.
There may be little blibets, but you are still going to be green because
you haven't crossed any threshold.
DR. APOSTOLAKIS: Like what issues?
DR. BARTON: I don't know, give us an example of where it
could be.
MR. GILLESPIE: Tag-out problems.
DR. BARTON: All right. Equipment tag-out.
MR. GILLESPIE: Equipment tag-out.
DR. BARTON: There you go.
MR. GILLESPIE: Not safety significant, didn't compromise
the system. Inspector sees it, it is a requirement.
DR. APOSTOLAKIS: But that is my point, though. Why should
you worry about it if all the assessment inputs are green?
MR. JOHNSON: And that is our comment is we don't worry
about it. That is left to the licensee.
DR. APOSTOLAKIS: But why would you even notice? Why would
you even bother to look at it?
MR. BARANOWSKY: Because you don't know, until you get all
the facts together, whether or not the facts indicate there is a
problem. You can't go and say there is no problem so I won't look for
facts. Your job is to go get the facts and then see if they indicate
that you should have a more aggressive examination or follow-up
activity. So the first step is getting the information, that is all in
the green is about. Baseline inspection program to get baseline
information. Performance indicators, inspections.
DR. APOSTOLAKIS: Wait a minute. All the assessment inputs
are green, everything. All the indicators, the areas, right?
MR. BARANOWSKY: Right.
DR. APOSTOLAKIS: There is nothing to correct.
MR. JOHNSON: No, that's not true.
MR. BARANOWSKY: The NRC won't be taken a corrective action.
DR. BARTON: But the licensee has some actions.
MR. BARANOWSKY: The licensee, when they find tag-out
problems, will be taking a corrective action. The theory is if you let
the little things go, they become bigger, and then they become bigger,
and then they become bigger.
DR. SEALE: Indeed, one of the concerns is that by the time
they reach -- things reach the level of recognition on the NRC's screen,
you may already be on a slippery slope that may be very difficult to
turn around. It is my understanding that INPO is looking at indicators
at a much more sensitive level, as many as a hundred per plant in some
cases, and that they would hope that using those indicators, the
licensees themselves, with INPO's help, would be able to anticipate
things so that you never let any -- nothing you ever saw got out of the
green.
MR. BARANOWSKY: Yes.
MR. GILLESPIE: There's actually -- there's INPO -- WANO and
INPO are kind of one in this -- and WANO's had a lot of input into an
IAEA document which is currently circulating in draft -- I don't believe
it's available for the public -- which takes the next step that we've
said we wouldn't take. Not only is it indicators from the facility, but
it talks about safety culture --
DR. SEALE: Yes.
MR. GILLESPIE: And it talks about indicators of management.
So there is internationally and nationally the industry's themselves
looking at it. The IAEA document is a document for station personnel,
and specifically in the front of it it says this is not intended for the
regulator.
DR. SEALE: Yes. Yes.
MR. GILLESPIE: So there is that activity.
DR. SEALE: Yes, but it does address these --
MR. GILLESPIE: Yes.
DR. SEALE: These culture-related concerns that we've raised
in the past in some of our discussions.
DR. APOSTOLAKIS: Well, again, it's okay for INPO and the
industry to do things like that. When we do something, I mean, we have
to be a bit more careful. And -- let me put it in a different way. I
get the impression that we are changing, you know, it is risk-informed
and so on, but we still -- we don't want to let go, in other words. We
still want to get involved in detail.
A bolder approach might be to say regulatory actions, none
in the first column, none in the second. Just be informed of what's
going on, and then start worrying about it at 3. And if they have
tagging problems, it's their problem, you know. We don't get involved
in these things.
MR. GILLESPIE: Well, I think, you know, we've kind of got
-- now we're stuck in the compromise position we're in. We have a set
of regulations and requirements currently out there. An inspector walks
through the plant, and while he's looking at risk-significant systems
and risk-significant spaces, and he's looking for safety problems, he
sees that tags are mistagged and valves are mislocked. It doesn't rise
to being -- it doesn't negate a system, doesn't negate a train, doesn't
maybe rise to that level. Does he walk away and ignore that and not
tell anybody? That's what you're suggesting, George? We can't tolerate
that. He has to document it.
Now how we dispose of it becomes the question. And what
we're suggesting here as the disposition is it gets turned over almost
as if he was a QA inspector and walking through the same space, he gets
integrated into the licensee system. Now we're trying not to put undue
pressure or undo priority on it, and in fact gets in their system and
gets fixed in accordance with whatever priority system they're using.
DR. BARTON: It's level 4.
MR. GILLESPIE: It's like the level 4 policy statement going
through now.
DR. BARTON: Right.
MR. GILLESPIE: I mean, that's the approach. The second
column on this, the darn senior resident lives at the site. Him
finding, asking, you know, why did you bust this threshold, when he's
already there, doesn't -- is kind of a real measured response. I mean,
I would be hard pressed to have someone bust a threshold and have one of
the two residents we've got at the site say I'm not going to ask. So
it's a real measured response. It's kind of proportional to the idea of
busting one threshold.
You have your resident at his normal weekly meeting and they
generally, although they're at six-week reports, they generally have
kind of a debriefing weekly, and what their findings are says hey, you
know, your quarterly report showed this. What's going on here? It
seems like a very measured response. Could it be none? I don't know
how we'd administer none and tell a resident not to ask that question.
DR. MILLER: These responses you judge are minimal.
MR. GILLESPIE: Yes. Oh, yes, column 2 is very minimal.
DR. MILLER: I don't know how you'd define it, but --
MR. GILLESPIE: It is.
DR. MILLER: They would just be routine, the senior
residents walking around or the residents.
MR. GILLESPIE: Right.
DR. MILLER: And we need to document it, because it does --
MR. GILLESPIE: Yes.
DR. MILLER: Break the written rule, but it has no effect on
risk.
MR. GILLESPIE: Right.
DR. SEALE: You're letting the licensee know that the
inspector is not brain-dead.
MR. GILLESPIE: Right. The other thing is you're letting
the public know that we're still looking. Now we've got another
audience. It isn't just us and the licensees. And, you know, one could
say well, the inspectors won't bother writing any inspection reports
unless they found a complete cornerstone completely degraded. And
that's not acceptable in the public nature of our business.
DR. MILLER: But the burden on the licensee in column 4 is
probably equal if not somewhat less than it is today. Is that right or
not?
MR. GILLESPIE: Yes. In fact, at the Commission meeting --
yes, column 4 -- at the Commission meeting said --
DR. MILLER: Column 2; I'm sorry.
MR. GILLESPIE: Oh, okay. I was going to say the
Commissioners actually commented on column 4 and 5. They spent some
time on this and said move the Commission over to column 4. That was
their comment. So, yes.
DR. MILLER: But column 2 is less than it is today, or equal
--
MR. GILLESPIE: Oh, yes. I would expect that column 2 would
just crossing one threshold where there's a readily available
explanation would not be sending a three-man team out to investigate it.
The first thing is the SRI, what's happening here. Do you have a
reasonable explanation? If the answer is yes, he documents reasonable
explanation. The other nature of the public agency we are, we're going
to be publishing these PIs quarterly. We're going to be -- it would be
irresponsible to publish a PI that shows a threshold crossed and have no
text that said here's what it was, and it appears to be okay.
DR. SEALE: I'm intrigued by that bottom row down there
where you have a differentiation or a dividing line between regional and
agency review. You've got a pilot coming up. Will all of those pilots
be in one region?
DR. BARTON: They could be in all regions.
MR. JOHNSON: No --
DR. SEALE: One in each?
MR. JOHNSON: Two pilot plants in each region.
DR. SEALE: Yes. Okay. Some in each region.
MR. JOHNSON: Eight.
MR. GILLESPIE: Eight pilots.
DR. SEALE: Some in each -- okay. It's going to be very
interesting to see the degree to which unanimity of what that means
emerges from the different regions. And that's going to be a job for
you, I think.
MR. GILLESPIE: Consistency --
DR. SEALE: Yes.
MR. GILLESPIE: Well, the starting point is we'll have
something we don't have today.
DR. SEALE: Sure.
MR. GILLESPIE: We've at least got a first shot at a scale.
Now we can adjust the scale in effect. That's the kind of comments
we're expecting to get. But at least we have a starting point to
measure people against. One of the major criticisms of the whole
oversight assessment process was we didn't write our criteria down up
front, and so here we're attempting to do that.
DR. SEALE: You may have heard that from us, too.
MR. GILLESPIE: Yes.
DR. APOSTOLAKIS: One of the things that still is not clear
to me, Frank, and maybe you can clarify it, is okay, you go down and you
do the inspection and the licensee -- the performance indicator on
scrams is high, above the threshold. And the licensee says there is
nowhere in my license that says that I have to keep this below 3. I'm
not going to take any corrective action. It was nine from Day 1. Why
are you asking me now to change it? What is the new regulation that's
forcing me to do this. And you say PRA.
MR. GILLESPIE: No, we're not going to ask them to change.
DR. APOSTOLAKIS: Okay. You're not going to say PRA.
MR. GILLESPIE: No.
DR. APOSTOLAKIS: What would you answer -- how would you
answer?
MR. GILLESPIE: At that point we would end up doing
additional inspection, since he's refused to do a root-cause analysis,
and the inspection would likely be on his root-cause analysis problem
identification process, and we would then leak that back to regulatory
requirements.
MR. JOHNSON: Right. What caused the scrams.
MR. GILLESPIE: And if he's not violating any regulatory
requirements, then you're right.
DR. APOSTOLAKIS: That's right. That's what I'm saying.
MR. GILLESPIE: The nine scrams is an indication we need to
look further and tie that to what requirements are either being violated
or what new requirements are needed. We're not going to force him to
meet the threshold. But if anyone has nine scrams in a year --
DR. APOSTOLAKIS: Let's say you find that no regulatory
requirements are violated.
MR. GILLESPIE: Oh, we could find that, in a difficult
startup after a long shutdown.
DR. APOSTOLAKIS: What was the frequency of scrams in the
United States 15 years ago?
DR. BARTON: About nine.
MR. GILLESPIE: Yes, nine.
DR. APOSTOLAKIS: Nine a year.
MR. GILLESPIE: Eight, nine a year.
DR. BARTON: That's in the ballpark.
DR. APOSTOLAKIS: Once we're licensed, the guy says it was
nine then, it's nine now.
MR. BARANOWSKY: Yes, but we have both Appendix B and we
have the maintenance rule, and both of those regulatory requirements
force you to take corrective actions when your reliability or
performance is not up to what's expected. In the maintenance rule we
have some targets in it for you to meet. And clearly this would be
outside of what would be called for in the maintenance rule, and
certainly Appendix B has elements of quality in it that this would
challenge.
DR. APOSTOLAKIS: Are the targets of the maintenance rule
utility-selected?
MR. BARANOWSKY: Yes.
DR. APOSTOLAKIS: Okay.
MR. BARANOWSKY: And I can tell you one thing, they're
certainly less than nine.
DR. SEALE: Yes. I think there's another way to look at
this. Nine years -- or ten years ago we had nine scrams. Ten years ago
we had the inspection system that we're still living with today. When
we don't have nine scrams a year, maybe it's time to take a look at the
inspection system and bring it into more of -- well, in tune with what
the practices are today. But you still use that as the thing that opens
the door to the kind of inspection that is more reminiscent of what you
have today if somebody does show up with nine scrams.
DR. APOSTOLAKIS: My fundamental problem is that the basic
regulations that people have to comply with are not risk-informed.
DR. SEALE: True.
DR. APOSTOLAKIS: And the inspection oversight program is
risk-informed, and nobody sees a possible conflict here?
MR. GILLESPIE: No, we do. We do. In fact, we highlighted
that in the Commission paper as a conflict, because we are right now out
in front of the regulations.
DR. APOSTOLAKIS: Yes.
MR. GILLESPIE: Absolutely.
DR. APOSTOLAKIS: We are changing the way we oversee things.
MR. GILLESPIE: And --
DR. APOSTOLAKIS: Without knowing why.
MR. GILLESPIE: No.
DR. APOSTOLAKIS: We have empirical knowledge --
DR. BARTON: So we shouldn't improve this process --
MR. GILLESPIE: We know why we're changing, but -- and
that's the difficult part. The only step we can take is, we can't
enforce thresholds. Thresholds are an indication as to when we go
further. And then we're forced, if you would, into making a link of
what is the requirement that links to the root cause.
MR. JOHNSON: We're forced to use the existing regulatory --
MR. GILLESPIE: Structure.
MR. JOHNSON: Structure to take action against the licensee.
MR. GILLESPIE: I would love not to have to do that, but
that's where we are. That's our only option.
DR. KRESS: The link is that you're assuming that you have a
body of regulations there and you're assuming that if a plant does meet
all those regulations that are out there, then it is more likely in the
green zone?
MR. GILLESPIE: That's inherent in the assumption.
DR. KRESS: And so you've got indicators that you don't have
to go in and see if they meet all the regulations, you just go look at
these indicators. Now if somebody gets out of them, your assumption is,
well, more than likely they don't meet some of these regulations, so
we'll go in and see.
MR. GILLESPIE: Right.
DR. KRESS: And you may find that they still meet the
regulations.
MR. GILLESPIE: That's true.
DR. KRESS: Or you may find that they don't. And your
response is going to be depending on what you find by that additional
inspection.
DR. APOSTOLAKIS: What do you think the response --
DR. KRESS: So I think they're forced to go in and their
response has to be well, do they meet the regulations or not? And maybe
those regulations will change over time, and so forth, but I think
they're right in saying --
DR. APOSTOLAKIS: What do you think the response --
DR. KRESS: That they're boxed in to saying meet the
regulations.
DR. APOSTOLAKIS: What do you think the response will be if
they find that they meet the regulations but two of the performance
indicators are above the threshold?
DR. KRESS: That's a good question. That's where the
conflict's going to be. What do you do about that? I think they have
to backfit or --
MR. GILLESPIE: No --
MR. BARANOWSKY: That's so highly unlikely, because what we
currently have today is performance that would be well within the green
band in which we still cite licensees on Appendix B violations. It's
exactly the opposite problem as the real world than you're postulating
right now occurring in the future.
DR. SEALE: There's also another response, and that is while
they may be complying with what the regulations state, they're certainly
not meeting the utility peer group's aspirations for excellence, and the
other utilities are going to be all over them, because they have the
same indicators, they're looking at them for the same kinds of concerns,
and nine is not a satisfactory way of doing things in their world where
they think in terms of excellence and performance.
DR. MILLER: And that peer pressure by some is perceived as
being more of a hammer --
DR. SEALE: That's right.
DR. MILLER: Than is the regulatory pressure.
DR. APOSTOLAKIS: What kind of world are you guys
describing? Plato came again down here and I didn't realize it. This
is so platonic. This is the ideal world. You know, you will be shamed
and your peers will come and correct you? Has that happened before?
DR. SEALE: It's happened before.
MR. BARANOWSKY: Actually the PUCs are also important here.
When the PUCs end up seeing something happening, there's a question as
to whether or not --
DR. SEALE: That's right.
MR. BARANOWSKY: There should be changes in rates and so
forth.
MR. GILLESPIE: And all of that, we're still an independent
regulatory agency, and we're setting our -- the fact that our thresholds
and our perspective is coincident with some societal pressures is very
nice in a system point of view, but clearly these are our thresholds in
our action statements.
DR. KRESS: When you come to enforcing things, I think you
really are bound by the existing regulations.
MR. GILLESPIE: Oh, yes.
DR. KRESS: If you end up having this conflict, George,
where -- and I'm sure it can exist -- where the risk performance metrics
appear to be way out of line and they're still meeting all the
regulations, I think you've got a problem with the regulations.
MR. GILLESPIE: The ideal piece where this would be
explained away --
DR. KRESS: Fix the regulation.
DR. BARTON: Fix the regulation. Right.
DR. KRESS: But that's a -- you know, that's a backfit and
that's a rule and you have to make new rules. But I think that's where
you have to attack. If you find this to be a real problem that's
pervasive, I mean, I think you can stand it --
DR. APOSTOLAKIS: Or you can live again with an unfortunate
situation like after the IPEs go down we realize that 19 PWR units were
above the goal but they meet the regulations. There's nothing we can
do. Right? Is that a happy situation for us?
DR. KRESS: Probably not.
DR. APOSTOLAKIS: Probably not.
DR. BONACA: I just have a comment I'd like to make.
DR. KRESS: The backfit rule is one of the things that's got
us into this box. You know, if you can't satisfy the backfit
requirement, you can't make a new rule. And that's one of the things
that have us in this box.
DR. BARTON: Mario?
DR. BONACA: Yes, I had a comment regarding -- one thing
that, you know, looking at these performance indicators, in fact, I
don't know how many plants would not meet these performance indicators
quantitatively. I mean, I haven't seen many plants that have more than
three scrams per year, I mean, in fact.
And the concern I have is the reverse one, which is the one
that rather than being an objective process again there is a lot of
reliance on the complementary or supplementary inspections. And so the
burden becomes again on the staff to show that in fact it is an
objective process that you are using.
Maybe the question I have is how did you get these
thresholds, and they seem to be in fact much more relaxed than what you
have from INPO.
DR. APOSTOLAKIS: In fact, they go to page H-18.
DR. BONACA: I know.
DR. APOSTOLAKIS: You will find one, two, three, four, five,
six, seven, eight plants whose number of unplanned scrams for a year is
above the threshold.
DR. BONACA: Yes, that's right.
DR. APOSTOLAKIS: By design. So what do these eight plants
do?
DR. BARTON: What they'd better do is have a corrective
action program to reduce the amount of scrams, but if you're shut down,
you're not going to be economically feasible, pa da pa da pa da, down
the path you go.
DR. APOSTOLAKIS: As a result of oversight?
DR. BARTON: Pardon?
DR. APOSTOLAKIS: As a result of oversight they have to do
that?
MR. GILLESPIE: No, there's going to be logical explanations
in some cases. If you have a plant that's been shut down for two years,
of which we happen to have several, and you go to start up, it is not
unprecedented even in the first month you would have three, four, five
scrams. But you'll generally identify exactly what it was very rapidly,
and you'll correct it and move on. That's an explanation.
DR. APOSTOLAKIS: Incidentally --
MR. GILLESPIE: It means we have to understand and explain.
It doesn't mean we have to take action. And the licensee just has to be
able to explain it and it has to make sense and be logical.
DR. APOSTOLAKIS: I don't know. I mean, that is related to
Dana's question earlier about not regarding the PRA. I don't know how
NEI produced these figures. Was it based on one year's performance?
MR. BARANOWSKY: No.
DR. APOSTOLAKIS: Was it based on a lot of years'
performance?
MR. BARANOWSKY: No. It was based on I believe five years'
performance and it ran from approximately 1992 through '97.
DR. APOSTOLAKIS: So for each plant it was the same
denominator, five years.
MR. BARANOWSKY: It was on a per-year basis, but they
collected for that five years, and then they took the peak value for
each plant in that five-year period.
DR. APOSTOLAKIS: Oh.
MR. BARANOWSKY: Now there is a couple of things that are
important.
One is performance has been improving over that period and
since that time, so there's a reasonable likelihood that there would be
less hits in any given year than would be indicated by the 95 percent --
DR. APOSTOLAKIS: Because I was wondering if you have five
years' worth of data for each plant, I mean the codes are available
now -- a two-stage Bayesian thing would be --
MR. BARANOWSKY: We got 'em.
DR. APOSTOLAKIS: You have them already?
MR. BARANOWSKY: We can apply a lot more sophistication than
we did.
DR. APOSTOLAKIS: Yes.
MR. BARANOWSKY: But we kept it very simple for this
go-around. We wanted to keep the mathematical eloquence to a minimum so
that it could be more easily understood by the everyday inspection kind
of person and licensees without going through a lot of arguments about
is this a good computer code, is that a good mathematical method, is the
prior proper and so forth -- we wanted to get away from that for now
because we just wanted to try this out.
It will evolve into something a lot more sophisticated as we
get more plant-specific and more risk-based and we improve on the
indicators, but for now, it is a very simple approach.
The thresholds there have minimal risk implications. We ran
some calculations using those 13 models I told you about and for that
particular threshold we didn't see anything that approached the 10 to
the minus 5 change in core damage frequency from what would be a mean
type of performance over that period of time, when we moved up to that
threshold.
In some cases we put all the plants' performance at the
threshold to see if the risk was high, so it was a relatively low risk
associated with operating within that performance band, and relatively
low risk, what I mean there is what we have today. That is low risk.
Increases in risk when one moves up to those limits are
small enough that they wouldn't cause us to be concerned with regard to
the kind of thresholds for risk applications Regulatory Guide 1.174.
MR. JOHNSON: So let me just tell you what we'll see --
DR. APOSTOLAKIS: Before we go to that though, let's talk
about the substance.
MR. JOHNSON: Which one, George? Back to the matrix?
DR. APOSTOLAKIS: This is pages 14, 15 of the Attachment
2 -- cross-cutting issues.
MR. JOHNSON: Yes.
DR. APOSTOLAKIS: Can we discuss those?
DR. BARTON: Go ahead.
DR. APOSTOLAKIS: You have a very nice discussion of safety
culture which is under the name Safety Conscious Work Environment, but
then you dismiss it -- "No separate and distinct assessment of
licensees' safety culture is needed because it is subsumed by either the
PIs or baseline inspection activities."
Does everyone agree with that?
DR. BARTON: Well, I think they are driven to that based
on -- it's the policy of the Commission is if you don't go and inspect
safety culture, management effectiveness and all that stuff I think you
are forced into this approach.
DR. APOSTOLAKIS: Why don't we say that?
MR. BARANOWSKY: That's not really -- I mean that may be one
reason why you get to that conclusion, but that's not why we got there
when we did this work.
The real reason we got there is we're looking at outcomes
that affect the objectives of the cornerstones that we identified.
DR. BARTON: Okay.
MR. BARANOWSKY: And our conclusion was that this is an
underlying cause to outcomes. As long as we measure the outcomes, which
is the performance indicators and the inspections, and if those outcomes
are acceptable we don't have to go and check into whether or not they
have the right attitude at their plant.
DR. APOSTOLAKIS: So you are measuring human performance?
MR. BARANOWSKY: Sure. People make mistakes and cause
plants to trip. People make mistakes and cause equipment to fail or be
unavailable. To the extent that those things don't occur, we are
satisfied with the human performance.
Now thee are some things that we can't measure with the
performance indicators like how will the operator respond to a real
accident.
That's where we would have inspection or related activities
associated with the operator licensing program. That is a complementary
area for inspection.
DR. APOSTOLAKIS: Well, let's see what you are saying about
post-initiator operator actions.
You admit that they are far less frequent -- that's page
14 -- while initial and requalification examinations provide a
predictive measure of operator performance during off-normal and
emergency operations, follow-up inspections of risk-significant events
will provide a more direct indication of the adequacy of post-initiator
human performance.
In other words, you are saying that, look, we really -- I
mean there is no way we can know how the guys are going to perform.
MR. BARANOWSKY: Actually, what we are saying is that we
have had a very successful operator qualification and training program
and that unless we have objective information or evidence given to us as
a result of abnormal events that really challenge the operations and
show that they didn't handle the situation properly, we are going to
rely on that program to give us confidence that that is an acceptable
area of licensee performance.
We had some incidents occur in which operators have made
some errors during these incidents, and those would be the things that
we would look into to see whether or not those are an occasional human
error problem or whether they are indicative of deeper problems with
their program.
DR. BARTON: I think inspectors go look at, if you put those
into the corrective action program, they go look at what is going on at
the site with respect to improving human performance and reducing
operator error.
The inspectors look at that, so that's in real life.
DR. BONACA: Yes, the industry is moving, however, in a
direction where you can group elements of human performance,
safety-conscious work environment, problem identification in safety
culture and you have very clear indicators of it.
For example, you can look at operator challenges, temporary
mods, operator work-arounds. Those are typical indications of poor
culture if you don't correct those, or for example, corrective action
programs that doesn't solve you issues or doesn't have the proper root
cause. That is an indication of that.
They may be worthwhile in fact at some point to group this
other indicator that you have inside here under some heading -- because
the industry is moving in that direction and I think they should be
encouraged to move in that direction, and I think something could be
done inside here.
MR. BARANOWSKY: In fact, we would encourage them to do that
kind of thing because that is like an early indication of an indicator,
if you will.
DR. BONACA: That's right.
MR. BARANOWSKY: And what they are trying to do is maintain
their performance above that threshold and so by having these kinds of
additional performance indication or inspection activities that they
perform themselves, which go really beyond what the NRC does, they have
the opportunity of controlling their own destiny.
DR. BONACA: That's right.
MR. BARANOWSKY: That's really the idea that we are trying
to foster here is that the NRC does one to five percent of all the
inspections and evaluations that go on at a plant. The licensee does
the other 95 or 96 percent or whatever it is, and it is their ability to
do that stuff correctly that we are trying to take sample information on
to see what the indications are.
MR. JOHNSON: Right. In fact, the industry was very
interested in looking at the things that you have just mentioned. They
were also very interested in having us not look at them if they don't
reflect themselves in actual performance, and so that is sort of how we
have come out.
DR. BONACA: But you do have significant expectations in
safety-conscious work environment and to me it seems you should
translate those expectations in measurable things. You know, I gave you
some example of those because there are relationships there, so I
understand the interest of the industry also not having you quantify
some of those, but to the degree to which you are interested in those
specific assessments, some correlation with the specific metrics could
be important.
DR. APOSTOLAKIS: When the Commission decided that it's none
of our business to look at safety culture, they interpreted safety
culture the way you interpret it.
Let me be a little more clear. If you read reports from
Vienna on safety culture, safety culture consists of two elements they
say. One is the organizational structure and the other is the
attitudes, the norms and so on.
You seem to be making a distinction. You are saying
safety-conscious work environment, otherwise known as safety culture,
really refers to the norms and attitudes because you still worry about
the organizational structure on the problem identification and
corrective action programs.
What is not clear to me is whether the Commission also
thinks about it that way.
DR. BARTON: That's a good point.
DR. SEALE: Well, didn't they really say don't look at
management --
DR. BARTON: Don't look at management effectiveness and
competence.
DR. APOSTOLAKIS: But what does management mean? It means
your work processes and the way you manage business or only the
managers?
DR. BARTON: I think it is the former.
DR. APOSTOLAKIS: Well, then, how come we have a full
section here on problem identification that says the scope of problem
identification corrective action programs includes processes for
self-assessment, root cause analysis, safety committees, operating
experience, feedback and corrective action.
These are processes. They are part of the organization
structure, so from this document I get the impression that this is
outside safety culture, whereas the INSEC documents is part of the
safety culture. You have to make sure you have good processes and so on
and so one, which is fine with me, but now that explains the
Commission's decision because I am a little puzzled by the decision if
it includes processes for self-assessment root cause analysis and so on.
I mean obviously we do care about these things --
DR. SEALE: It's my impression, George --
DR. APOSTOLAKIS: -- so these are not safety culture.
DR. SEALE: It is my interpretation, this personal
interpretation, that they don't -- do not want to be in the position of
having appeared to have told a utility, whether it has a vice present or
a -- whatever -- and exactly how many people report to that person and
whether their training organization reports to a Nuclear Executive Vice
President for the whole organization or the guy in charge of a given
plant, those kinds of questions.
DR. APOSTOLAKIS: Okay, so somehow we have to make a
distinction.
DR. BARTON: George, I think if you look at this, this is
really looking at safety culture, if you are looking at the thing that
you read.
DR. APOSTOLAKIS: Yes.
DR. BARTON: If you look in the corrective action program,
self-assessment, root cause analysis, safety committee, you are really
looking at what is the safety culture of that organization. You really
are. I don't care what you call it.
DR. APOSTOLAKIS: And that is what INSEC said, but the
previous section here said "a safety-conscious work environment, also
referred to as a safety culture" and then it concludes by saying, "In
short, no separate and distinct assessment of licensees' safety culture
is needed."
Then there is immediately another section that says, ah, but
for processes that have to do with identification and correctional
programs, we worry about those, so I think a lot of this is semantics.
By the way, I don't disagree with you and that clears up a
lot of things in my mind because I think Bob is right, Bob Seale, that
the Commission did not want to appear as if they were interested in
which Vice President reports to the President.
DR. SEALE: And the driving force behind that is that there
are documentable cases in the past where certain people who are
employees of the Commission maybe overstepped their authority and
appeared at least to have interfered with that sort of thing and it did
raise a considerable amount of ire and fury and heat.
DR. APOSTOLAKIS: Yes.
DR. BARTON: That came out in the Towers-Perrin Report.
DR. APOSTOLAKIS: And there are other parts of the safety
culture that I do worry about. For example, operator training and so
on. Yes -- in the broad definition, that is part of it, so we really
are excluding here, and I think rightly so, the attitudes, goal
prioritization, things that you can't really see, right?
MR. GILLESPIE: Right.
DR. APOSTOLAKIS: So maybe just change the words a little
bit to make sure that that is what you are doing.
This is not really -- safety-conscious work environment, the
way you describe it here, is not also referred to as "safety culture."
MR. GILLESPIE: I don't think we used the words "safety
culture" but if we did we shouldn't have.
DR. BARTON: Yes, you do.
MR. GILLESPIE: We do.
DR. BARTON: You have got it thrown in there a couple of
times.
DR. APOSTOLAKIS: Page 14.
MR. BARANOWSKY: That's a throw-away. I am glad to have
something the ACRS thought of something here.
[Laughter.]
DR. SEALE: You put it in quotes so it is --
MR. BARANOWSKY: I admit it -- we made a mistake.
DR. APOSTOLAKIS: I am not asking you to admit you made a
mistake. It's good that you did.
[Laughter.]
DR. APOSTOLAKIS: But it is a very important part though
that you really worry about -- in fact, if you go to a plant or your
resident inspector realizes there is no mechanism, there is no process
for operating experience feedback, he probably won't like it. Right?
DR. SEALE: Well, any root cause analysis that doesn't
address concerns having to do with human action is idiocy.
DR. APOSTOLAKIS: Which is most root cause analysis.
MR. BARANOWSKY: When you cross the threshold and the
resident inspector goes and says why is this occurring, and they see
that they don't have a good root cause analysis activity, then you are
getting into these things, but that is a cause of the problem kind of
thing. That's why we are saying you won't have an indicator for that by
itself because if you did I mean that would be fine but we don't know
how to relate some of these things to the cornerstones directly. There
isn't a simple algorithm for doing that.
DR. SEALE: The thing you detect is the consequence, not the
action.
MR. BARANOWSKY: Right.
MR. GILLESPIE: And in fact independent of which cornerstone
or which PI or which inspection activity might cause us to engage
further in doing a root cause analysis, the end of the root cause
analysis might find that problems exist in places not first indicated,
because at that point you are in an analytic mode and how pervasive is
it is the kind of question you are asking, how limited is it, was it
isolated, was it not isolated, so you roughly go from one kind of
inspection process and procedure to a different one when you get to that
point.
DR. BARTON: I have seen you put your light out. Does that
mean you are done?
MR. JOHNSON: That means --
MR. GILLESPIE: I think he gave up on the last viewgraph.
He's not going to put it up.
[Laughter.]
MR. JOHNSON: We have moved beyond the last one. I can put
it up but I don't see a point.
DR. APOSTOLAKIS: I am done with my questions, Mr. Chairman.
DR. BARTON: You are done with your questions? All right.
Throughout the document, and I think you mentioned this
morning you still have got some work to do and there's some not just
shoe-strings to tie but you actually have some other open issues that
are mentioned throughout the document.
When do you plan to have that supplemental information
ready? When can we hear? Will that be after pilot programs, after this
thing goes out for comment? What is the timing on when we'll hear what
is missing in here?
MR. GILLESPIE: Let me ask Al Madison to come up -- Alan is
the Team Leader for the Transition Task Force -- and ask him to address
part of the schedule.
DR. BARTON: Is that the last slide you weren't going to
show?
MR. GILLESPIE: No. This is one that wasn't even on the --
MR. MADISON: I am Alan Madison with NRR. We have met
before. Your specific question, so I can address it?
DR. BARTON: You go through the document and it says thee is
more work to be done; we've got to do something here -- throughout, all
right?
My question is when will you fill in those blanks and when
will you have that work done?
MR. MADISON: Well, I am glad you recognize there is a lot
of work to be don, a lot of work left, and we're, as the paper alludes
to, we are going to implement or establish a task force which we have
begun to do. Because of the broad scope of activities that are involved
there will be a lot of folks involved in that task force.
Some of the items -- Mike, you'll probably want to list the
key transition tasks.
These are some of the key tasks we think are involved in
there and you probably noted a few others. But we have broken it out
into the work remaining and developing the procedures and the policy
documents that are going to need support both the inspection procedure
and the assessment procedures. Enforcement policies also fit into
there. The performance indicators, how we are going to collect the
performance indicators, how licensees -- the format they are going to
report them in. What are we going to do with it? How are we going to
get that out to the public? All that is yet to be developed, the
support infrastructure for those changes, as well as other changes in
various areas.
We have got a lot of training to do, we have got
development. Some of the jobs that were left to be done, including --
well, it was called benchmarking, we are now really calling that a
feasibility study. Is the process -- is it feasible that we can take
the performance indicators, that we can take inspection data and compare
those and come up an assessment in the cornerstone areas? We plan to do
that during the transition.
One of the biggest things, and that would be the next slide
with the transition strategy. One of the biggest things we recognized
right away is there is a lot of communication and change management
issues that we need to deal with in this. And so we, as part of
establishing the transition task force, we have named what we call a
change champion who is going to be kind of our voice out in the industry
and the agency, and to the public, and that is Sam Collins, Director of
NRR. He will be our primary communicator of what we are doing and where
we are going.
To assist him, to help him out in that, and to help us out,
we are in the processing a change coalition, which we feel are the -- we
are hoping these folks are what we are calling the opinion leaders in
the various areas, and they may or may not be in management. We are
focusing right now on the regions, but we are also going to develop some
folks here in headquarters that will be on that change coalition. We
see them as our primary source of information back from the field, how
the transition, how the processes are being perceived by the people in
the field, and changes, recommended changes and improvements that we can
make to the process.
We are also going to try to leverage off of this a group of
individuals, the senior individuals in this change coalition, who may be
able to provide some management oversight and some management control
for the process implementation that we go through.
We also recognize there is industry and other external
stakeholders, such as the states, that we are going to have to
communicate with and bring on board. And we are developing some -- or
we are working with some opportunities, I guess, where we can take some
action in that area.
One of the transition task force members will be -- that
will be his primary job, to look after the change management and the
communication issues that we have. Another member of the task force
will look at the training issues that we have and develop training for
the staff. And we will probably end up developing some joint training
with industry to get the plants up to speed on the performance
indicators, how they are going to report them, and how we are going to
-- what we are going to do in that area.
Now, the schedule. There's what we see as our major
milestones as we go along. Now, throughout that time period we expect
to be communicating and keeping the ACRS informed, and we will try to
schedule some opportunities that we can do that. We look at having a
Commission paper sometime in the March timeframe and receiving final
Commission approval on the recommendations, not on the whole package.
We are not going to have all the details filled in.
We plan to start a pilot in June. Now, to support the
pilot, we are going to have to have a good draft of the inspection
procedures and the program documents that will support those inspection
procedures. And we are also going to have to have provided some
training to those selected pilot plants and the NRC staff that is going
to be involved in that, that includes the senior resident inspectors,
the branch chiefs, and probably some DRS personnel, and others in
various areas.
And we are also looking at, in October, providing a workshop
that would be a joint industry-NRC workshop, public workshop, to provide
training for everybody involved.
Does this sort of answer some of your question?
DR. BARTON: That's part of it, Alan, but I was under the
impression that you owed the Commission some more supplemental paper,
some more documents by next month sometime.
MR. MADISON: The March paper, I think, promised -- the
March paper is -- the promised March paper has to deal with completing
some of the items, I think, and the Office of Enforcement is to come up
with a recommendation, a final recommendation of where they went ahead
with the enforcement policy. I think we are going to come up with a
better draft. We didn't provide them any documentation on what we are
going to do with the inspection data to relate it to performance
indicators, and we owe them a document that at least outlines the
concept that we have and gives some direction of where we are going with
that. I am not sure how much detail we owe them on that.
DR. BARTON: Well, are we going to get to see any of this
stuff? Do you need a letter from us before this goes out for public
comment or what? Where are we in your kind of chain here?
MR. GILLESPIE: Let we tell you where are in the cycle of
things. It is out for public comment. To maintain this aggressive
schedule, it went out for public comment before the Commission even got
it, and the Commission went on faith that the staff had a good document,
which was -- I appreciate that because it allowed us to move forward.
We will be going back to the Commission, I believe the
meeting is tentatively scheduled for March 18th. About the first week
in March, we will be going back with a smaller paper -- we are not
sending 400 pages back up -- that will deal with the exceptions and the
comments. And that will include dealing with any comments now we get
from an ACRS letter that comes to the Commission from this meeting in --
DR. BARTON: February.
MR. GILLESPIE: February, I forget when the date is.
DR. BARTON: Fourth, 5th, something like that?
MR. GILLESPIE: Fourth or something. Yeah. So the ACRs
letter will be factored in to this comment period.
DR. BARTON: We are going to really -- we are really
commenting on this policy.
MR. GILLESPIE: Yes.
DR. BARTON: SECY-99-007, January 8th document.
MR. GILLESPIE: Yes. Yeah. And now the hole, the real big
hole that the Commission focused on was this subjective or potentially
rule-based scale that an inspector could rapidly assess his results
against and know whether -- I just turn this over to a licensee, or does
it warrant further pursuit. That was the significant hole that was
identified. We are going to be working on that.
As a minimum, our best insights on that will have to go in
the March paper. As a maximum, maybe we will get it done sooner and get
it out for comment to everybody, including the ACRS for information and
comment.
DR. SEALE: When the March paper is available, we will get a
copy of that?
MR. GILLESPIE: Yes, absolutely.
DR. POWERS: Mr. Chairman, I think we should devote some
time to this letter. I think we have to go beyond just the details and
the specifics, but point out what we think are the strengths of the
approach that they have adopted in what I think is an extraordinary
piece of work.
DR. BARTON: Yes.
DR. POWERS: And that stands as a template for a lot of
other activities that the NRC is undertaking.
DR. MILLER: However, you are going to -- you recommend we
comment on the process of getting to this document as well as the
document itself?
DR. POWERS: That's right. I think that -- I think some of
the ideas that emerged in this document are strokes of genius.
DR. SEALE: They are worthy of emulation.
DR. POWERS: And could be used in a lot of other contexts to
get over some of the rough spots of moving in the direction of more
risk-informed regulations.
DR. SEALE: I have one comment --
DR. POWERS: Other than that, we ought we praise the
document wherever we can.
MR. GILLESPIE: Well, we also appreciate, because this sets
us up for the future. I promised research, you have always got to give
them a plug, this is the beginning, and we know there's weaknesses in
the approach that deserve future consideration. So any insights as to
where, first, future consideration might go would also be appreciated.
Because once this goes in place, the next step is figure out to make it
better.
DR. SEALE: I have one comment on the schedule or the
outline of future action you made. I heard about task force members or
team members. I heard things about piggy-backing or some term like
that, some administrative or leadership, or something like that in all
of this process, and I never did hear any word about the regional
administrators. And it seems to me that while this is going to be in
the pilot stage, only a small part of what is going on in each region,
that it is very important that the regional administrators be brought up
to speed on this process very early, because they can either make it or
break it.
MR. GILLESPIE: Yes, absolutely. And that is -- I hope
nothing I said led you believe that we weren't going to involve the
regional administrators. Yes, and we are wrestling with that. And the
agency right now is a quandary, and the quandary is we have taken on, as
an institution, what the consultants tell us we probably shouldn't have,
all at once. We are reorganizing. We just went through a massive
number of retirements in the management ranks, including the regions.
And at the same time, at least as far as the regions are concerned, we
are maintaining a current program and we are trying to completely turn
it upside down. And we are trying to do it all at once.
DR. BARTON: Business as usual. Right.
MR. GILLESPIE: Yeah. And it has put strains on the system.
While we would have liked to have actually had a regional administrator
volunteer to come in and work with us for six months and run this show,
I would have liked the relief, it wasn't in the cards, because two of
the regional administrators are basically new.
DR. SEALE: Well, it is not in the nature of those beasts.
MR. GILLESPIE: Yeah. But we are fostering, as part of the
change coalition, and we are trying to -- we have been brainstorming
this. I will throw this out on the table. This is a concept. I won't
even say I have talked it over very much with people, at least we have
touched it with Sam, and that would be putting a senior level committee
together which might be deputy regional administrators, regional
administrators, or projects division directors, who are the guys who
really have to make this work when we get it out -- one person from each
region who is kind of the senior member of this concept of a change
coalition -- and we wouldn't be on it, and, actually, subject ourselves
about once every two to three weeks to using them like a murder board,
where we would spend one day briefing them, and a half-a-day letting
them tear us apart. And at the Commission table at the end, in March,
we would ask the chairman of that group to come and sit at the table
with us.
Now, that is kind of a conceptual concept that we are
looking at. We haven't really run that by the regions yet, and they
haven't volunteered for it. But that is kind of the degree of thinking,
as we are putting the task force together, that we are going to. So any
suggestions, any -- that is kind of the gelling, it is as close as we
have come right now, conceptually, to how we would both get critical,
very, very critical -- Can this be applied? Can we do it? -- advice,
and have them at the table to say, hey, they took my criticisms and they
told me why not, or they took them into account, here's my concerns that
still exist.
DR. SEALE: I have another question I want to ask, and it
goes back to -- I don't know, for some reason, I have always had a
picture of autonomous fiefdoms as you move from one region to the other,
and I don't think that is an entirely erroneous picture. In this -- in
that chart you had up there earlier, you talked about the dividing line
between regional review and Commission review. In all of these mergers,
acquisitions and things like that that have gone on, how many cases do
we now have where an operating company has plants in more than one
region?
MR. GILLESPIE: I couldn't answer. It is getting worse. I
mean with Entergy buying Plymouth, we have got Region IV to Region I,
which is the maximum extent. I don't know. We could look it up.
DR. SEALE: Yes. Well, it is interesting to me because it
seems to me that that -- and I am not sure how you get it, but it might
be interesting to try to find out, in the eyes of a victim, whether or
not they see equal treatment from the regions.
MR. GILLESPIE: Yeah, that's an interesting point. It has
happened, by the way.
DR. SEALE: I am sure it has, but, you know, I think it is
something that the regions ought to be aware of as a reasonable and
perfectly correct source of performance data that they have to be
prepared to live with. Just a wild suggestion.
MR. GILLESPIE: Yes, and actually putting on another hat.
Within the same branch, we are putting together a program performance
group of four or five people. We used to do this about four or five
years ago, got strapped for resources, couldn't do it. And with the
demise, if you would, of the end of the A&E program, we now have some
people available to get back into more of a routine -- How is the
program going? And that is a good point, because we would be visiting
some utilities and plants, and I hadn't necessarily, I don't think
anyone has suggested that we ask that particular question.
I am going to guess there is a half-a-dozen companies.
DR. SEALE: I would think so, yeah. Yeah.
MR. GILLESPIE: That cross some line, some way, shape or
form.
MR. MADISON: There is a -- NEI is having a user survey
which they are going to survey pre-implementation of this process and
post-implementation, which may address that question. We are going to
try to utilize that survey.
DR. POWERS: I want to go back to your question about the
cultural change that you have to make within the agency itself. You are
not the first institution to undertake a re-engineering of its
procedures. It has gone on long enough that there is a body of
management science that has been assembled on the difficulties of these
things. I, myself, am unfamiliar with that literature, but I wondered
if it had anything that was of use to you.
MR. GILLESPIE: Yes.
MR. MADISON: Definitely.
MR. GILLESPIE: In fact -- okay, go ahead, Alan. He is the
holder of the textbook, he has got the formula.
MR. MADISON: Well, I don't know if I have got the formula
or the answer, but we are trying to get the answer now. We are
utilizing the contractor that the NRC does, Nick Mann, as well as his
subcontractors, to give us some insight into where we ought to go.
As I said earlier, we have one individual who is going to be
on the task force, August Specter, who has a background in this area.
He is going to be the focus for this so that we have somebody who is
really concentrating on this full-time. He and I are going to go to --
actually, outside to the industry, to Salem, to try to get some input
and some insights from the changes that they have gone through and the
processes they have used.
Yeah, we are going to try to collect some of this body of
knowledge and use it.
MR. GILLESPIE: Well, no, no, no. Think about this, and it
is funny, the people from Salem were very, very open with us, and they
said we have done it twice and we messed it up the first time. And, so,
I am not sure that they might not be kind of -- when you are going for
insights, and they gave us some interesting insights and documentation
already which has caused us some thinking.
One was people want to hear from the first-line supervisor.
They don't want to hear from outsiders. Which is interesting, because
the traditional way NRC does things is we kind of collectively as a team
would have gone out to the regions and lectured the first-line people
with the supervisors in probably the same group.
And you start to develop well, we've got to go talk to the
branch chiefs and division directors separately first and make sure not
that they're saying yes, we'll do what you're telling us to do, but yes,
we understand the underlying principles and can explain it to our staff.
So some of the insights of what Alan's been doing are already -- it
slowed us down a little bit, because we said we need to think about
this. If you do it in the wrong order, you could mess it up badly.
DR. BARTON: I thought it was a surprise.
DR. SEALE: There's a variation on that that the INPO people
use in their plant assessments now, their visits, namely they use local
peers --
DR. BARTON: Right.
DR. SEALE: On their visiting team, and the presentation of
the results of the visit is made by the local peer.
DR. BARTON: Right.
DR. SEALE: Which is kind of the same thing. The guy that's
telling you the finding knows your plant. He's not somebody from Mars
that came in and --
MR. GILLESPIE: Yes. And maybe we should have known this
right away. We didn't. And to me that was, if nothing else, an
extremely valuable insight that said slow down, understand the steps,
adjust who you're talking to, you want to talk to different people with
different levels at different times.
And then you have to say what are they going to do with the
information? And the first-line supervisor, division director, do
different things. So we're trying to do it right, and in fact we're
looking at the literature and trying to use the right experts to give us
the input.
DR. POWERS: I mean, I think that's the answer is simply not
be arrogant enough to think that you know all the pitfalls to this,
because I think more than one company has had serious difficulties
making these kinds of changes.
MR. GILLESPIE: Yes.
DR. POWERS: The other thing is, of course, to recognize I
think the personality of your organization and the personality of the
Salem organization are about as orthogonal as two organizations can be.
So I laud you for taking the words from them, but I caution you they may
not be 100 percent transferable.
DR. MILLER: They say they messed it up the first time. I
don't think the second time the story's been told totally yet.
DR. POWERS: Yes. I mean, this is --
DR. MILLER: We don't know the result in the second time.
DR. POWERS: Right. This is a classic theory x and theory y
organization. They're just not going to mesh well.
MR. MADISON: But they do have some interesting concepts
that we want to find out what they -- how they really made or are making
them work and see if we can apply them to our own case.
DR. POWERS: I think you're doing the right thing, going and
collecting information from wherever you can and then use that part
that's useful to you and throw away that part that's not.
MR. MADISON: Exactly.
DR. POWERS: That's all you can do, and pray.
MR. MADISON: We may not have the answer yet, but we're --
this is probably the first time that in my recollection with the NRC
that we've made this large of an attempt at managing the change rather
than just making it.
DR. POWERS: The other piece of wisdom that I've heard
frequently expressed in this area is that it takes three years, that if
you try to bring about a change in the culture and do it in six months,
and then we'll make another change at the end of that six months, you
make no change at all.
MR. MADISON: Um-hum.
DR. POWERS: And it's -- it is so tempting for managers to
say okay, here's the time line when we will have made this change, and
lay it down by the calendar rather than the personality of their
organization, and I think that has been one of the biggest discoveries
people have made is that --
DR. MILLER: Each organization has a different personality.
I'll guarantee any university will take a lot more than three years.
DR. BARTON: Three to five years --
DR. MILLER: Some organizations might take less.
DR. POWERS: Well, I think you have certain advantages in
some organizations would change and certain you don't, and NRC is likely
to be slow, simply because conservatism has been the watchword of the
way you do things, not only within this agency but in all of nuclear
power, the whole thing is think carefully about making a step. Always
make sure you've got your foot on firm ground before you take a step
forward. I mean, we tell people to do this all the time, and that
infects the nuclear business, whereas in Silicon Valley they don't --
not too concerned about whether they're floating on air or not.
DR. MILLER: Dana, wouldn't you expect with all the changes
in the upper management they reflect and so forth, that could be in some
cases disadvantageous, and in other cases advantageous in this culture?
DR. POWERS: I think you're right that whenever you change a
high-level boss, people anticipate change and are more open to it. On
the other hand, they're always suspicious.
DR. MILLER: But aren't there like somebody reflected
there's fiefdoms, I think Bob Seale, at the different regions, which is
probably true. Now you've had two regional administrators who are
probably in control of those fiefdoms depart. If the fiefdoms are going
to change, it's a good time to change them, if indeed that's what we're
going to do.
DR. BONACA: I have one more question regarding the report.
I realize that I got this report, I've seen it for the first time just a
few days ago, so -- and the question I have is the report sets
essentially four objectives for this change in the inspection process.
One is improved objectivity, scrutability, risk-informed, and also a
secondary goal seems to be the reduction of burden. And clearly reading
it scrutability is improved, risk-informed is there, okay.
The only thing that I was asking was, I would have liked to
see some comparison of the areas of inspection that we used to do before
that would not be done by the new process and vice versa to convince me
that in fact we have enhanced objectivity and also that we have reduced
burden. Without that it was a little bit difficult to make a judgment.
And I wonder if you have done that kind of comparison, just as a
minimum, you know, large pockets of perspective, what we're not going to
look at anymore that we used to look at before, and vice versa, what is
new that we were not looking at before.
MR. GILLESPIE: Yes, let me -- we actually kind of -- I wish
I could give a flat yes or no -- we actually did a check going the other
way, and the question was would this program as proposed in the areas of
inspections catch the things that we've seen at plants in the past which
are not covered by PIs. And therefore you still see a large -- almost
the same emphasis as we have today on engineering. So the engineering
area didn't change very much.
For the other areas, we did it, but didn't write it in, and
there's a chart in there that talks about the risk-informed baseline and
section, it comes up to like 1850 hours. We actually have a table
that's in the inspection manual today that we check each year to see
that we're kind of in the ballpark on the current core inspection
procedure, which is kind of the equivalent. And for a two-unit site,
that comes out to be about 2200 hours. The best estimate on if you did
the sampling process and looked at the things that are in here was about
1850. So we are anticipating that it will be as a first estimate about
20 percent less.
But we didn't write that down as a formal report. But it's
really comparing this table to the table that's in there. And that was
done.
DR. BONACA: The main concern I had was that, you know, I
see a lot of restructuring. The question is, are we really changing
what we're going to look at? And if we are, what are we going to lose
in the process to gain in that process?
MR. GILLESPIE: Yes. If there was a major area of loss, and
this may sound counterintuitive, but I think in the whole set of
inspectable areas, the major reduction is in operational areas. I'll
give you an example relative to maintenance. The current procedures
would cause maintenance procedures to be observed on some periodic
basis, and I think it's once every two months or something now.
Well, what's the point of that? What's the safety
significance of it? I mean, those kind of questions are asked. I think
you'll find in the inspectable areas that the focus is on observing the
postmaintenance testing to assure that the proper characteristics will
work, which doesn't mean sitting there eight hours and watching a
maintenance procedure. But it does mean watching the test procedure,
which will be much truncated.
The PIs tended to be very operational PIs, numbers of
scrams, and risk insights that Research came up with focuses that --
John, help me out -- some what percentage of the risk actually looks
like in real experience came from maintenance, wasn't it? You used to
have a pie chart you showed --
MR. FLACK: Yes, that was a -- and actually that was the big
discrepancy. A lot of the events, the risk-significant events that we
were observing through the S program was coming from maintenance and not
so much from operations.
MR. GILLESPIE: Operations.
MR. FLACK: And I think that's where you're seeing the
change, the tradeoff now is they're going to be looking more at
maintenance and less on operations.
MR. GILLESPIE: Yes, and the way we look at maintenance in
this process becomes different, hopefully more intelligent way of
looking at it rather than just the gross approach of we're going to
watch this guy do maintenance on a valve from beginning to end. And you
know when we watched him do the maintenance we didn't hang around to see
the postmaintenance test? Because that wasn't in the procedure.
DR. POWERS: You have to advertise this. I mean, you just
really have to advertise that you've taken advantage of these insights
that you've been getting out of your past inspection program. I mean --
MR. GILLESPIE: And some of this is practical insights.
DR. BONACA: I'm saying that as you go forward in a
transition, I think it would be helpful to have somewhere a brief
discussion of this, because you are leaving behind issues or reviews
that may be significant, and the question is you have a burden of proof
that this transition in fact is effective --
MR. GILLESPIE: Yes.
DR. BONACA: And also you have a burden of proof of closing
on these objectives that you set in your, you know --
MR. GILLESPIE: Right.
DR. BONACA: And the other thing is that it would be
certainly useful to the industry and certainly to me to understand the
differences there and to really -- it would give me much more comfort
that in fact the objectivity is truly improved in the process.
MR. GILLESPIE: Let me -- I'm only hesitant because this is
an opportunity lost, you might say. If this was a research project,
literally doing a kind of a more academic research project, an
experiment that doesn't work still is a successful experiment, and it
tells your hypothesis was an incorrect hypothesis. And some of us only
look at -- only if it proves a hypothesis is it successful.
And when Bruce was putting together the program, he went
through a process of identifying over 100 inspectable areas which
basically encompassed the program as his group who were actually
inspectors, sorry, today. And so they basically took today's program
and cut it up into all these pieces, and we ended up with about 40 areas
when you implied all the insights. What we lost was they didn't write
down the results of the brainstorming sessions which necessarily caused
them to filter those things out.
And I don't know how recoverable that is, but I don't mind
going back and asking the question, and going back through some early
drafts. Steve Stein was on the group, and I talked to Steve, and I
said, Steve, did you save the file folders to show the traceability of,
basically, how do you go from everything we look at now to what you look
at here, and why?
DR. BONACA: As a minimum, if you had written that here, I
would have been more comfortable.
MR. GILLESPIE: Yes, and that was a reaction a number of us
had when we started reading, and we said, oh, no, we left off the
negatives. We left off the reasons why we left things off.
We will look at the file folder and see if it is
recoverable. I don't want to promise it, that it is.
MR. FLACK: Just to clarify, they are not really lost. I
mean for the baseline inspection, it does a certain level of
inspections, but, again, you revisit these depending on your performance
indicators and findings. So it is not totally lost, it is just that you
will get into it after something else leads you there.
MR. GILLESPIE: Yes, but what is lost is -- What did we
trade off? What would we be inspecting today that we are not going to
inspect a year from now because we have this performance indicator?
MR. FLACK: As part of the baseline.
MR. GILLESPIE: As part of the baseline. And that, I am
afraid, is lost.
MR. JOHNSON: But I almost think a better way to do it, or
at least as good a way to do it, and one that we had anticipated, was
that we will go look -- we have talked a lot about success criteria, and
one of the success criteria is do we meet the objectives. And I think
it is going to be important that we go back a year from now, and we look
in terms of what it is we are actually inspecting. And what did the old
core provide? What are we looking at now? Do we have a sense that we
are more objective? Do we have a sense -- I mean have we hit our
targets in terms of burden reduction? Have we hit those things that we
outlined for ourselves? And we have promised to do that all along, and
the Commission is holding our feet to the fire on that.
DR. BONACA: The other thing is that I would expect some of
your resident inspectors would want to know why I am not going to
inspect this anymore, I think is important.
MR. GILLESPIE: And that is going to have to be factored
into the training. And I will give you one last bit of insight that is
hidden in this paper, in Pat's write-up some place, and it goes -- of a
thousand LERs that came in about a year, only 1 percent, or 10, were
significant enough to be screened at a 10 to the minus 6 or greater risk
number, which is significant in that --
DR. POWERS: It is not hidden, it is highlighted in there.
MR. GILLESPIE: Yeah. I'll tell you the truth, I have
talked to a lot of people who said they read the paper and they missed
this sentence, and I remembered it, because it stuck.
DR. POWERS: It stands right out.
MR. GILLESPIE: If that is in any way reflective of also the
kind of things that an inspector sees as he is walking around a plant,
then that is going to be the most significant cultural change that we
will have, that 99 percent of what we have been observing in the
facilities is likely not risk significant.
Now, I don't know that I can take that and directly make
that conclusion, but, certainly, it says that many of the things that we
see going on day to day, even though they are viewed as incorrect or not
right, may not necessarily have a risk significance. That is the one
most significant cultural change that is going -- that we are challenged
with right down to the lower levels, because we have trained ourselves,
we have trained the industry, that this is a problem, regulatory
significance, procedural compliance.
DR. BARTON: Verbatim compliance with regulations, it is not
quite that bad, but it is almost there.
MR. GILLESPIE: So I don't know the answer, but this is the
change process. It is not just changing how you inspect and what you
inspect, but it is the fundamental understanding of why I am just
turning this over to a licensee, when only a year ago, this might have
been a level 4, level 3 violation.
DR. SEALE: It is interesting.
DR. APOSTOLAKIS: We are not requiring verbatim compliance
anymore?
DR. BARTON: I hope not.
DR. APOSTOLAKIS: If you read the nuclear safe culture
thing, we do. We do. We do.
MR. BARANOWSKY: They might be NCVs, non-citable violations.
DR. BARTON: That is how to handle it.
MR. GILLESPIE: Anyway, it is that kind of data that needs
to be put together in an understandable -- that is going to be a
challenging training course, if you would, or adjustment to the P-111
course that we currently have. To get this kind of information, and the
context, that also fits with what people feel they are intuitively
seeing at the facilities is going to be difficult.
Even Sam Collins gave me an incident, it was a morning
report that came into the EDO. A BWR flange had a steam leak on it, it
was going up in power. A 10 percent power tech spec says check your
high pressure injection turbine-driven pump. They let steam in. Steam
flange wasn't tightened. Steam leak. They turned off the steam, they
tightened the flange, it continued up in power. The system got very
exercised. It happened to be the most important thing that happened
that day in the morning report.
It was not risk significant. So everyone got exercised all
the way up through the EDO, you might say, in a communications sense,
for something that was not -- the system worked. The tech spec said
test, they tested. Yes, they have to go back and find out why weren't
the bolts tightened the first time around. And Sam looked at me when I
said, this isn't risk significant. He said I can't believe you, go get
a risk guy. So I took it down to the risk guy and I said --
DR. BARTON: Well, it is going to take years to change that
mindset.
MR. GILLESPIE: Is this significant? And the risk guys
says, you know that is not significant. I said, yeah, but you have to
write it down for me. So he wrote it down, he put his name on the
bottom. I went back in to Sam, and said, see, Sam, it is still not risk
significant. And he just looked at me and said, we are really going to
have to change the way we think around here.
DR. BARTON: You are darn right you are. It is also not
only risk, it is also you are afraid that you are going to be accused of
not knowing what is going on out there.
MR. GILLESPIE: Well, and that is -- yeah, and that is the
other sense.
DR. BARTON: That is their part, whether it is risk or not.
MR. GILLESPIE: When you start stepping back, if you do less
inspection, you have less information.
DR. BARTON: Right.
MR. GILLESPIE: When you are less intrusive, you have less
knowledge on a day to day basis.
DR. BARTON: Yeah.
MR. GILLESPIE: It is a different approach. And it is going
to be a challenge to get this into the culture of this organization.
DR. BARTON: Oh, yeah.
DR. SEALE: Somehow you have to bring into the attributes of
this system the idea that you are inspecting smarter.
MR. GILLESPIE: Yes.
DR. SEALE: And maybe that 1 percent example is a good
lead-in to that line, to that point.
DR. MILLER: The same number of hours in inspection on the 1
percent rather than on the 99, is that?
DR. SEALE: Well, --
MR. GILLESPIE: That is really the focus of a risk-informed
inspection program is you are inspecting somewhat less, but you are
actually inspecting the risk important things more.
DR. MILLER: That's the whole objective.
DR. POWERS: I think it is important to consider a human
element of the inspector's life, it is what I call my proof-reading
theory of facility inspection. I sometimes get to inspect facilities.
If I don't -- if I have people working for me developing a facility and
they don't make a certain number of errors, my inspection becomes very
poor. As soon as they cross a threshold of a certain number of errors,
then I get them all. And so it seems to me that I would tend to portray
this as -- it is true, 99 percent of these things are not risk
significant, but they may well portend a degradation, and you need to
bring them still to the licensee's attention, it is just that we don't
do anything about them.
MR. GILLESPIE: That was our earlier discussion.
DR. POWERS: Yeah, I think we had this earlier discussion.
MR. GILLESPIE: Exactly.
DR. POWERS: That what you don't want is inspectors being
bored.
MR. GILLESPIE: Yeah, you don't want the inspector stepping
aside, because what you are going to have -- you are going to miss
something you should have picked up. And that is actually the challenge
to the screening process, which is the big IOU to the Commission of
inspection results. Having some false positives when you screen is
probably okay. In other words, some things that you do a little more
work on, you might not have had to do a little more work on, because I
feel comfortable we are talking about fairly small numbers of things.
But having a false negative is totally unacceptable.
So the challenge in setting up -- it has got to an
asymmetrical uncertainty in the screening process, and Alan has taken
this challenge on, so now, officially, he is hooked. So -- and that is
why it might end up being kind of a multi-stepped thing, a rule-based
thing that is looking at certain questions. Is it a train, is it a
system? If it is a program failure, it is kind of like asking a
question -- Are there enough program failures that the coincident effect
at any given point in time is risk significant? That is a real tough
question to answer, and that starts getting at the sense of regulatory
significance or not, because it is not -- Was it wrong and fixed, wrong
and fixed, wrong and fixed? If it is wrong and fixed, and things don't
happen coincident to degrade several systems or several trains, then it
won't hit a safety threshold, yet, you have to give it to the licensee
because you don't want all those things happening.
So that, to me, is the one major, major, major intellectual
challenge in this whole system that we have ahead, is doing that for the
inspection findings. Because what I said doesn't depend on numbers of
findings, it depends on the severity of the finding. So, in inspection
space, we may be dealing with the safety significance of the finding,
not the number of findings. And, again, that is a different approach.
If you have a lot, it has to be safety significant, right, that is the
intuitive feel. It may not be. And it is going to be very difficult,
because now we are counter-intuitive with our inspection force, and that
is going to be -- that is going to be real hard.
DR. MILLER: So an inspector who found one very risk
significant situation would get more credit, so to speak, that somebody
who found a hundred insignificant ones, is that the thing you are
alluding to here?
MR. GILLESPIE: No, it is a matter of -- the general sense
is, gee, I found eight things wrong in procedural compliance, and the
typical enforcement letter that would go out, signed by a division
director, would declare it a programmatic failure, and it would be a big
deal in today's environment. I don't know what it would be in
tomorrow's environment, because if those eight procedural items happen
disconnected in time and never actually affected a piece of equipment,
and we are trying to develop a safety outcome score, or goal, or
judgment, it puts it in a different perspective. And it is not the way
we have been trained.
DR. POWERS: I have a question about public perception. If
I am a member of the public sitting next to this -- living next to a
plant, and I see less stringency on the part of the NRC, the number of
level 4 violations has gone to zero, lots of things that on the face of
it would have been violations today are not tomorrow, and I, as a
reasonable person, come to you and I say, how come? And you say, oh,
they weren't risk significant.
And I respond, gee, how would I know that? And you say,
well, it is real simple, you run the sapphire code with the right kinds
of inputs and outputs, and you take the things and you do a risk
assessment worth on this, and you look at your 95 percent confidence
level on a T-scale with 5 degrees of freedom and utility comes this
number.
Do you think you have really persuaded me that you are not
--
DR. MILLER: Have you persuaded anybody on this Committee
even?
DR. POWERS: You're not relieving a burden upon your
licensee at the expense of increased risk on my part? And by risk in
that context, I mean my qualitative perception of the risk.
MR. GILLESPIE: Actually, we are working very closely with
public affairs on trying to answer that question. But there's some
flaws in your basic assumption, they are different than our basic
assumptions. One, we expect the inspector will write down when he sees
something wrong. Now, what we have to talk about is what do we call it.
The current enforcement policy change, as yet unapproved, but I
understand the vote sheets are in, would take all these level 4s and
call them non-cited violations, would not have us follow up on every
single one of them, but would have us audit, on an audit basis, to
follow up on them. We would expect --
DR. POWERS: If I am a member of the public who knows a
little bit about your processes, what I see is violations used to be
cited, now they are not. Violations used to be followed up, now they
are not. And when I -- as a reasonable question, I mean I don't think
it is an outrageous question, before I make any accusation, is I ask you
why, and you give me this claptrap about risk analyses and things like
that.
MR. GILLESPIE: Like I said --
DR. POWERS: What other conclusion can I arrive at?
MR. GILLESPIE: Well, I think, first, the intent is that we
are not going to ignore the items. Now, what we are doing is calling
them something different, and that gives us, you might say, a public
perception challenge to deal with, because we are calling them something
different, and we will not be following up on every single one we used
to follow up on. So we are going to have to very articulate in saying
why. We haven't been there yet.
In fact, one of the criticisms of this whole report, that
they opened up the meeting with, was a member of the public would have a
very difficult time reading this report. It is not user-friendly.
But let me go back and say why this is better than what we
have, and that is kind of where we are at right now. What we have right
now in judging overall safety of the facility is the SALP process. The
SALP process is actually very, very, very subjective, there is no doubt
about it. In fact, the insights we would have from this program that we
have kind of run so far would be that a lot of facilities, or a large
group of facilities, are run so safely that it is almost impossible to
discern between them, yet SALP was trying to force itself to discern
between several very well run facilities because we were obligated to
write a report.
This will allow us now to have more information out to the
public directly related to their operations and more frequently, and
that is the plus. The minus from the system is we are not going to
follow up on every one of the items. The plus to the system is, by
publishing the performance indicators, and an interpretation of what
they mean on a quarterly basis, and something bigger on an annual basis,
we will be displaying to the public more and current information on the
safety of the facility. So we are substituting one process for another
process.
How we put that together is going to be a real challenge,
and how we communicate that to the public is going to be another real
challenge. In fact, public affairs offered that maybe we could go talk
to the League of Women Voters and a whole bunch of other people to try
to deal with the concerns.
DR. POWERS: I think the important thing, I mean I hate to
be so complimentary, and I will try to avoid that --
MR. GILLESPIE: Oh, no, but this is a hard one. You know,
it is easy to deal with the darn technical issues. It is very difficult
to deal with the translation of what you think is a good judgment into
you are backing off, you are not doing as much.
DR. POWERS: One thing I liked about the draft response you
articulated here, you went for about two minutes without ever once
bringing up the word "risk" in that, and I think that is the start of
formulating a good response, is bring up the risk after you have said
all those things you did, more information, better information, better
interpretations and things like that.
Get to the results of going in this direction and then if
someone wants to ask you why do you have all this good insight, then you
can bring up risk. Rather than saying, oh, well, we have used these
analyses that it is impossible for me to reproduce, or even understand,
to focus my efforts so that I get the most bang for the buck. You know,
that just leaves me cold as a response, whereas, when you tell me -- ah,
you are getting much more, you are really getting much more out of this,
and much -- using much the words you did and don't bring up this arcane
technology that requires a degree in statistics to understand.
DR. SEALE: A little confession may be good for the soul
here, too. I have heard that it is, at least in some circles, although
it is not too popular these days.
DR. POWERS: But in both the states that you and I live in
--
DR. SEALE: Very definitely. And the confession may be, and
it may take a little bit of homework to verify it, but the confession
may be that, of all those violations that you had previously cited, the
ones that you are no longer publicizing -- or going into the record
with, and so on, were ones that had no impact on -- when you finally get
around to the point of public risk.
DR. POWERS: But I don't think I would try to bring that
story to the public --
DR. SEALE: Not up front.
DR. POWERS: -- because he didn't think -- he didn't know
whether they did or not. All he knew was that they were being stringent
and now they are not. I think -- I really like this idea of no, no, no,
it is not less, you are getting way more. I really like that approach.
DR. SEALE: Well, we are looking at the things that count.
DR. POWERS: I wouldn't even go that far. I would say we
are looking --
DR. MILLER: You are getting more.
DR. POWERS: We have always looked at the things that count,
and we are continuing to look at the things that count, and we are
giving you far, far more than you ever got before, because you used to
get these funny looking SALP scores that meant absolutely nothing to
anyone on the face of the planet, not even those in the NRC. Now, you
are getting all this other wonderful stuff. I think that is the germ of
a public -- of risk communication that may actually work.
MR. GILLESPIE: It's funny, that's where Bill Beecher came,
in a lengthy meeting we had with him yesterday, trying to brainstorm.
It was tell them what you are giving them, don't --
DR. POWERS: That's exactly right.
DR. MILLER: Tell them the glass water is getting fuller not
empty.
MR. GILLESPIE: It is getting fuller, not emptier, yeah.
DR. BARTON: I think we need to wrap this up, and figuring
out what do we want to talk about at the full Committee meeting, I guess
next week.
DR. APOSTOLAKIS: We are all here.
DR. BARTON: Pardon?
DR. APOSTOLAKIS: We are all here.
DR. BARTON: Yes, we are all here. What else? What --
DR. APOSTOLAKIS: Let's take the time to write a letter.
Well, I don't think we need a presentation, we are all here.
DR. SEALE: Well, the only thing I can think of is if we ask
them if they could, at that point, tell us what additional information
you are liable to be able to put in the Commission letter.
DR. BARTON: The supplemental information.
MR. MADISON: I think at this time it is premature. We have
got a draft concept that we are refining and I really want to put it
through a lot more exercises before we make it -- bring it here.
DR. SEALE: Okay.
DR. BARTON: What about the policy issues that remain that
you mention in this document, evaluating interface of Part 50,
revisiting event response evaluation, revisiting N-plus-one,
organizational impact. Is that something that you are prepared, that we
could talk about, that we could address in our letter? Since the
Commission has to make decisions on those policy issues?
MR. MADISON: No, the Commission -- those four policy issues
were a heads-up to the Commission. And we actually had a fair amount of
discussion on. These are not today's decisions.
DR. BARTON: Okay.
MR. MADISON: They are next year's decisions. In some
cases, they are six months away. But the basic question was, if you do
go down this path, and there is less inspection, but you keep
N-plus-one, residents tend to be generalists, not specialists.
DR. BARTON: Right.
MR. MADISON: If you need engineering specialists and what
you have got is generalists living at the plants, that gives you an
organizational problem and causes you to need, if you go down this path
very far, to step back and say, I have got an institutional problem I
have to address, not a technical problem.
The reconciliation or harmonization with Part 50, that was
our number one issue on the list.
DR. BARTON: Right. Right.
MR. MADISON: It is just highlighting to them, we are
getting out in front and if Part 50 changes their scale, then we are
going to have to have feedback from that, and they are going to have to
be cognizant of Commission decisions made. If the Commission approves
our basic scale of where our bands are, then it is going to be
hard-pressed for the Commission to necessarily change their mind without
some justification later. So approving what we are doing actually has
future implications that says hey, the bands are set. Here's the
licensee response zone for whatever it is.
So, no, those four issues were a heads-up. I think they
need to be dealt with in the course of probably the next 12 months.
They can't be ignored, particularly if we are successful --
DR. BARTON: That's true. Right.
MR. MADISON: -- with the pilot program, but not in the next
three months.
DR. BARTON: Okay. What else do we want to hear? George, I
think the problem we've got is it's already in the Federal Register,
right?
DR. APOSTOLAKIS: You can take that time to write a letter.
DR. POWERS: Or at least a summary from the Commission
Chairman in order to -- or from the Subcommittee Chairman at the very
least. I don't think it would be a bad idea to bring up these policy
issues even though they are six months down the line, because that is
about our time schedule for thinking about the subject.
DR. BARTON: Okay.
DR. POWERS: And it would be opportune for putting it on our
agenda so that when you come in here, we are not having to spin up from
zero -- that and a summary from the Subcommittee Chairman would fulfill
our obligations.
DR. BARTON: At the full Committee meeting.
DR. POWERS: At the full Committee meeting.
DR. BARTON: Okay.
DR. APOSTOLAKIS: How much time do we have?
DR. BARTON: I don't know.
MR. MARKLEY: 10:30 on Wednesday.
DR. MILLER: One of the policy issues -- it may not be a
policy issue -- we raised a concern about the fact that the inspection
assessment process is leading the regulatory situation. Is that an
issue you want to raise?
DR. POWERS: Yes. I think that's a --
DR. MILLER: I am beginning to think that maybe one could
look at the advantages of this leading the regulatory process -- if we
want to look at the half a glass of water fuller.
DR. POWERS: And I will remind you --
DR. MILLER: It might lead them in the right way --
DR. POWERS: I would still like to know about your
validation of your tools for your Appendix H and Appendix I analyses.
MR. GILLESPIE: That's an IOU we'll take away and get back
on Sapphire Code and what happened to that.
DR. WALLIS: Well, I took a vow of silence when I came in
this morning.
DR. POWERS: Why?
DR. MILLER: It's about those hundred Senators being quiet.
DR. WALLIS: But Dana woke me up when he said how would the
public perceive this, and I think it is not just a question of public
perception but too many of these changes, which I think you have done
very well, appear to be motivated by what is good for the utility or
what is good for the administrators in the NRC, and it would be good if
somehow it could be put in perspective to give a really fair measure of
what has the effect of all this been on nuclear safety.
I don't know how you would do that but really that is a
measurement I would like to see or get some better feel for. I like
what you are doing, but how do I tell if it's been good for nuclear
safety or are we backing off or what?
Maybe we back off but it's still better for nuclear safety,
but how do I know?
MR. GILLESPIE: Yes, that's -- let me explore that a little,
because this is kind of a key point underlying the paper.
In the paper it talks about this is a process to ensure that
safety is maintained so we are clearly not putting something in place,
and from the way Pat described his statistical and the way the first
threshold was set, we are clearly trying to maintain what we believe is
an acceptable level of safety. We are not forcing it to be better.
Quite honestly, I don't think we could force less than the
.8 scrams on the average per plant that exists today, so how is this
good not to do that? That is a public perception problem. You mean the
NRC is not forcing it to be better and better and better and better?
I don't have a good answer for that because we haven't
digested it. We understand it but that is kind of one of those things
that is hidden in the paper.
DR. KRESS: Well, let me take a stab at that at the risk of
breaking my vow of silence.
DR. POWERS: What is this vow of silence?
DR. KRESS: This process we are going through could lead up
to this bit of a problem that George pointed out that you will find some
plants for example that may get outside your inspection lines but then
you go in and look at it more closely and find that they still comply
with all the regulations, and what you will have done there is perhaps
this particular plant while it still complies with the regulations as a
specific plant has -- let me put it this way -- not quite as good a
desired risk status.
MR. GILLESPIE: Yes, that's --
DR. KRESS: And it is because your regulations have allowed
that. Now this allows you to focus on the problem of risk, which is
really your regulatory mission, and you can say, well, have our rules
done the job or not ,and which part of the rules are keeping us from
doing our job and it will allow you to go back and say now do we need to
change the rules some way and in what manner do we need to change them
in order that we can get plants within a risk status that we would
better desire.
Now of course that raises questions of backfits or
substantial improvements in regulatory analysis, but I mean you're stuck
with that, but it will allow you to focus your attention on the rules
and whether they are doing the job correctly or not, and I think it is
worthwhile.
I mean that is a public benefit that is going to come from
it in my mind.
MR. GILLESPIE: I'm going to say it kind of harshly. I
don't mean it -- I don't know how else to say it, but if the industry
disagrees with the basic premise that we have tried to pick thresholds
that in fact they could fairly operate within and in normal
circumstances at a reasonably well-run plant, not a perfect plant, it
can operate within those thresholds. If that basic premise is
incorrect, the industry has to stand up and be counted and say that.
DR. KRESS: That may be the way -- your hands may be tied --
MR. GILLESPIE: I can only take on so much burden and this
team in developing an approach that has to be somewhat generic, and it
has to give the public the ability quite honestly to compare one plant
to another, but in comparing one plant to another when you get
structured like this, it's there and, yes, one plant could have, if it
continually violated a threshold but was in compliance with all the
rules, it could have a less desirable public image, and that is kind
of -- you know, the comment on SALP today -- a similar comment.
I hope it will be less of a problem than the system today
gives us in that same arena but we will likely have not fixed it
perfectly, so the industry now has to speak up and that is why this
thing is out for comment.
DR. BONACA: Your point is a concern I had before when I
talked about the fact that most units meet those kinds of quantitative
criteria that we have here. From past experience I can remember exact
cases where units met those criteria that were provided by INPO and so
on and so forth entirely, and yet their performance was very negative
because of other issues which are tied to inspection.
Now to the degree to which you have a much more scrutable
process here, it's going to be very hard to deal with that kind of
situation.
I am trying to understand, you know, because I do believe
that the inspections bring about a lot of insights that tell you whether
or not a plant is doing well.
What I am trying to say is that the numerics may force you
to disregard a lot of these other insights from inspections if
consistently they give you all greens.
See what my concern is now?
MR. GILLESPIE: Yes, and this is the crux of a past
criticism we have gotten from GAO in past years. Let me take what you
have just said, if I could, to the extreme, and that is when we go in
and shut a plant down because all of a sudden -- Cook.
We did an A&E inspection, raised questions about
engineering, went in and looked at engineering even more. They went in
and looked at their engineering even more. The ice columns became -- it
just -- it kind of, you might say, expanded to the point where the
facility has been shut down for an extended period now to get the design
process straightened out.
One of the things we were being pressed several years ago
was to always identify a problem to allow a licensee to correct it
before it got to the point where you had to shut down to correct it. I
believe -- and I am not going to commit to the team, unless they want to
chime in, that these indicators plus the inspection -- and I can't take
my eye that there's still substantial inspection being done in very
focused areas, particularly engineering, will still give us an early
enough indication that I think we can beat that goal.
Now we looked back retrospectively and Cook was one of the
ones that Bruce Mallett looked back on and said there is no indicator
for this problem they had in engineering -- are we still looking at
engineering? And Bruce said yes, we are still looking at engineering,
and we would have had an opportunity to pick that up.
Cook still would have been shut down, because you would have
found a problem in engineering, they would have pursued it more, the
problem would have gotten bigger, and they'd be shut down. The Cook
shutdown by this new process likely would not have been avoided.
DR. MILLER: But it might have been shorter.
MR. GILLESPIE: It might have been shortened because we
would have a different perspective on it possibly but I don't think the
shutdown itself would have been avoided, so for Cook we wouldn't have
made this goal that was set up for us.
Inspection insights are still going to be there. I hope
they are the insights in the right area. We have tried to focus them
that way.
I would like to bring up one other thing that is in the
paper and no one here touched upon. It's something that Mike is going
to grimace when I say it, and it's the -- what do we call it? -- the
executive exemption? No --
MR. JOHNSON: Executive override.
MR. GILLESPIE: Executive override. We are allowing a five
percent executive override and I regret the terminology but I am pleased
with the concept.
In fact, if there are significant insights that cause us in
a very limited number of cases to question the performance indicators,
we do need to go in and examine those cases and if we find the
indicators are correct, fine. That reinforces where we're at. If we
find they were wrong, the executive override concept has great value in
program evaluation space relevant to being kind of like a proof test.
One of the things that we are going to be kind of committed
to is kind of looking at the terminology and the basis we had in there
for that.
David Lochbaum, UCS, says I don't like these executive
overrides at the Commission meeting -- there should be zero executive
overrides -- because it had a negative connotation, but actually having
an occasional question to test the system is probably a positive move,
so we are going to need to go back and re-look at how we have termed
that in there to take care of that concern, not everyplace but I think
as a programmatic group in Headquarters we have to be conscious that we
need to test our assumptions somehow, and the only way to do that is
from the outside looking in. You can't necessarily do it always from
the inside looking out.
DR. SEALE: Are you familiar with the term "the halo
effect" --
MR. GILLESPIE: Yes.
DR. SEALE: The halo effect was on the basis of
personalities. The executive override that you are talking about is on
the basis of specific physical observables. I think that is a
reasonable trade, and not have the personalities which have been a
problem in the past.
MR. GILLESPIE: So that is one aspect in there. There was
some discussion on that. You know, if that is a piece -- you know --
those are the kinds of pieces and kinds of comments that we're searching
for that will help us hone the system down or in, get it in tune, and
move forward with it.
I think the concept was right, the move was right, it's the
right thing to do, but we worded it wrong.
DR. SEALE: Yes.
MR. GILLESPIE: And that just comes as time evolves and you
say, well, why did we say it that way? Why do we really think this is
the right thing to do?
That is kind of an evolving nature, which is why we are not
committed to rewriting this entire 400 pages, but now taking the
comments, adding them to these 400 pages, and then moving forward and
writing the actual program documentation.
With that, I am worn out.
DR. BARTON: Well, one thing. I know you said you didn't
want to rewrite the document but there's areas -- the document could
probably be streamlined and a lot thinner because there's a lot of
redundancy.
Also, I don't remember whether there is a master table of
acronyms in there, but you use ERO in two different sections, Emergency
Response and then Effluent something and Radiological Emergency. You
may want to either put a master acronym thing in here or else change one
of the two because you use it in two separate --
MR. GILLESPIE: You can tell we had multiple authors putting
this --
DR. BARTON: Yes, and that is probably the result and why it
got that way, but you want to look at that so you don't end up with
confusion down the road or at least put a master --
MR. GILLESPIE: Pat's comment was more than multiple. In
fact, there was about 24 authors who wrote different sections.
DR. MILLER: There is strength in the weakness.
DR. BARTON: Anything else we want to talk about next week
or here that we haven't heard before? If not, then --
DR. SEALE: I have an entirely different matter to ask the
people.
DR. BARTON: While these guys are here?
DR. SEALE: No, no. Just don't you guys leave, that's all I
am saying, the members.
DR. BARTON: And we are done and we will break for lunch.
Thank you for all the hard work and coming in here and
putting it up again. I guess we are adjourned.
[Whereupon, at 12:31 p.m., the meeting was concluded.]
