GAB Decision 699
November 1, 1985
Delaware Department of Services For Children, Youth and Their Families;
Ballard, Judith A.; Settle, Norval D. (John) Teitz, Alexander G.
Docket No. 84-181
DECISION
The Delaware Department of Services for Children, Youth and Their
Families (State or Appellant) appealed a determination by the
Administration for Children, Youth and Families of the Office of Human
Development Services (OHDS or Agency) that the State was ineligible for
fiscal year (FY) 1983 funds under section 427 of title IV-B of the
Social Security Act (Act). That section provides that a state may
receive additional funds for child welfare services, beyond the amount
available to each state under section 420 of the Act, if the state meets
certain requirements for the protection of children in foster care. /1/
The Agency evaluated states' compliance with section 427 on two levels.
First, the Agency determined whether a state had established policies or
procedures for implementing the requirements of the Act. Second, the
Agency reviewed a sample of case records to determine whether these
policies or procedures were operational. In conducting the sample, the
Agency sought to determine whether a state had complied with the
applicable requirements of section 427 in 80 percent of the foster care
cases. We discuss further below the sampling methodology used by the
Agency. If a state failed to establish a requirement as a matter of
policy, or a state failed the review because of the application of the
Agency's sampling procedure, the Agency required the return of the
section 427 funds.
The Agency initially approved Delaware's written request for additional
funds based on a written certification by the State that it met the
requirements of section 427.(2)
However, following a review conducted to validate the State's
self-certification for FY 1983, the Agency advised the State that it was
ineligible for FY 1983 funds since the Agency concluded on the basis of
its sampling procedure that the State failed to comply with the Act in
80 percent of cases. Out of a total of 85 cases sampled, the State
failed 21 cases. According to the Agency, the State needed to pass all
but 17 cases of the 85 cases sampled to pass the section 427 review.
For purposes of this appeal, the State challenged the Agency's finding
for four sample cases. For three of the cases, the State argued that
they should have been found acceptable by the reviewers. For one case
which failed, the State argued that it should not have been reviewed at
all, since it was used to replace a case which was improperly excluded
from the sample population. Alternatively, the State has argued that
the statistical methodology used by the Agency, as applied in its review
of the State, was improper. The State argued specifically that the
Agency may not fail a State merely because of the failure to reach a
positive "decision point" at the sample size where the Agency has
chosen
to "truncate" the sample. We discuss the State's arguments in greater
detail below.
In Part I of this decision, we conclude that the Agency improperly
failed the State at the 85 case level because the Agency did not follow
its own policy with regard to the confidence level of its procedure and
the failure to follow that policy was not in accordance with the
theoretical basis of sampling. We therefore remand this case to the
Agency either to draw more cases at random and continue the sequential
sampling until a decision point is reached, or to use some methodology
in reviewing the State case files that properly considers the confidence
level of the finding. We emphasize that we are not saying that there is
anything wrong with the technique of sequential sampling, but, rather,
that the Agency did not use it properly in the facts of this case. In
Part II of the decision, we address the four cases identified by the
State as having wrongly failed the review. We find that three of the
four cases were correctly decided by the Agency and that the fourth case
improperly replaced another case in the sample and that the Agency
should not have reviewed that replacement case.
The Agency's sampling methodology
In determining whether a state is satisfactorily operating a case review
system, the Agency samples part of the state's foster care case records.
The purpose of sampling is to avoid the time and expense of evaluating
all the foster care cases in that fiscal year, by mathematically
extrapolating performance in the universe of cases from an evaluation of
the sample.(3)
For section 427 reviews, the Agency uses, with the consent of states, a
special type of sampling known as "sequential sampling." Sequential
sampling differs from standard sampling in that the sample size is not
fixed, but rather is determined by the results of observed cases as they
accumulate. Its goal is to allow the sampling of a smaller number of
cases than the fixed sample size of standard sampling. /2/ Sequential
sampling thus creates further cost savings for the Agency and aids the
states by avoiding the need to present large numbers of foster care case
records.
Using this methodology, the reviewers start sampling a collection of
records chosen in a scientifically random manner. The Agency uses a
prepared "decision table" that prescribes how many cases of the total
sampled must have failed in order to conclude that a state fails the
section 427 review, as well as how many cases of the total sampled thus
far must have passed in order for the Agency to stop sampling and
conclude that a state is eligible for section 427 funding. The decision
table methodology as it applied to the State in its 427 review is
illustrated by the graph reproduced on page 4 of this decision from the
Agency's Exhibit 12.
The two top diagonal lines of the chart on page 4 represent, under two
different assumptions, the points in the sequence which indicate that a
state should fail the review. (The reason that there are two parallel
lines toward the top of the graph is that the Agency changed the region
of the(4) (SEE ILLUSTRATION ORIGINAL)(5) reject zone before the start of
the review. See n. 3, below.) The lowest diagonal line (starting from
about "20" on the horizontal axis) represents the decision points
indicating that a state has passed. The area between the two diagonal
lines is known as the "zone of indifference" where neither a positive
nor negative decision point has been reached. The review of a state's
files will proceed along the zone of indifference until a decision point
is reached or until some point is reached at which the Agency chooses to
terminate the review, even though no decision point is reached. (This
is referred to as the point of "truncation.")
The acceptance and rejection lines are derived from a "sequential
probability ratio test" (SPRT), which mathematically defines these
acceptance and rejection zones, with a given level of statistical
certainty. The SPRT, in essence, is the formula which projects how many
cases must be passed or failed to extrapolate properly what the
performance would be in the universe.
The theoretical justification of any type of sampling procedure is that
one may draw conclusions about the universe of cases from a sample,
within a certain range of probability of error, described as the
"variance" or "confidence level" of the inference. In determining
the
confidence level of the sequential sampling procedure, the Agency took
two types of risks in making an inference about the "universe" of
cases
from the sample. The first type of risk, designated the "alpha" risk
(or risk of a "type I" error), is the possibility of wrongly rejecting
a
state from eligibility for section 427 funding, when the state would
actually pass 80 percent of the cases in the universe. The "beta"
risk
(or risk of a "type II" error), by contrast, is the possibility of
wrongly accepting a state which, if the entire universe were sampled,
would fail the review. For states such as Delaware, with less than 1000
foster care case records, the alpha value was defined as .05 and the
beta value was defined as .01. In other words, in terms of the alpha
value, the decision table methodology as described by the Agency was
designed to evaluate whether the State complied with section 427
requirements in 80 percent of cases in the universe, with a five percent
probability of wrongly concluding that the State should fail the review.
According to the Agency's policy statements, these alpha and beta values
are incorporated into the decision table methodology. /3/ See Agency's
Ex. 9, Program Instruction (PI) 82-06, Att. E, pp. 9-10.
(6)
I. Sequential Sampling as Used in this Section 427 Review
The State has not contested the general use of sequential sampling in
evaluating whether it should have passed or failed the section 427
review. It objected instead to the Agency's policy of "truncating"
the
sequential sample at a pre-defined level and automatically failing a
state if no decision point had been reached by that level.
We conclude that the Agency's decision to fail the State at the point of
truncation was improper. There are two reasons for this conclusion. As
explained below in Part A, the Agency's policy statements themselves
provide that sequential sampling will be conducted so as to measure a
compliance level of 80 percent, given an alpha value of .05. We conclude
that the Agency did not assume an alpha value of .05 when it decided to
fail the State after reviewing 85 case records. Further, as we explain
in Part B, the Agency's conclusion that the State would fail 80 percent
of cases was not statistically valid, since, by not assuming a (7)
specific alpha value, it did not account for the possibility of
statistical error that is present with any type of sampling. In Part C,
we address the Agency's other technical justifications for its decision,
where we find in part that while it is proper to truncate a sequential
sample, this must be done in a manner that considers the confidence
level of the Agency's conclusion.
A. The Agency's own policy statements require consideration of an alpha
value of .05.
In its policy statements explaining the section 427 sampling
methodology, the Agency clearly provides that, for states with less than
1,000 foster care case records, the methodology is designed to measure a
compliance rate of 80 percent in the universe, with an alpha value given
as .05 and a beta value given as .01. The Agency's Program Instruction
issued in 1982 provides:
Similarly, the four quantities that completely determine the
sequential sampling plan for states with less than 1,000 foster care
cases are:
(a) P1 = .20
(b) P2 = .35
(c) alpha = .05
(d) beta = .01
At the hearing, the Agency stated that a policy decision was made with
regard to states which remained in the zone of indifference at the 80th
case (or 150th case, for states with a total number of foster care case
records over 1000). In the initial year in which a state had applied
for section 427 funding, the state would pass if no decision point was
reached by the 80th case. In the review of any subsequent year, the
state would fail. Since the review of the State here in FY 1983 was in
a subsequent year's review, the Agency determined that the State should
fail. Tr., pp. 195-196. /4/
If the Agency had passed or failed the State before the 85th case on the
basis that the State had reached a positive or negative "decision
point," the Agency's inference that the State would pass less than 80
percent of cases in the(8) universe could be made with a certainty
corresponding to an alpha value of .05. However, the Agency's policy to
fail a state merely because no decision point was reached at the 85th
case assumed no such alpha value. Since a state in such a situation is
no longer in the "reject zone" as defined by the SPRT, the corresponding
alpha value would appear to have been something greater than .05. /5/
The State estimated that the alpha value corresponding to the Agency's
decision to reject a state in the zone of indifference at the 85th case
may have been as high as 51 percent. Appellant's Post Hearing Brief, p.
6.
If the Agency accepts an alpha value of .05 as a condition of sequential
sampling, we find it improper for the Agency to ignore that level of
significance when it fails to reach a decision point by the 80th or 85th
case. Nothing prevents the Agency from making any reasonable changes in
how it measures state compliance for future years. However, the
"policy" to fail the State here was clearly contrary to the Agency's
own
statements that it would consider a particular level of significance of
any conclusions about the universe of a state's foster care case
records.
B. The Agency's conclusion was not statistically valid.
The State's expert witness confirmed that the Agency's decision to fail
the State did not assume an alpha value of .05. See Tr., pp. 30-42.
The State presented further testimony and cited scholarly authority to
the effect that the Agency's policy to fail the State in the zone of
indifference was therefore not statistically valid, since any
methodology must account for the confidence levels of its findings.
Specifically, the State's witness calculated that, with an alpha value
given as .05, the State would need to have failed 27 percent of cases in
the sample to conclude that the State would fail more than 20 percent of
cases in the universe. Tr., pp. 35-36. /6/ Instead, the State here (9)
failed only 24.7 percent of the 85 cases sampled, or 21 cases.
Twenty-seven percent of 85 is 22.95, so the State would need to have
failed one or two more cases than the 21 cases which the Agency found
that the State failed, according to the State's calculation.
We conclude that the Agency's determination to fail the State at the 85
case level was not statistically valid since the Agency did not consider
the confidence level of its inference that the State would fail 20
percent of cases in the universe. As we stated above, the theoretical
justification of any type of sampling procedure is that one may draw
conclusions about the universe of cases from a sample, within a certain
range of error. While we understand that sequential sampling has some
differences from standard sampling of a fixed quantity of cases, any
valid sampling scheme must account for the confidence level of the
conclusions it seeks to draw. Otherwise, any inferences about the
universe of cases from the sample would be based upon pure chance, with
no attempt to define or limit the error that is likely with any sample
of a small number of observations.(10)$% One argument of the Agency
appeared to be that the theory of confidence levels and variance between
samples and a universe are not relevant to sequential sampling (thus
appearing to admit that its methodology as used in its review of the
State did not account for the confidence level of its finding). The
Agency's witness thus submitted that the State witness's detailed
explanation of variance, and how the Agency's decision rule did not
consider variance, was irrelevant.
There is no authority to which we have been referred which exempts
sequential sampling from the theoretical rule that inferences about a
universe population from a sample can be made only with a specified
degree of scientific certainty. The treatise by the original developer
of sequential sampling makes no such statement, and, in fact, considers
in great detail the effect of the methodology on the values of alpha and
beta. See A. Wald, Sequential Analysis (1947), especially pp. 40-44.
The goal of sequential sampling appears to be that it will afford the
likelihood of sampling a relatively small number of cases with the same
degree of scientific certainty as with standard sampling.
The State's position in this appeal is based on simple logic, as well as
being in accord with the theoretical justification for sampling. If the
significance of the "zone of indifference" is that the confidence
level
is too low there to make a decision with a proper level of certainty, we
fail to see why the level of certainty would be any more acceptable at
the 80 or 85 case level than earlier on in the sequence. Since the
purpose of defining the "accept" and "reject" lines was
to avoid results
which are statistically invalid, the decision to reject the State merely
because it remained in the zone of indifference at the 85th case is
equally invalid.
C. Other Arguments of the Agency
The Agency argued that the decision to truncate the sample at the 85
case level itself made valid the Agency's decision to fail the State at
that point. Tr., pp. 172-174. In support of this, the Agency referred
to the Average Sample Number (ASN) curve as developed by A. Wald in
connection with sequential sampling. The Agency appeared to imply that
the ASN curve dictated the choice of 85 cases as the truncation point
and that therefore it was proper to fail the State at that level.
The State's rebuttal of this argument was two-fold: first, that the ASN
curve does not, in fact, say where one should truncate a sequential
sample, and, secondly, that the decision of where to truncate the sample
does not dictate the decision to fail states which remain in the zone of
indifference at that point.(11)
The ASN curve, as its name implies, explains the point in the sequence
where, on average, a decision point will be reached with a given rate of
compliance in the universe, in this case, 80 percent. See A. Wald,
Sequential Analysis (1947), pp. 25-26. Although the Agency implied that
Wald himself advocated the relevant point on the ASN curve as the proper
truncation point, a reading of Wald's treatise does not indicate this.
Instead, Wald described a logarithmic function for deciding the point of
truncation, which he described as a "simple and reasonable rule."
Id. at
61; see also pp. 25-27, 61-65. /7/
We do not finally decide whether the ASN is a proper point at which to
truncate the sample. As we find above, it appears significant only that
some methodology is used that properly incorporates consideration for an
appropriate confidence level of a finding. If it is possible
mathematically to account properly for the alpha value at a truncation
point that is equal to the ASN, then the Agency should be able to
truncate the sample at that point.
The State itself acknowledged the possibility of properly truncating the
sample at the ASN number by offering an easy rule which it claimed would
result in an alpha value of .06 and a beta value of .07 at that level.
See Appellant's Post Hearing Brief, pp. 4-5. This rule is to divide the
zone of indifference in half and accept the State if it is closer to the
accept line and reject the State if it is closer to the reject line.
The Agency rejected the suggestion of the State to "split the
difference" in the zone of indifference, as it described the State's
proposal. Agency's Post Hearing Reply Brief, pp. 2-3. As we explain
above, we do not address whether this particular rule is the one the
Agency must follow. We find only that if the Agency chooses not to
continue sequential sampling until a decision point is reached and
truncates the sample, it must adopt some approach that considers the
statistical significance of its finding by properly accounting for the
alpha level that has been accepted for the sampling methodology as a
whole.
In its final brief, the Agency appeared to argue that the State's
proposed rule to "split the difference" at the point of truncation
would
unfairly increase the likelihood of(12) passing the State when its
actual compliance level in the universe would be unacceptably low.
Agency's Post Hearing Reply Brief, pp. 2-3. As we find above, we do not
advocate any particular methodology as to how the Agency might consider
a state's performance at the point of truncation. We find only that if
the Agency chooses to truncate the sample, it must do so in a manner
which guarantees the State the same .05 alpha value that was established
as an acceptable level for the sampling procedure as a whole or uses
some other alpha value to which the State agrees. Further, the Agency's
arguments about the risk of unfairly passing the State do not go to the
alpha value at all, but to the beta value. If the Agency is
dissatisfied with the beta value at the truncation point, it can always
continue sequential sampling until a decision point is reached.
Another argument of the Agency was that, for an "alternative SPRT,"
the
State would in fact have reached a negative decision point by the 85
case level. Agency's Post-Hearing Reply Brief, pp. 4-5. In other
words, the Agency appeared to argue that if a different decision table
were constructed according to a function of another SPRT, the State
would have failed the section 427 review before the levels of 80 or 85
cases were reached. The Agency calculated the SPRT for alternate
compliance levels of 90 percent and 87 percent. Applying an "Operating
Characteristic (OC)" curve, the Agency calculated that the State would
have failed the review before reaching the truncation point. /8/
This calculation does not affect our analysis since it says nothing
about the State's performance at an 80 percent compliance level. The
Agency adopted the policy of accepting states' performance as
satisfactory if 80 percent of the total cases were acceptable, and it is
not relevant how states might have been evaluated hypothetically using a
different level of compliance. We therefore fail to see the relevance
of the Agency's arguments about the OC curve to our conclusion.
II. The Four Cases Identified by the State
In part I of this decision, we remand the case to the Agency with the
option of continuing sequential sampling until a (13) decision point is
reached according to the decision table as defined by the SPRT. We
therefore address the four cases /9/ which the State argued were wrongly
decided by the federal reviewers, in order to guide the parties in
knowing how many cases would need to pass or fail in the continued
sequence. /10/
Applicable Law
As one of the conditions for the receipt of additional child welfare
funds under title IV-B, section 427(a)(2)(B) of the Act requires that a
state have implemented and be operating to the satisfaction of the
Secretary --
A case review system (as defined in section 475(5)) for each child
receiving foster care under the supervision of the State. . . .(14)
Section 475(5) provides that--
(5) the term "case review system" means a procedure for assuring
that--
* * *
(B) the status of each child is reviewed periodically but no less
frequently than once every six months by either a court or by
administrative review (as defined in paragraph (6)) in order to
determine the continuing necessity for and appropriateness of the
placement, the extent of compliance with the case plan, and the extent
of progress which has been made toward alleviating or mitigating the
causes necessitating placement in foster care, and to project a likely
date by which the child may be returned to the home or placed for
adoption or legal guardianship, and
(C) with respect to each such child, procedural safeguards will be
applied, among other things, to assure each child in foster care under
the supervision of the State of a dispositional hearing to be held, in a
family or juvenile court or another court (including a tribal court) of
competent jurisdiction, or by an administrative body appointed or
approved by the court, no later than eighteen months after the original
placement (and periodically thereafter during the continuation of foster
care), which hearing shall determine the future status of the child
(including, but not limited to, whether the child should be returned to
the parent, should be continued in foster care for a specified period,
should be placed for adoption, or should (because of the child's special
needs or circumstances) be continued in foster care on a permanent or
long-term basis); and procedural safeguards shall also be applied with
respect to parental rights pertaining to the removal of the child from
the home of his parents, to a change in the child's placement, and to
any determination affecting visitation privileges of parents.
Section 475(6) provides:
(6) The term "administrative review" means a review open to the
participation of the parents of the child, conducted by a panel of
appropriate persons at least one of whom is not responsible for the case
management of, or the delivery of services to, either the child or the
parents who are the subject of the review.(15)
A. The case of "Loretta"
The State argued that one of the cases it identified was improperly
failed because the reason a periodic review and dispositional hearing
were not held in a timely manner was because a court had postponed a
scheduled hearing which would have fulfilled those requirements.
State's Ex. M. However, as we explain further below, the State
presented no reason why both the periodic review and dispositional
hearing that were required for the child were postponed. Assuming that
the postponement of the dispositional hearing was excused by the court's
action, there appears no reason why the administrative agency concerned
could not have held the required periodic review, which need not be held
in court. We therefore conclude that the Agency properly found this
case unacceptable.
The dispositional hearing and periodic review for this child ("Loretta")
were both due on December 31, 1982, according to the State. State's Ex.
M.; see Tr., pp. 100-101. A Termination of Parental Rights (TPR)
hearing scheduled in court for November 24, 1983 was designed to fulfill
the requirements for both a dispositional hearing and periodic review.
The TPR hearing was postponed by order of the court to January 28, 1983,
some four weeks after the periodic review and dispositional hearing were
due. The State introduced a series of telephone messages which appeared
to explain that the reason for the postponement was the inability of a
psychiatrist to testify at the scheduled November hearing. State's Ex.
M-5; Tr., pp. 72-73.
We find that the Agency properly found the case of Loretta unacceptable.
Even if the court's decision to postpone the TPR hearing were to excuse
the failure to hold a timely dispositional hearing, the State was not
prevented from holding a timely periodic review, which could have been
held by the administrative agency itself. Section 475( 5)(B)
specifically provides that a periodic review may be held by "either a
court or by administrative review." The State presented no evidence as
to why it did not attempt to hold, at the least, a periodic review in a
timely manner. We therefore do not address the State's suggestions that
there were "due process" reasons for postponing the TPR hearing which
excused its delay. Tr., pp. 73-74.
B. The case of "Willie"
For one case, the State argued that a court's "arbitration hearing"
should be accepted as a timely dispositional hearing, even though it was
not intended initially to comply with the requirements for a
dispositional hearing. State's Ex. L. The arbitration hearing, whose
timeliness for(16) purposes of fulfilling the dispositional hearing
requirement was not challenged, was the only hearing offered to fulfill
the dispositional hearing requirement for this child ("Willie").
The arbitration hearing for Willie was essentially a delinquency
proceeding brought for a charge of "offensive touching." See Tr.,
p.
89. The arresting police officer was present at the proceeding, which
prescribed a list of punishments for the offense. A State's witness
testified that arbitration hearings have previously been accepted as
dispositional hearings, although he provided no specific examples of
when they were. Tr., p. 160.
We do not decide whether arbitration hearings could ever be acceptable
as dispositional hearings, for we find in this case that the hearing for
Willie has not been shown to satisfy the statutory purposes of a
dispositional hearing and that the Agency correctly found the case
unacceptable.
The function of a dispositional hearing is to decide the "future status
of the child (including, but not limited to, whether the child should be
returned to the parent, should be continued in foster care for a
specified period, should be placed for adoption, or should (because of
the child's special needs or circumstances) be continued in foster care
on a permanent or long-term basis). . . ." Section 475(5)(C) of the Act.
A witness for the State testified that a function of the hearing for
Willie was to decide the "future status of the child" and that "(p)
otentially, (the court) could have made a decision for him to be removed
from Murphy's School (apparently the child's foster care placement) and
placed elsewhere." Tr., p. 90.
However, the State's witness did not attend Willie's hearing and the
record of the proceeding, State's Ex. L-4, clearly indicates that the
court considered only the violation for which Willie was charged and the
punishments which it deemed appropriate. The report does not discuss
the general care of the child and there is no indication that those
responsible for his care even attended the proceeding. The court's
record of the previous dispositional hearing for Willie, by contrast,
described the terms of the child's placement, noted that it was attended
by all those responsible for his care, and commented that the "Social
Worker shall check with the Foster Home Locator at least once a month to
ascertain whether a foster home has become available." State's Ex. L-3.
We therefore conclude that the arbitration hearing conducted for Willie
did not fulfill the statutory purposes for a dispositional hearing and
that the Agency properly found unacceptable that one case.(17)$% C. The
case of "Janet"
For one case which failed, the State argued that the Agency should have
accepted the State's "supervisory review" as an "administrative
review"
that would have fulfilled the statutory requirement for a periodic
review. State's Ex. N. The supervisory review for the child ("Janet")
was instead rejected because it was not conducted in part by an
independent person, as required by section 475(6) of the Act. The basis
for the State's argument was that the Agency had accepted supervisory
reviews in the previous year and the State had been led to believe that
they were acceptable.
We find that the Agency was correct in finding that the case of Janet
should not pass, since the words of the statute are unambiguous in
requiring "administrative reviews" to be conducted in part by an
independent person. Section 475(5)(B) of the Act requires as part of a
case review system demand that the status of foster care children be
evaluated every six months by either a court or by an administrative
process that will assure many of the same protections as would a court.
Section 475(6) requires that administrative reviews be conducted by a
panel of persons, at least one of whom is independent of the care of the
child.
The State did not dispute that supervisory reviews as conducted by the
State were not conducted by a panel of persons that included someone
independent of the child's care. Instead, it argued that the Agency
should be equitably estopped from failing such cases on the basis that
the Agency approved the State's practice in the previous year's section
427 review. The Agency's report of the previous year found:
Internal Administrative Reviews are reasonable methods of meeting
Certification requirements and DCPS policy with regard to this matter is
good. However, the review process will meet the purposes for which it
was established only if parents participate. By the same token, it is
our belief that the process will be strengthened by the participation of
independent persons who do not have responsibility for the management of
the case. Therefore, it is recommended that DCPS continue to
concentrate on the implementation of this policy to bring the Internal
Review to its full potential.
Agency's Ex. 1, p. 9. The State argued that this constituted approval
of the State's practice of holding supervisory(18) reviews as periodic
reviews and that the State detrimentally relied upon that advice. See
Appellant's Opening Brief, pp. 7-13. /11/
While we respond to the State's estoppel argument more specifically
below, we uphold the Agency's determination because the State's use of a
supervisory review was not in accordance with the clear statutory
requirement that an "administrative review" must be "conducted
by a
panel of appropriate persons, at least one of whom is not responsible"
for the care of the child. Section 475(6) of the Act. In enacting this
provision, Congress chose to define specifically what types of reviews
could qualify as periodic reviews. While Congress may have given the
Agency some discretion to consider whether the requirements of section
475(5) and (6) were fulfilled, /12/ we find that the Agency had no
discretion to consider whether the State must comply with a clear
definition of what constituted an administrative review.
While the State may argue that the Agency had some discretion to decide
how stringently it would enforce the requirements of section 475(5) and
475(6), we are not persuaded that this is so. The words of section
475(6) clearly define what constitutes an acceptable administrative
review and, as explained above, the Act gives no authority to alter that
definition. What the State may regard as the Agency's discretionary
authority relates only to the Agency's compliance review process, and
not to the statutory requirements of section 475(5) and 475(6). The
Agency categorized requirements of the Act into "major" and "minor"
safeguards, of which only "major" safeguards need be entirely complied
with in order to find a particular case acceptable by the section 427
reviewers. See Agency's Ex. 9, PI 82-06. Yet, whether a requirement is
categorized as a major or minor safeguard for purposes of the review of
a state's files, the State knew, or should have known, that the
statutory definition of an administrative review required the presence
of an independent person. (19)
As one federal court found, in reviewing a state's efforts to comply
with sections 427 and 475:
The only possible limiting language of the statute is the clause "has
implemented and is operating to the satisfaction of the Secretary."
(Section 427(a)(2) of the Act). But this language is relevant, if at
all, to the question whether Congress intended an exclusive remedy to
lie with the Secretary. . . . It cannot be read to limit the existence
or scope of the obligation imposed on the states. To interpret the
phrase "to the satisfaction of the Secretary" as transforming an
otherwise clear obligation to comply with the requirements of section
(475(5)) into a mere duty to achieve whatever degree of conformity with
those requirements is needed to satisfy the Secretary would be contrary
to the Congressional purpose.
Lynch v. King, 550 F. Supp. 325 at 350 (D. Mass., 1982).
Given our finding that the presence of an independent person at an
administrative review is a clear statutory requirement, we cannot accept
the State's claim of estoppel. The State argued that it had established
the elements necessary to demonstrate a prima facie case of estoppel,
including the requirement that the one asserting the estoppel must have
reasonably relied upon the other's advice. Appellant's Opening Brief,
p. 10. /13/
Since we find that the language of the statute was clear, the State's
attempt to rely on an interpretation of the statute contrary to its
clear meaning was inherently unreasonable. The State may have thought
that it would be found in compliance in spite of some failures in
meeting the clear obligation of having an independent person present,
but the State could not use this as an excuse for deliberately failing
to meet its obligation.
D. The case of "Bonnie"
For one case which failed, the State argued that the case should not
have been reviewed since it replaced another case(20) which was
improperly rejected from the sample population. /14/ State's Ex. K. The
State argued that the case file for the child ("Bonnie") should have
been reviewed, rather than the case which failed, since a periodic
review or dispositional hearing was "due or held" before Bonnie's
18th
birthday, a policy which was agreed upon by both parties at the time of
the review. As we explain further below, we conclude that Bonnie's case
should have been reviewed, rather than the alternate case which failed
the review.
The parties appeared not to dispute that some days before the section
427 review of State records the State informed the Agency that foster
children over 18 years of age were no longer in the custody of the
State, because of a ruling of the State Family Court. See Tr., pp.
78-80; Agency's Ex. 8. The State therefore advised the Agency that
children over 18 should not be considered in the Agency's review of
foster care records. State's Ex. E. On the day of the review, the
State and Agency representatives apparently agreed on the policy that
children whose periodic review or dispositional hearing was either "due
or held" before their 18th birthday would be included in the sample
population, but those children whose dispositional hearings or periodic
reviews were not due or held until after their 18th birthday would be
rejected and replaced with an alternate. The rule was written on a
blackboard in the room in which the reviewers actually evaluated the
State's foster care case records. See Tr., pp. 120-121, 134.
The Agency did not dispute that the reviewers agreed on a policy that
only children whose periodic reviews or dispositional hearings were due
or held before their 18th birthday would be included in the sampled
cases. See Tr., p. 226. The Agency instead argued, first, that the
Agency was informed of the State's determination to exclude 18 to 21
year olds only two days before the review of files began, and, secondly,
that the State representatives were given a choice as to whether those
children were included or excluded. See Agency's Post-Hearing Brief, p.
5: Tr., pp. 225-226. /15/
(21)
We do not consider these factual disputes as identified by the Agency,
since we fail to see how they affect the rule agreed upon on the day of
the review as to which children should be included. Since the Agency
does not dispute the existence of the rule, we conclude simply that the
reviewers erred by excluding the case of Bonnie in the sample
population. (The Agency's representative apparently made the same error
again at the hearing before the Board. See n. 15, above).
We therefore find that the Agency, on remand, should consider the case
of "Bonnie" rather than the case which failed the review. /16/ If
the
Agency chooses to continue sequential sampling, it should include the
case of Bonnie as part of the sampling. Similarly, if the Agency
chooses to develop a new methodology in reviewing the State at a point(
22) of truncation, it should first consider the case of "Bonnie" to
evaluate the State's proper position on the decision table.
Conclusion
In Part I of the decision, we remand the case to the Agency because we
find that the Agency's determination that the State was out of
compliance with section 427 of the Act in FY 1983 was not in accordance
with the Agency's own policies regarding the sampling of state's files
and was not in accordance with the statistical basis of sampling. On
remand, the Agency has the option of either continuing sequential
sampling until a decision point is reached, or adopting some policy that
considers an appropriate confidence level of its finding even when no
decision point is reached. In Part II, we uphold the Agency's
determination with regard to three of the four cases identified by the
State which the State argued were improperly found unacceptable by the
reviewers. We find that the State failed to comply with statutory
requirements for those three cases (sections A-C of Part II). In
section D of Part II, we reverse the Agency's determination with regard
to one case, where we find specifically that one case was wrongly
rejected from the sample population and was replaced with an alternate
case that failed. On remand, the Agency should instead consider the
case of "Bonnie" in its reconsideration of the State's records. /1/
The
additional funds are a proportional share of the amount
appropriated for title IV-B which exceeds $141,000,000, in this case,
$293,914. /2/ The Agency's policy instruction explains: 1. In
standard inspection (review) a sample of predetermined size is drawn
from the state files in random fashion. Based on a complete review of
all case records in this sample the state is considered eligible,
provisionally eligible, or ineligible using the standards for
eligibility previously discussed. 2. In sequential sampling inspection
(review) the size of the sample is left undetermined and the procedure
is to review one or several case records at a time with the review
continuing until the cumulative evidence is sufficiently strong for the
state to be considered eligible, provisionally eligible, or ineligible.
Agency's Ex. 9, Program Instruction 82-06, Att. E, p. 2. (Provisional
eligibility is relevant only to an initial year's review of section 427
eligibility; the review of the State here was in a subsequent year.)
/3/ At some time before the start of the review of the State's
files, the Agency made two changes in the manner in which it would
review a state. The Agency increased the number of total files that
could be reviewed from 80 to 85, and it changed the alpha value from .05
to .01. to 85, and it changed the alpha value from .05 to .01. See Tr.,
pp. 170-171. The new sample size of 85 cases became the point of
truncation at which the Agency found the State should fail the review.
At one point in this appeal, the State argued that the increase in
sample size penalized the State. Since we find for the State on the
central statistical issue in this dispute, we do not address whether the
increased sample size harmed the State. The change in the alpha value
from .05 to .01 also appears irrelevant to our analysis. The effect of
the decreased alpha was to "widen" the zone of indifference, thus
making
it more difficult for the State to reach a negative decision point
during the sequential sampling. (This is the explanation for the two top
diagonal lines on the chart on page 4; the top solid line corresponds
to an alpha value of .01 and the broken line directly below it
corresponds to an alpha value of .05. The cross-hatched area between the
two lines corresponds to the change in the width of the zone of
indifference). Since the State never reached a decision point under
either the assumption of an alpha of .01 or of .05, the change is
irrelevant for purposes of this appeal. The parties usually referred to
an alpha value of .05 in their arguments before the Board and we assume
this same value for purposes of this decision. /4/ The Agency's
written policy statement, Program Instruction 82-06, appears to advocate
this approach. However, the explanation of the policy is confusing
because it speaks only in terms of the percentage of failed cases in the
sample, as if the performance of the State for the universe of cases
would be identical. See Agency's Ex. 9, Att. E, p.
corresponding to the decision to fail the State appeared confusing. See
Tr., pp. 185-218. In post-hearing briefing, however, the Agency made no
effort to deny that the alpha value of the Agency's decision was not
.05. Agency's Post-Hearing Reply Brief, pp. 2-5. /6/ The
State's witness derived the formula which calculated the result of 27
percent, given an alpha value of .05 and a compliance rate of 80
percent. The calculation may be explained in the form of a bell curve:
(SEE ILLUSTRATION ORIGINAL) See State's Hearing Ex. P. In the diagram, p
= accepted rate of failure in the universe; 24.7 = actual percentage
rate of failure of the State = 21/85; 27.0 = minimum acceptable
percentage of failures according to calculation. The cross-hatched area
is the range of numbers of failed cases which one can conclude imply a
failure rate of 20 percent of cases in the universe, given an alpha of
.05. /7/ The Agency cited no other authority for truncating the
sample at the ASN. According to the State, one authority advocates
truncating the sample size at three times the ASN. See Appellant's
Post-Hearing Brief, p. 3, n. 2, citing B.K. Ghosh, Sequential Tests of
Statistical Hypotheses, 5.2 at 221 (1970). /8/ The ASN curves and OC
curves are the two tests described by Wald in evaluating any sequential
sample. Wald states, "The OC function describes how well the test
procedure achieves its objective of making correct decisions, and the
ASN function represents the price we have to pay, in terms of the number
of observations required for the test." A. Wald, Sequential Analysis
(1947), p. 27. /9/ The State initially argued that an additional
case was failed because a periodic review was cancelled since the child
was placed in a pre-adoptive placement at the time the review was due.
Appellant's Opening Brief, p. 4. The State presented the records for
one such case in response to the Board's request. State's March 21,
1985 response to Question 5 of Board's Order to Develop the Record;
State's Ex. G-1. The Agency objected that this case did not present the
legal issue of whether children in pre-adoptive placements need to have
periodic reviews, since the periodic review there was in fact both due
and cancelled one week before the pre-adoptive placement was to go into
effect. We presented a preliminary conclusion that the Agency was
correct, but allowed the State to make further argument on the factual
or legal issue if it wished to do so. Confirmation of Telephone
Conference and Notice of Hearing, May 1, 1985, p. 3. The State made no
further argument on the issue. /10/ The Agency objected to the
State's "resurvey" which identified the four cases discussed in this
part, because it was not "verified" by the Agency. Agency's
Post-Hearing Brief, p. 9. However, it appears that the only manner in
which states could usually challenge OHDS's survey findings would be to
identify specific cases which they allege were wrongly found
unacceptable. The Board has considered such cases in other section 427
decisions and we find it proper to do so here. See, e.g., Arkansas
Department of Human Services, Decision No. 553, July 16, 1984 and
Florida Department of Health and Rehabilitative Services, Decision No.
643, April 19, 1985. /11/ We note that a witness for the Agency
testified that the Agency verbally informed the State at the
time of the previous year's review that supervisory reviews were not
acceptable substitutes for periodic reviews. Tr., p. 236. /12/
The source of such discretionary authority is the language of section
427(a)(2), that a state must have ". . . operating to the satisfaction
of the Secretary -- (a) case review system (as defined in section
475(5)) for each child receiving foster care under the supervision of
the State. . . ." (emphasis added). /13/ The Supreme Court has
emphasized the requirement that a party's detrimental reliance must be
reasonable, in order for a claim of estoppel to lie. "(The) reliance
must have been reasonable in that the party claiming the estoppel did
not know nor should it have known that its adversary's conduct was
misleading (citations omitted)." Heckler v. Community Health Services of
Crawford County, 467 U.S. 51, 104 S.Ct. 2218 at 2223 (1984).
/14/ The reviewers compiled a pool of "alternates" to be used to replace
cases that were rejected from the original sample population. The
"rejection" of cases from the sample should be distinguished from
cases
which failed or were found unacceptable; rejected cases were not
reviewed, but were replaced with alternate cases. /15/ At one
point in the hearing, a witness for the Agency testified that the case
of Bonnie was properly rejected from the sample under the agreed upon
rule. Tr., pp. 226-227. After articulating the rule that only children
whose periodic reviews or dispositional hearings were due or held before
their 18th birthday should be included in the sample population, the
witness stated that Bonnie was properly rejected, since "(t)he child
turned 18 on 12/16/82 and there was no requirement prior to the child's
birthday." Tr., pp. 227. This is clearly wrong. While no hearing or
review may have been due before the child's 18th birthday on December
16, 1982, the State Family Court held a hearing on September 22, 1982,
which the Agency found satisfied the requirements for both a periodic
review and dispositional hearing that were both due by December 31,
1982. /16/ The State argued that if the case of Bonnie were not
included in the review, and the cases of Loretta, Willie, and Janet
(discussed in sections A-C, above) were found acceptable, the State
would have passed the review by the 45th case. Tr., p. 140. The State
"reconstructed" the decision table to demonstrate this. See State's
Ex.
O. Since we have found for the Agency in sections A-C, we do not need
to address the effect of the reconstruction. The State also initially
argued that the alternates for cases rejected from the sample were taken
in the improper order, but admitted at the hearing that its
reconstruction of the decision table revealed that the order of the
alternates had no effect on the outcome. Tr., p. 149.
JANUARY 14, 1986