<DOC>
[109 Senate Hearings]
[From the U.S. Government Printing Office via GPO Access]
[DOCID: f:29503.wais]
S. Hrg. 109-923
AUTOPILOT BUDGETING: WILL CONGRESS
EVER RESPOND TO GOVERNMENT
PERFORMANCE DATA?
=======================================================================
HEARING
before the
FEDERAL FINANCIAL MANAGEMENT, GOVERNMENT
INFORMATION, AND INTERNATIONAL
SECURITY SUBCOMMITTEE
of the
COMMITTEE ON
HOMELAND SECURITY AND
GOVERNMENTAL AFFAIRS
UNITED STATES SENATE
ONE HUNDRED NINTH CONGRESS
SECOND SESSION
__________
JUNE 13, 2006
__________
Available via http://www.access.gpo.gov/congress/senate
Printed for the use of the Committee on Homeland Security
and Governmental Affairs
U.S. GOVERNMENT PRINTING OFFICE
29-503 PDF WASHINGTON : 2007
---------------------------------------------------------------------
For sale by the Superintendent of Documents, U.S. Government
Printing Office Internet: bookstore.gpo.gov Phone: toll free (866)
512-1800; DC area (202) 512-1800 Fax: (202)512-2250 Mail: Stop SSOP,
Washington, DC 20402-0001
COMMITTEE ON HOMELAND SECURITY AND GOVERNMENTAL AFFAIRS
SUSAN M. COLLINS, Maine, Chairman
TED STEVENS, Alaska JOSEPH I. LIEBERMAN, Connecticut
GEORGE V. VOINOVICH, Ohio CARL LEVIN, Michigan
NORM COLEMAN, Minnesota DANIEL K. AKAKA, Hawaii
TOM COBURN, Oklahoma THOMAS R. CARPER, Delaware
LINCOLN D. CHAFEE, Rhode Island MARK DAYTON, Minnesota
ROBERT F. BENNETT, Utah FRANK LAUTENBERG, New Jersey
PETE V. DOMENICI, New Mexico MARK PRYOR, Arkansas
JOHN W. WARNER, Virginia
Michael D. Bopp, Staff Director and Chief Counsel
Michael L. Alexander, Minority Staff Director
Trina Driessnack Tyrer, Chief Clerk
FEDERAL FINANCIAL MANAGEMENT, GOVERNMENT INFORMATION, AND INTERNATIONAL
SECURITY SUBCOMMITTEE
TOM COBURN, Oklahoma, Chairman
TED STEVENS, Alaska THOMAS CARPER, Delaware
GEORGE V. VOINOVICH, Ohio CARL LEVIN, Michigan
LINCOLN D. CHAFEE, Rhode Island DANIEL K. AKAKA, Hawaii
ROBERT F. BENNETT, Utah MARK DAYTON, Minnesota
PETE V. DOMENICI, New Mexico FRANK LAUTENBERG, New Jersey
JOHN W. WARNER, Virginia MARK PRYOR, Arkansas
Katy French, Staff Director
Sheila Murphy, Minority Staff Director
John Kilvington, Minority Deputy Staff Director
Liz Scranton, Chief Clerk
C O N T E N T S
------
Opening statements:
Page
Senator Coburn............................................... 1
Senator Carper............................................... 6
WITNESSES
Tuesday, June 13, 2006
Hon. Clay Johnson III, Deputy Director for Management, U.S.
Office of Management and Budget................................ 8
Eileen Norcross, Government Accountability Project, Mercartus
Center at George Mason University.............................. 9
Adam Hughes, Director for Federal Fiscal Policy, OMB Watch....... 11
Alphabetical List of Witnesses
Hughes, Adam:
Testimony.................................................... 11
Prepared statement with attachments.......................... 89
Johnson, Hon. Clay III:
Testimony.................................................... 8
Prepared statement with attachments.......................... 25
Norcross, Eileen:
Testimony.................................................... 9
Prepared statement........................................... 49
APPENDIX
``A Working Paper in Government Accountabiity'' by Eileen
Norcross and Kyle McKenzie..................................... 61
AUTOPILOT BUDGETING: WILL CONGRESS
EVER RESPOND TO GOVERNMENT
PERFORMANCE DATA?
----------
TUESDAY, JUNE 13, 2006
U.S. Senate,
Subcommittee on Federal Financial Management,
Government Information, and International Security
of the Committee on Homeland Security and
Governmental Affairs,
Washington, DC.
The Subcommittee met, pursuant to notice, at 2:57 p.m., in
room 342, Dirksen Senate Office Building, Hon. Tom Coburn
(Chairman of the Subcommittee) presiding.
Present: Senators Coburn and Carper.
OPENING STATEMENT OF SENATOR COBURN
Senator Coburn. Good afternoon. The Federal Financial
Management Subcommittee of the Homeland Security and
Governmental Affairs Committee will come to order. Senator
Carper will be here in a moment. We apologize for the delay.
There was an official photo. We also have a conflict. There is
a briefing ongoing now by the Secretary of Defense and the
Secretary of State, which will limit Senator Carper's time with
us. So we are going to go on and go forward so we have it in
the record. I apologize for the conflicting schedules.
Americans have a crazy idea, that they should get something
for their money, even when the money is spent by the
government. It is a simple concept, and in policy-speak we call
it performance-based budgeting. I know I am new in the Senate,
but I am still surprised by the level of resistance in
Washington to holding people accountable by measuring their
performance. And it is a difficult thing to do. A
multitrillion-dollar government imposing some sort of
standardized outcome evaluation is difficult at best, and what
it implies is that the tool will be very crude. But that does
not say we should not attempt to make measurements, and I want
to be one of many who should commend both Mr. Johnson and the
Bush Administration, and the President himself, for being the
first to attempt to do it.
It is not novel. It is required in the competitive business
environment that we find ourselves worldwide. It is being used
effectively in many State governments, and it is something that
is long overdue. The Performance Assessment Rating Tool (PART)
was first introduced by the President 4 years ago as a tool to
review the strengths and weaknesses of government programs to
influence funding and programmatic decisions. The annual PART
reports offer needed sunshine in government and provide good
data for government managers to improve their programs.
Today, the Office of Management and Budget has reviewed 793
programs, which account for $1.47 trillion in taxpayer money.
Almost a third of these programs have proven to not meet up to
standards based on the PART analysis. I have already admitted
that it is a blunt tool. One-third of $1.5 trillion is $500
billion. Maybe this is why PART scores so far have created a
stir not only among the agencies but among the Members of
Congress who make budgeting decision.
Some Members of Congress want to stick their heads in the
sand and keep funding pet programs on autopilot year after
year. To my amazement, just last week, the Appropriations
Subcommittee that funds the Departments of Labor and Health and
Human Services passed language prohibiting the use of PART
assessments on those agencies. They may not like PART's
message, but they should not shoot the messenger. This sort of
Orwellian immunization against any hint that our favorite
programs may not be performing up to the idealized, utopian
goals of their Congressional champions is one of the reasons
why Americans are mad at Congress.
The approval ratings for Congress are in the tank, and this
prohibition of accountability for failing government is why the
voters who fork over their hard-earned dollars every year may
just have something new to say come this November. I am not
sure why so many of my colleagues are afraid of assessment
tools on performance. It may reflect their own performance.
As part of our investigation for this hearing, we learned
that low PART ratings do not always mean that OMB will
recommend a budget cut or a cut in the program or a
recommendation to go on the terminations list. In some cases,
programs rated ineffective have had budget reductions
recommended. But in other cases, the reason they were low was
because they were not funded appropriately to begin with, and
therefore, they could not accomplish what they were intended to
because they did not have adequate funding.
Each program is unique, and I do not know that a PART score
should be the last word. But I do know that measurement of
performance is something that every member of a Congressional
authorizing or Sppropriations committee should be reading and
using to inform their oversight work. Congress consistently
neglects the duty to conduct oversight of Federal programs and
spending. Instead, we spend most of the time passing spending
bills that ignore PART ratings, the President's termination
list, or any other performance data as if the spending were on
autopilot. Congress might as well write a blank check.
By 2008, OMB will have applied PART to the entire
government. In the last 4 years, OMB has scored 793 government
programs. Here are the results: 15 percent were found to be
effective; 29 percent were found to be moderately effective; 28
percent were rated adequate; 4 percent were found to be
ineffective--that is one in every 25 programs--25 percent could
not demonstrate results to get a rating and were labeled
results not demonstrated.
I do not believe the spin that results not demonstrated can
mean that the program is either good or bad; we just do not
have enough information to tell. On the contrary, the results
not demonstrated designation is a red flag marking a program so
poorly conceived by us or so directionalist that that
unaccountability seems to have been built into it by design.
Programs rated ineffective or results not demonstrated account
for $152 billion in budget authority. Imagine what we could do
with $152 billion right now. The ideas are endless.
Outside of Washington, DC, any business or family with
finite resources sets priorities and creates a budget based on
the actual amount of bang they get from their buck. It is only
inside the Beltway where that kind of information is not
considered relevant, and in fact, some are even attempting to
ban the collection of such information. But then, it is only
Washington where you never have to declare bankruptcy, and debt
is allowed to grow on the backs of future generations with
impunity.
Let me give you one case study, and my co-chairman on this
will disagree, but my firm believing is the following: We held
a hearing last year on the Advanced Technology Program that was
created in 1988 to subsidize high-risk research and
development. This program has never demonstrated results. What
it has demonstrated is corporate welfare. Its 2002 PART report,
that the majority of ATP grants go to multibillion dollar
corporations and that the GAO has found that ATP projects are
very similar to private sector R&D undertaken without a
government subsidy. An amendment to eliminate this funding that
was offered last year lost by a vote of 68 to 29. In the end,
Congress wasted a portion of $79 million last year for that
program. The 2007 Senate budget resolution promises to fund the
program at almost twice that amount.
It would be one thing if we were operating in a surplus.
Then, we could have a legitimate debate about whether to keep
failing programs, hoping that they would improve, or to give
that surplus back to the taxpayers. But that is not where we
are today. With a debt burden of $25,000 per man, woman, and
child, we simply cannot afford to keep funding programs that
cannot prove their worth. Non-defense discretionary spending
has increased 45 percent since 2001. The President has
requested a $2.8 trillion budget, and that does not include any
of the so-called emergency, ``supplemental bills in our
future,'' nor does it include the late night pork barrel frenzy
each time Congress schedules an appropriations bill vote.
Entitlement spending will tank our economy if we do not do
something to get spending under control. The question remains:
How do we get Congress to act? I would like to see OMB sell
their PART terminations list more aggressively, forcibly sell
the reforms and savings to Congress, fight for the cuts by
taking the terminations list to the American people with the
power of the bully pulpit. The President should veto spending
bills that continue to issue blank checks for failing programs.
There is a bit of hope on the horizon. I was encouraged to
see that the House Appropriations Committee wrote in their 2006
budget savings report that the only way to establish
accountability in the budget process is to stop spending on
programs that have outlived their usefulness or could be
delivered more effectively at the State or local level. I will
believe that when I see it, but I welcome any help that we can
get.
The best place to start is by immediately defunding all
programs on the termination list and adopting other PART
recommendation reductions. Granted, the list only cuts $20
billion from a $2.8 trillion budget, but we have got to start
somewhere. What is more, we should suspend the creation of any
new program until further notice or it is compared to the
existing programs that it is meant to supplement. We need
sunset legislation that would phase out government agencies on
a timed basis, where we force ourselves to look at them and to
reauthorize them.
These are challenging times, and we can no longer afford to
run on a budget that is on cruise control. I want to thank our
witnesses for being here.
[The prepared statement of Chairman Coburn follows:]
OPENING PREPARED STATEMENT OF CHAIRMAN COBURN
Americans have a crazy idea: They should get something for their
money, even when the money is spent by government. It's a simple
concept--in policy-speak, we call it ``performance-based budgeting.'' I
know I'm new in the Senate, but I'm still surprised by how much
resistance there is in Washington to performance-based budgeting.
Now, to be fair, taking a multi-trillion dollar government and
imposing some sort of standardized outcome evaluation on it is
difficult at best. So I concede that any instrument we use will be a
blunt instrument. But I want to commend President Bush for being the
first to try.
The Performance Assessment Rating Tool (PART) was first introduced
by the President 4 years ago as a tool to review the strengths and
weaknesses of government programs to influence funding and programmatic
decisions.
The annual PART reports offer needed sunshine in government and
provide good data for government managers to improve their programs. To
date, the Office of Management and Budget has reviewed 793 programs
which account for $1.47 trillion in taxpayer money. Almost a third of
these programs have proven either totally ineffective or are not
demonstrating results. One-third of $1.5 trillion is $500 billion.
Maybe this is why the PART scores have created a stir--not only
among the agencies, but among the Members of Congress who make
budgeting decisions. Some Members of Congress want to stick their head
in the sand and keep funding their pet programs, as if on autopilot,
year after year.
Just last week the House Appropriations subcommittee that funds the
Departments of Labor, Education and Health and Human Services passed
language prohibiting the use of PART assessments on those agencies.
They may not like PART's message, but they shouldn't shoot the
messenger. This sort of Orwellian immunization against any hint that
our favorite programs may not be performing up to the idealized utopian
goals of their Congressional champions is why Americans are mad at
Congress. The approval ratings for Congress are in the tank, and this
prohibition of accountability for failing government is why the voters
who fork over their hard-earned dollars every year may just have
something to say come November.
I'm not sure why some of my colleagues are so afraid of PART. As
part of our investigation for this hearing, we learned that low PART
ratings don't always mean that OMB will recommend a budget cut or put
the program on the Terminations List. In some cases, programs rated
``ineffective'' had budget reductions, but in other cases their budgets
increased. Each program is unique and I don't know that a PART score
should be the last word, but I do know that the PART is something every
member of a Congressional authorizing or Appropriations committee
should be reading and using to inform their oversight work.
You see, Congress consistently neglects the duty to conduct
oversight of Federal programs and spending. Instead, we spend most of
the time passing spending bills that ignore PART ratings, the
President's terminations list and any other performance data. It is as
if we're spending on ``auto pilot''--Congress might as well just write
a blank check.
By 2008, OMB will have applied PART to the entire government. In
the last 4 years OMB has scored 793 government programs. Here are the
results: Just 15 percent were found to be ``effective''; 29 percent
were rated ``moderately effective''; 28 percent were rated
``adequate''; 4 percent were found to be ``ineffective''; and 24
percent cannot demonstrate results to even get a rating and were
labeled ``results not demonstrated''! Don't believe the spin that
``results not demonstrated'' could mean that the program is either good
or bad, we just don't have enough information to tell. On the
contrary--the ``results not demonstrated'' designation is a red flag
marking a program so poorly conceived or directionless that
unaccountability seems to have been built into it by design.
Programs rated ``ineffective'' or ``results not demonstrated''
account for $152 billion in budget authority. Imagine what we could do
with $152 billion.
Outside of Washington DC, any business or family with finite
resources sets priorities and creates a budget based on the actual
amount of bang they get for their hard-earned buck. It is only inside
the beltway where that kind of information isn't considered relevant
and in fact, some are trying hard to ban the collection of such
information. But then, it's only in Washington where you never have to
declare bankruptcy and debt is allowed to grow on the backs of future
generations with impunity.
Let me give you one case study. We held a hearing last year on the
Advanced Technology Program. The program was created by Congress in
1988 to subsidize high-risk research and development. The program
cannot demonstrate results. It is corporate welfare. The 2002 PART
reported that the majority of ATP grants go to multimillion dollar
corporations and that the GAO has found that ATP projects are very
similar to private sector R&D undertaken without a government subsidy.
An amendment to eliminate funding for ATP that I offered last year was
voted down in the Senate 68-29. In the end, Congress wasted another $79
million last year for the program. The 2007 Senate budget resolution
promises to fund the program at almost twice that amount.
It would be one thing if we were operating in a surplus. Then we
could have a legitimate debate about whether to keep funding failing
programs hoping they will improve or to give that surplus back to the
taxpayers. But that's not where we are today, with a debt burden of
$25,000 per man, woman and child in America. We simply cannot afford to
keep funding programs that cannot prove their worth.
Nondefense discretionary spending has increased over 45 percent
since 2001. The President has requested a $2.8 trillion budget and that
doesn't include any so called ``emergency'' supplemental spending bills
in our future, nor does it include the late-night pork-barrel frenzy
each time Congress schedules an Appropriations bill vote. Entitlement
spending will tank our economy if we don't do something to get spending
under control.
The question remains, how do we get Congress to act? I would like
to see OMB sell their PART and Terminations List more aggressively:
<bullet> Forcefully sell these reforms and savings to Congress.
<bullet> Fight for these cuts, by taking the terminations list to
the American people with the power of the bully pulpit.
<bullet> The President should veto spending bills that continue to
issue blank checks to failing programs.
There's a bit of hope on the horizon--I was encouraged to see that
the House Appropriations Committee wrote in their 2006 Budget Savings
report that ``the only way to establish accountability in the budget
process is to stop spending on programs that have outlived their
usefulness or could be delivered more effectively at the State or local
level.'' I'll believe it when I see it, but I welcome any help we can
get.
The best place to start is by immediately defunding all programs on
the Terminations List and adopting the other PART reduction
recommendations. Granted, the list only cuts $20.4 billion from a $2.8
trillion budget, but we've got to start somewhere. What's more, we
should suspend the creation of any new program until further notice. We
need ``sunset'' legislation that would phase out every single
government agency, department or program after a certain deadline if
the Congress fails to act or if the program consistently performs
poorly. These are challenging times and we can no longer budget on
cruise control.
I want to thank our witnesses for being here today and for the time
they spent preparing testimony.
Again, I apologize for the lateness of our attendance, and
Senator Carper, you are recognized.
OPENING STATEMENT OF SENATOR CARPER
Senator Carper. Thank you.
Senator Coburn. And I have already explained that you will
probably have to attend the briefing that is ongoing.
Senator Carper. Thanks, Mr. Chairman. To our witnesses,
welcome. It is good to see each of you. We appreciate you
joining us and providing your testimony today. As the Chairman
mentioned, Secretary of State Rice and Secretary Rumsfeld are
briefing us as we speak over in the Capitol, and I want to slip
out in a little bit and hear what they have to say and
hopefully rejoin you before you leave.
Mr. Chairman, thank you for holding this hearing. It is an
important hearing, as we both know. And as we have discussed in
any number of our similar hearings in the past over the last
couple of years, our country is facing a large budget deficit
for as far as the eye can see, and we are just about to embark
on another appropriations season here in Congress, where we
will be called on to make some difficult decisions about what
to do with relatively scarce Federal resources.
At the same time as GAO and other observers have pointed
out again, and again, we are at a crossroads in our history,
where we need to decide what we want our government to do in
the 21st Century. Nearly 5 years after the attacks of September
11, 2001, we have a whole new set of needs, a whole new set of
priorities that must be balanced against some of our older
needs and priorities in scores of popular programs. And with
the challenge of retiring baby boomers, guys like me, our
generation on the horizon, we just cannot afford to do all of
the things that we might want to do.
That is why initiatives like OMB's Program Assessment and
Rating Tool (PART) are interesting and, I think, important. We
should never be afraid of taking a hard look at Federal
programs, my programs, Senator Coburn's programs, whatever, to
determine whether or not they are accomplishing what was
intended for them to accomplish when we first created them. And
in this day and age, we simply cannot afford to allow either
poorly conceived or poorly managed programs to continue without
reform or, frankly, for a program that has run its course and
achieved its goals, to continue draining resources from other,
newer priorities.
That said, we need to be certain that PART or whatever
mechanism we use to make these evaluations is in itself
effective. I think to be effective, a program like PART must be
totally separated from politics and ideology, at least to the
extent we can make that happen. It must be closely coordinated
with existing mechanisms agencies and Congress use to align the
budget with program goals and outcomes such as the older
government Performance and Results Act. And perhaps just as
importantly, we also need to make sure that a program's
intended beneficiaries outside of Washington have a say before
an evaluation is actually completed.
Let me just add in closing, if I could, Mr. Chairman, that
we are not going to close the budget deficit, we know, by
reducing spending on a program here or eliminating a program
there, although every little bit helps. But even if a program
were to eliminate every single one of the programs receiving
failing grades through PART, I still think the savings would
cover just a fraction of our budget deficit, but they would
cover a portion of our budget deficit.
Non-defense discretionary spending, which is the target of
many of the spending reductions and program eliminations in the
President's budget proposals, make up a relatively small
percentage of the Federal budget. I am sure we can find ways to
improve the management of some of the funding in that 16
percent or even to find and eliminate waste and inefficient use
of resources within that 16 percent.
If we truly want to tackle the fiscal problems facing us
right now, however, we, and that is the Congress and I think
the Administration needs to take a look at the entire budgetary
picture. We need to look on both the spending and on the
revenue side, and we need to make some tough choices.
Thank you, Mr. Chairman. Again, we look forward to your
testimony today.
Senator Coburn. Thank you, Senator Carper.
I am going to ask the witnesses to limit their verbal
testimony to 5 minutes. Your complete written statements will
be made a part of the official hearing record, and we will hold
our questions until you have given your testimony.
Let me first introduce Clay Johnson III, Deputy Director
for Management at OMB, and in his capacity, he has provided the
government-wide leadership to the Executive Branch agencies to
improve agency and program performance. Formerly, he served as
Assistant to the President for Presidential Personnel,
responsible for the organization that identifies and recruits
approximately 4,000 senior officials, middle management
personnel, and part-time board and commission members. At OMB,
he oversees PART process.
Eileen Norcross, Senior Research Fellow, Government
Accountability Project, The Mercatus Center at George Mason
University; she joined that center as a research fellow in
January 2003. Her research areas include the U.S. budget, the
use of performance budgeting in the Federal Government, tax and
fiscal policy, and environmental regulation. She is one of the
leading experts on performance-based budgeting, and her
scholarship plays a vital role in the debate on PART and the
importance of measuring outcomes.
Adam Hughes is the Director for Federal Fiscal Policy at
OMB Watch. He oversees Federal budget and tax policy, income
and wealth trends, and government performance issues at OMB
Watch. Senator Carper and myself very much appreciate the work
that OMB Watch has done in their pursuit of transparent and
accountable government and for the support of the Federal
Funding Accountability and Transparency Act that we both
authored. This bill would create an online public database that
itemizes Federal funding so taxpayers can see how their money
is being spent.
I want to welcome you all. I will recognize Mr. Johnson
first.
TESTIMONY OF CLAY JOHNSON III,\1\ DEPUTY DIRECTOR FOR
MANAGEMENT, U.S. OFFICE OF MANAGEMENT AND BUDGET
Mr. Johnson. Mr. Chairman, Senator Carper, thank you very
much.
---------------------------------------------------------------------------
\1\ The prepared statement of Mr. Johnson with attachments appears
in the Appendix on page 25.
---------------------------------------------------------------------------
The title of this hearing is Will Congress Ever Respond to
Program Performance Data? In preparing my response, I rephrased
that to Does Congress Care Whether Programs Work or Not? My
answer is ``I am not sure, but I sure hope so.'' I believe that
taxpayers want Congress to ensure that they, the taxpayers, get
what they pay for. I believe that we all, to widely varying
degrees, however, want Federal programs to do what they are
supposed to do and get better every year.
I believe that money is tight, as you all have pointed out,
and the biggest opportunity we have to add new services and
expand existing services to more citizens is through causing
our existing programs to work better, not spend more money. I
believe that career employees want to be held accountable for
how their programs perform. They tell me this in focus groups.
And I also believe that career employees care about how their
programs perform.
Because of this, I believe it is important to have certain
things. I believe it is important to have clear outcome goals
for each Federal program. We do not have that now. I believe it
is important to have Federal program performance information
that is objective, as objective and reliable as possible. I
believe that we need to have lots of transparency about how
well programs are performing. If we do all of this in the dead
of night, it cannot be used to hold people accountable.
I believe that we need lots of debate about these
performance assessments and how to make them better. As you
said, Mr. Chairman, program assessment is going to be a blunt
instrument, particularly in the early years. And it will only
get better every year, but a blunt instrument is better than no
instrument at all. I also believe it is important to have lots
of discussion about how to help programs work better. We talk a
lot about using the PART to make budget decisions. I believe
the primary use of PART information is to help programs get
better. If we cut programs, we might save $10 billion here or
$15 billion there per year. If we cause 1 percent improvement
in program performance each year, that is $28 billion a year.
Two percent is obviously twice that.
After 5 years of effort, not 5 months, comprehensive
program performance information is still time consuming and
very hard to come by. We have program outcome goals,
performance information, and lots of transparency, which other
countries and several States are working to adopt, and most
good government groups applaud. What we do not have from most
Members of Congress is a lot of constructive debate about these
assessments and how to improve and use them to improve program
performance. We have asked for feedback. We have asked for
engagement by Congress but have not gotten it.
Currently, a majority of Appropriations subcommittees have
no objection to the way agencies use performance information to
justify their budgets. Some of these subcommittees actually use
the PART to justify program funding in their bills. A few
Members of Congress have advanced greater use of performance
information in decisionmaking. Congressmen Platts and Tanner
have proposed separate pieces of legislation, while Senators
like you, Senator Coburn, and Senators Carper, Ensign, and
Allard have spoken out on the subject, and Congressmen Cuellar,
Conaway, and Diaz-Balart have spoken out on it as well.
But these expressions of interest in program performance
are the exceptions. There is a big, a huge opportunity for
Congress to challenge programs to clearly define success and
their plan for achieving it, and then to hold agencies
accountable for doing what they said they were going to do.
That concludes my remarks, and I look forward to any
questions.
Senator Coburn. Ms. Norcross.
TESTIMONY OF EILEEN NORCROSS,\1\ SENIOR RESEARCH FELLOW FOR THE
GOVERNMENT ACCOUNTABILITY PROJECT, THE MERCATUS CENTER AT
GEORGE MASON UNIVERSITY
Ms. Norcross. Thank you, Chairman Coburn, Senator Carper,
for inviting me to testify today on Autopilot Budgeting: Will
Congress Ever Respond to Government Performance Data? Our work
in the Government Accountability Project at the Mercatus Center
at George Mason University focuses closely on performance
information in government, and I note that the views expressed
in my testimony are not an official position of the university.
---------------------------------------------------------------------------
\1\ The prepared statement of Ms. Norcross appears in the Appendix
on page 49.
---------------------------------------------------------------------------
I would like to submit for the record our paper on the
results of the fiscal year 2007 PART for your reference.
Senator Coburn. Without objection, the document will be
included in the record.\2\
---------------------------------------------------------------------------
\2\ The Working Paper in Government Accountability appears in the
Appendix on page 61.
---------------------------------------------------------------------------
A program is a tool to achieve a policy goal. Do economic
development programs lead to prosperous communities? Are
homeland security programs protecting the Nation? Congress
needs to know the answers to these questions in order to make
decisions about how to spend resources. Without performance
information, Congress cannot reliably accomplish its policy
aims. Not knowing its consequences, Congress has created
anywhere from 180 to 342 programs dealing with economic
development in over 24 agencies; 44 job training programs in
nine agencies.
Program duplication on this scale tells us that Congress is
not sure which programs are reaching their goals. It has no way
of comparing programs around common outcomes. Not knowing if a
job training program is employing people means not spending
money on programs that are employing people. Not evaluating
programs on a regular basis prevents the program from
effectively reaching grantees or delivering results;
performance information from its dialogue between agencies, the
Executive Branch, and Congress around jointly defined
objectives.
Congress took the initiative in 1993, when it passed GPRA.
GPRA has encouraged the development of performance measures and
data, but it was not until OMB's Program Assessment Rating Tool
that real progress towards developing measures was made. That
is because the Administration does not just require the
information; it uses it. Congress has identified the need for
performance information. It must now commit to using it.
Otherwise, measuring and gathering data is a paper exercise.
For the past 2 years, the President has issued a major
savings and reforms report detailing his reasons for
terminating or reducing funding for programs. Of the 154
recommended for termination or reduction in funds last year, 54
were PARTed. The document indicates where PART played a role.
Other factors include lack of a Federal role, obsolescence, or
completion of mission.
The Administration uses PART along with other information
and does not limit itself to the evaluations. It does not
automatically reward satisfactory programs or cancel
underperforming ones. By contrast, the House Committee on
Appropriations report ``On Time and Under Budget'' lists 53
programs that were terminated. It only offers explanations for
three of the terminations. We do not know if the remainder were
terminated because they were underperformers or politically
easy choices.
The Administration's report gives a rationale for each
recommendation. The House report only provides a list.
Ultimately, the goal is not to randomly kill programs. Making
judgments about how to fund agency activity should be
constructive, not destructive. Performance information helps
make policy effective. We want to know what works, what does
not, and why.
The only way to give budgetary decisions credibility is to
base them on a reliable evaluation of their performance. Is
PART that system? PART's methodology has been criticized.
Improvements can and should be made. But what is important
about PART is not the ratings; it is the Management 101
questions PART asks of agency activity. Is the program purpose
clear? Is it effectively targeted? Has it demonstrated progress
towards its goals? These questions are the substance of PART.
These are the questions Congress should be asking before
allocating resources.
PART has a few virtues. It has identified and catalogued
agency activity. It is transparent. It holds programs
accountable to the same standards. It measures outcomes. Once
strength often cited as a weakness: PART rates programs on
statutory limitations. Though a source of frustration for
agencies, here, PART provides a service by identifying those
aspects of a program that are barriers to success. The hope is
that Congress review the statute to see if it is preventing the
program from meeting its objectives.
Some limitations of PART: It rates programs against their
own performance. We would like to see PART advanced to compare
like activities. In some cases, scores may not fully reflect
program performance, and there is a potential for different
budget examiners to reach different conclusions. We do not
believe Congress should adopt PART wholesale. We hope Congress
would consider using the kinds of questions PART is asking as
the basis of developing its own method of evaluating agency
activity based on common outcomes.
Indiscriminate cancellation of programs discredits the
budgetary process. We leave program managers confused about why
their programs failed. Programs need to deliver according to
clear expectations and be given a chance to perform. When you
do not meet the expectations, reduction in funding or
termination should be the result. It should not be a surprise.
We believe performance information is best used in
conjunction with other criteria. All of these form the basis
against which Congress should continually scrutinize agency
activity. Efforts to advance what PART has set in motion can
only aid Congress in its work and give the American people
confidence that our Nation's problems are being solved. Thank
you.
Senator Coburn. Mr. Hughes.
TESTIMONY OF ADAM HUGHES,\1\ DIRECTOR OF FEDERAL FISCAL POLICY,
OMB WATCH
Mr. Hughes. Chairman Coburn, thank you for having me here
today and for holding this hearing. As you mentioned, I am the
Federal Fiscal Policy Director at OMB Watch. OMB Watch was
founded in the 1980s and has spent over 20 years advocating for
government accountability, transparency, and access to
government information, and citizen participation in
governmental processes.
---------------------------------------------------------------------------
\1\ The prepared statement of Mr. Hughes with attachments appears
in the Appendix on page 89.
---------------------------------------------------------------------------
OMB Watch believes citizens must take an active role in
holding their government accountable and that the Federal
Government, when supported by sensible fiscal policy, can
develop effective programs and safeguards that meet the
public's needs.
The issue of government performance, as you mentioned
earlier, has taken on added importance during the Bush
Administration, as a combination of factors, some avoidable and
some not, have plunged the Federal Government into debt. Large
and sustained deficits over the past 5 years have made
efficient use of government resources all the more important.
In light of the anticipated budget crunch due to the baby
boomers' retirement over the coming decades, the fiscal
situation in this country will only deteriorate further.
Performance measurement can therefore become a particularly
attractive alternative for those who want to set Federal
priorities based on the current fiscal prospects of a strained
and shrinking revenue base.
OMB Watch has been commenting on government performance
issues for the better part of its existence. We have spent
increased time and resources analyzing the Government
Performance and Results Act and the Program Assessment Rating
Tool over the last 10 years, as government itself has spent
more time focusing on performance and results. We are strongly
supportive of improving the Federal Government's capacity to
meet the public's needs. OMB Watch has worked to protect and
improve that capacity, and we have been open to the possibility
of using performance measurement as one means for achieving
those ends.
We bring a strong belief in the importance and potential of
government itself to the work we do, and because of that
belief, we, perhaps maybe more than anyone else, want
government to be responsible to community needs, spend money
wisely, and accomplish its goals. We are advocates for
government and therefore have a strong motivation to see
government programs succeed.
PART, however, is a very poor mechanism for measuring
program performance and results, introducing biases and skewed
ideological perspective into a model claiming to present
consistent and objective performance data and evaluations of
government programs. Oftentimes, the PART actually decreases
the efficiency and effectiveness of government through
increased administrative burdens, distracted managers, and
compliance costs.
Ironically, we feel the PART mechanism itself does not
produce the right type of results to support and improve
government. We believe PART ratings should not be directly
connected to the budgeting process of Congress because of
significant deficiencies within the mechanisms, namely, the
substantial biases and limitations embedded within the tool and
the additional distortion and manipulation we have observed in
OMB's actual application of the PART.
Based on our studies of the PART and our longstanding
commitment to open, accountable government that is responsive
to the public's needs, I would like to make three points today.
First, we feel the PART continues a troubling trend we have
seen in other recent Executive Branch proposals and even some
Congressional proposals, namely, a trend towards increasing the
power of the White House and the Executive Branch even into
some areas that have been constitutionally designed to be
committed to Congress.
Second, the PART is a limited and distorted tool that
should not be used for either management of programs or for
budget and appropriations decisions. In both the design of the
tool and the process by which the tool is implemented, PART
systematically ignores the reality and the complexity of
Federal programs and judges them based on standards that are
often deeply incompatible with the purposes those very programs
are expected to serve. As one agency contact memorably
explained to us, PART assessments are akin to a baseball coach
walking to the mound to remove his star player and then
chastising him for not kicking enough field goals.
My third point is that there is a better way. Specifically,
Congress already has the means to investigate and produce far
more sophisticated analyses of the usefulness, effectiveness,
and results of government programs in a deliberative way,
including the opportunity for input from a wide array of
stakeholder interests. The openness of the Legislative Branch
allows the Congress to be informed and make better decisions,
but it also serves to balance competing agendas and
perspectives from both inside and outside Congress.
The oversight and evaluation process is one of the primary
if not the primary role for the Legislative Branch. While the
oversight function of Congress may not be as robust as it once
was because of significantly shorter legislative sessions and
delays due to sharply divided political climates, the capacity
to judge the results of government programs already exists
within the existing structures, structures that we feel do not
carry the significant limitations, biases, and negative
consequences of the PART.
In conclusion, we all agree that everyone in government,
the President, agencies and departments, and their staffs, and
especially Congress, needs to be focused on achieving results
in a fair, effective, and balanced way. However, this job
should most of all fall on Congress, which already has the
necessary tools and resources in place to do the most robust
and equitable review of the entire Federal Government.
Relying too heavily on PART ratings will not only gradually
remove Congress from its funding and oversight responsibilities
but will also continue to close the door on opportunities for
outside stakeholder interests, the views of the public, to be
infused into the Congressional budgeting and evaluation
process. The limited perspective of the PART is one of its most
glaring deficiencies. While subjectivity and bias will almost
always creep into any rating system, the PART does not have a
mechanism for balancing out the results of its one-size-fits-
all, Executive Branch-focused perspective.
While the expansion of the Executive Branch powers has been
present in our government since the turn of the last century,
the overreach of those powers into areas historically and
constitutionally given to Congress, the structuring of
programs, appropriating and authorizing of revenues, and
oversight of government is a disturbing trend. Because of this,
PART should not be taken with just a grain of salt or even a
hefty dose of skepticism. Unless the tool design and
implementation systems are significantly modified, the PART
ratings should probably be largely ignored by Congress.
Thank you. I look forward to your questions.
Senator Coburn. Thank you.
I wonder if either of you might want to comment on Mr.
Hughes' testimony. It is certainly different than what we heard
from either Mr. Johnson or Ms. Norcross, and I have several
questions for Mr. Hughes as well, but I thought--Mr. Johnson,
would you like to comment?
Mr. Johnson. Yes; several other countries around the world
think the PART is great; other States in America think the PART
is great. Most good government groups think it is great. It is
an instrument. It has had blunt; will get better every year.
Most people that observe Congress, that have been around
Congress a long time, believe that the Executive Branch is more
interested in how well programs work than Congress is. David
Walker has said that in hearings; so have Dick Armey and
others. I would bet you agree. It is very hard to produce
performance information and program assessments. What the
Administration has done with PART is a place to start. We have
been working 5 years on this. I do not believe Congress is
going to invest 5 years to put together the information that we
have right now. The PART information is a starting point for
building better mechanisms to holding agencies and programs
accountable for what they do.
So I believe, in spite of its flaws, that PART is an
excellent tool. It is a wonderful beginning. It is the product
of 5 years of effort. I do not see this as a power grab by the
Executive Branch. The subject of this hearing is why won't
Congress pay attention to PART, so I don't think Congress is
actually reeling with this onslaught of performance information
from the Executive Branch. Our challenge is to get them to pay
attention to it.
Senator Coburn. Thank you. Ms. Norcross.
Ms. Norcross. What is the alternative to not using
performance information? PART has given us--at least we have
moved the discussion away from the policy preferences of an
administration towards evaluating programmatic activity. So I
do not know what the option would be. Should we revert back to
a system where we simply do not use performance information,
expect it, gather it, or analyze it? And if there is discomfort
with OMB performing these assessments, perhaps Congress should
undertake that.
I understand Congress only engages about 7 percent of its
time in oversight. So the current legislative mechanisms that
are supposed to be engaged in this activity are not working up
to speed. So that would simply be my response is if Congress is
supposed to be evaluating these programs, where is the evidence
that it is, in fact, evaluating them and providing guidance to
agencies along the lines of performance?
Senator Coburn. All right; thank you.
Mr. Hughes, you mentioned that there is significant bias
and distortion and manipulation. Would you give me examples of
bias, please?
Mr. Hughes. Sure. There are a number of different types of
biases that can be involved in this. One is the perspective of
the OMB officer. The budget officer at OMB is the person who
has the final say on what the language will be for the answers
to the questions, how that language that is written will
translate into a yes or a no or a few of the modified answers
that are possible now under the PART and also how those yeses
and noes get translated not only into the numeric raw score but
also into the actual rating.
There are a lot of inconsistencies between the guidelines
that have been laid down for what raw score equals what rating
and what the programs that have been reviewed actually get.
That is one type of bias, and that is from a kind of
implementation perspective. There are other biases in the
actual design of the tool. I think that the format under which
it was designed, which was designed to be accessible to people
who may not be policy experts or who want to just know, like
you say, come and look and see whether the government is
getting results and whether the program is working, that
necessitates that certain things are left out.
One of those things is whether the Congress has designed a
program to have multiple goals. Many programs in the Federal
Government are designed to have multiple goals. That sort of
thing is not taken into consideration within the PART.
Oftentimes, those goals can be conflicting. That does not
necessarily mean that it is a bad design. That just means that
it is a complex program. And that kind of complexity is lost in
the way the tool was designed to apply to people who may not be
policy experts.
Senator Coburn. Are you saying that there could be another
PART program that would take into account for that? What is
wrong by demanding a clear program mission from agencies?
Mr. Hughes. Certainly nothing.
Senator Coburn. And questioning how a program fulfills that
goal; is there anything wrong with those two things?
Mr. Hughes. No.
Senator Coburn. So you do not disagree that a PART program
might be designed better to take out more bias, but you do not
disagree with the fact that knowing what a program's goal is
and measuring performance against that goal, it should be an
effective tool. You would not disagree with that?
Mr. Hughes. No, theoretically, I agree with you.
Senator Coburn. The one problem I had with your testimony
is the problem I have with the rest of Congress is we are lazy.
And the fact is that this is the 37th oversight hearing of this
Subcommittee. Go find another one that has done that. And the
point is that ideally, Congress does have the responsibility,
but they do not live up to it. And so, what we are working with
is in a vacuum, is Congress ideally should be doing this. I do
not disagree with you, but they are not.
And to have a blunt tool that is getting better, even
though it can be criticized, and I think Mr. Johnson would
agree that it is subject to some criticism, as is any
assessment tool when you first start using it. But to say we
should not have them doing it because it is Congress' role--I
agree; that is why I am doing it; that is why we have done 37
of them, begs the question of how do we motivate Congress to do
oversight?
So if we are critical of this one, answer me the question
how I motivate my peers to do the appropriate thing when it
comes to authorizing a program, and in that authorizing, saying
we are going to measure it and then having the incentive to
have Congress do the oversight to see whether or not they have
a goal, and they are meeting that goal.
Mr. Hughes. That is, of course, a very difficult question,
one that I will probably be very insufficient in answering,
giving a satisfactory answer for. I think that the oversight
role of Congress, and you are correct, of course, in citing the
fact that Congress does not really do oversight any more. That
is indicative of larger things about our political system,
about the way that the electoral process works, about the
importance of fundraising. There are multiple things that are
in there that actually have nothing to do with whether Congress
should do oversight or not that are enormous problems that
would be difficult to tackle.
I think your leadership on this issue is important. I think
we need to have more folks in Congress who are paying attention
to these sorts of issues. I do not know if there is a magic
bullet procedural change or a statute or something that we
could do that would make it so that Congress would be forced to
do oversight more. I do think that some of the suggestions that
have been made in front of this Subcommittee in the past about
taking the Program Assessment Rating Tool or a modified version
of it outside of the Office of Management and Budget, perhaps
maybe having the Government Accountability Office do it or
establishing a committee within Congress that would provide
oversight in that regard. I think those ideas are worth
exploring.
I do not think that you can just remove the PART the way it
exists now and give it to GAO and have it work well. I think
there are design flaws that need to be corrected, that need to
be adjusted, and I am, of course, sympathetic to the point that
if you change it too much, the previous reviews would not be as
useful. But it is not necessarily just a problem with the way
that the tool gets done at OMB. We think there are deficiencies
within the way it was designed as well.
Senator Coburn. Well, you would be agreeable, then, to
submit to this Subcommittee the things that you think are
deficient in the design so that we can look at that?
Mr. Hughes. Sure, and that was reflected in my written
testimony. There is a section on that.
Senator Coburn. One of the problems with oversight is that
a lot of agencies do not respond to our questions. Let us say
we had oversight, and they do not respond. The only way you can
solve that is either have somebody who can squeeze them on
their money, or we have to squeeze them until they respond. But
that requires the sausage-making process to be able to
accomplish that.
The thing that is disconcerting is I have little faith that
Congress is going to step up to the bar until they are
absolutely forced to through a financial disaster to make the
hard choices. Congress wants to avoid hard choices, and as long
as they do not feel the pinch, they will not make the hard
choices. And that is why 2016 is going to be a very tough year
for this country, because that is when the pinch starts, the
big pinch. And so, having an assessment tool, blunt, maybe
somewhat biased, maybe somewhat distorted is, in my mind,
better than nothing at all.
Mr. Johnson, and you may not care to comment on this, but
you might comment on the motivation behind it: The House
Subcommittee on Labor/HHS put a prohibition in their bill this
past week that precludes any money from being spent on the PART
assessment. Any comments on the motivation behind that or what
you see? I am not trying to create a problem for you with the
Subcommittee, but how did we get there?
Mr. Johnson. Well, there is one unelected staff member who
is opposed to the PART. He worked on the Treasury/
Transportation bill last year and put a similar prohibition in
there. He was at HUD before that, and he disagreed with HUD's
use of the PART, and he was at OMB before that. One unelected
staff member is responsible for the provision. The chairman of
the committee had no knowledge that it was in the bill. It is
inexplicable to me that language like that is in the bill. That
is my only comment.
Senator Coburn. OK; one of the other things, Mr. Hughes,
with your testimony which I find, well, less than congruent is
the statement that the PART increases the White House's power.
And the problem with that is Congress ignores the PART
assessment. We have been able to do nothing with the PART
assessment. Even when I look at all of it, and I look at the
agencies, and I have done the oversight, and I try to get
somebody to do something about it, Congress ignores it.
So there is not a power grab there, because Congress is not
paying any attention to it. So explain to me your reasoning
behind--is it a potential? Because it is certainly not, in
fact, acted out. There is no effect of the PART right now on
the Congress, because they ignore it.
Mr. Hughes. I actually would agree with you, and I would
say that would probably be a poor choice of words on my part. I
do think it is a potential problem. Let us do a for instance.
Let us suppose that Congress will appropriate funds according
to whatever the rating on the PART is. Why do we even need
Congress? Let's just let OMB do it. So I think it is a slippery
slope. I think that particularly with respect to budgeting, we
have been working more in trying to explore the management side
of it as well, of PART, and the usefulness within agencies.
I think there is more potential for a productive use of the
information there. I do not think that you can look at a PART
score and say, OK, well, I know how to fund programs now
because of these problems. So the way I chose my words is
probably poor. I do not think it is a problem right now; as you
say, you are correct, that Congress does not pay attention to
them.
Senator Coburn. Well, but let me create a scenario for you.
Let us say that Congress is doing great oversight on
everything. We are sunsetting things; we are reauthorizing
them; we are bringing them back up; we really know what we are
doing and that we are doing a good job of that. Let's make that
assumption. That is an absolute lie, but let's make that
assumption.
Would you deny the fact that the Administration should have
a performance tool themselves to measure what the goal is of
the program and whether or not they are meeting that goal as a
management tool to become more effective in carrying out the
will of the Congress?
Mr. Hughes. No; I think the problem exists when the tool
that the Administration designs, or it does not even have to be
this one, the Executive Branch designs portrays itself as an
unbiased, objective evaluation of how programs and management
are going at agencies when, in fact, it is anything but that.
So I do not think that--again, in theory, that this is
necessarily a problem. But with this particular instance, it is
kind of like a wolf in sheep's clothing. You have a situation
where they are saying we are doing this; it is systematic; it
is transparent; it is on the Web; the public can view it; this
is an innate good.
But the kind of things that we worry about are the things
that are not transparent within the PART, that you do not
necessarily see up front when you look at the one-page review.
That is where you get into a tricky situation, and it is
perfectly fine for the Executive Branch to have their own
systems and whatever they like, but the problem occurs when
they try to sell that to Congress as the one objective
evaluator.
Senator Coburn. But they have not. They have just said
since you are not doing one, we are going to do one, and here
is what we have found, and here is what our recommendation is.
We still control the purse strings, and it is obvious from the
PART assessment that Congress has totally ignored the
Administration when it comes to evaluating programs. So that is
not seen as a risk to me whatsoever.
Mr. Hughes. Well, that is encouraging to hear.
Senator Coburn. Well, they have not.
Mr. Hughes. Well, I would say that they have not succeeded.
Senator Coburn. I think it is very discouraging to hear,
because they are not looking at the other as well.
Mr. Hughes. Fair enough.
Senator Coburn. They are paying attention to nothing and
continue it. One of the battles I have, and I will share it in
the Subcommittee, is there are a lot of bills that I block;
they are authorizing bills. And I go to Members of the Senate,
and I say these are the things that I have problems with. And
they say, well, why do you have problems? And I say, well, you
have not looked at the programs that are already there before
you authorize another program, and you have not said we are
going to eliminate this program and put this one in. You are
authorizing another program to do the same thing that is
already happening without deauthorizing another program.
And what I get told: Well, we do not do things that way.
Well, the American people do things that way. Business does
things that way. States do things that way. Why should Congress
not do it? So, really, we are shooting the messenger here. The
messenger--there is a vacuum in terms of oversight, and we now
have an Administration that has attempted, whether we think
their tool is good or not. And you do not doubt that the tool
is getting better as they have used it? They are using a tool
that is improving, that does have maybe some bias and does have
some risk for manipulation in it, but the fact is it is the
only thing available right now, especially since this
Subcommittee has time getting even agencies to come and testify
before it or to give us information.
Mr. Hughes. I will respond with two points. One, your
shooting the messenger analogy, I think that may be part of our
criticism of it, but our problem with it is that when the
messenger leaves with his message, and when he gets to his
destination, he is carrying two different messages. There is a
problem with the transmission along the way, and that is
something that is important to realize, regardless of where the
criticisms are being pointed at.
I think the second thing is, and I sympathize with your
frustrations about oversight in Congress, and that is certainly
something that we would like to see a ton more of. I think you
can kind of get around some of the rhetoric around what
government--we have all these programs, and they do not do
anything that is important. If we had more oversight, if we had
more openness about what the government actually does, I think
people will actually have a greater appreciation of things.
Senator Coburn. Right.
Mr. Hughes. So I think our criticism--try to be focused on
this particular instance of PART, the way that this PART
assessment works. I do not think that it should be thrown in
the garbage can. I think that it is very important that people
in Congress and people in the agencies and the public know that
this should be, despite the fact that there is not a lot going
on elsewhere, this should be a really tiny part about
evaluating how government works. That would be my caveat
about--I am sympathetic to the fact that it is not going on
elsewhere, but try not to latch on to it and say this is the
tool, and this is what is going to get us there.
Senator Coburn. Nobody has in Congress. Would all three of
you agree that some type of assessment of goals and measurement
against the goals changes expectations of program managers?
Mr. Johnson. I agree totally.
Senator Coburn. Ms. Norcross.
Ms. Norcross. Totally correct.
Senator Coburn. Mr. Hughes.
Mr. Hughes. In my limited experience, I would say that is
right.
Senator Coburn. Ms. Norcross, you have some experience with
performance tools in New Zealand, and I also know that South
Korea has adopted assessment programs. Could you comment on
those two things?
Ms. Norcross. Morris McTeague, with whom I work at the
Government Accountability Program, has direct experience with
the New Zealand experience in developing performance
information systems and applying them to remedy some of New
Zealand's budget crises. And if I could answer that question
later, I could get you more information in specific on some of
the reforms that they have undertaken. We are right now doing
an analysis of that.
Senator Coburn. OK.
Ms. Norcross. So I could provide that for you.
Senator Coburn. Mr. Johnson, the question of bias in the
instrument that you use, give us an example of three or four of
the questions that PART asks about programs.
Mr. Johnson. Well, it asks if the program has a clear
definition of--this is not exact wording, but it asks about do
you have a clear definition of success? Does it have a good way
of measuring your performance relative to that? Is it meeting
its performance goals? It asks about the quality of management
the program has. Do the program have an efficiency goal? Is it
Management 101, or it is Accountability 101?
These assessments are put together by the agency and OMB,
not by OMB alone. The agency and OMB are supposed to agree on
the program performance goals. Just as agencies are afraid to
disagree with Congress, agencies are sometimes afraid to
disagree with OMB about its assessment, But if they really
disagree with the assessment, agencies can submit their
disagreements to an appeals board that I chair and that is made
up of deputy secretaries from four or five agencies. We get a
number of appeals every year, and we review them. Some of them,
we approve, and some of them, we reject.
And we also conduct what we call consistency checks, where
we review if the PART follows the rules we have for answering
the questions. We also review whether the answers in a PART are
consistent with each other.
As we look also at programs dealing with the same subject
across agencies, we pull all the relevant program assessments
together when we start doing a cross-cut analyses to make sure
we are equally attentive to the issues, equally focused on the
quality of the performance measures and so forth, because we
are going to be using this information to compare one program
to another. We use cross-cutting analyses to see if there is
something an ineffective program can learn from an effective
program dealing with the same topic.
So there is a lot of effort to make the assessments
consistent; to make the information reliable; and to remove
bias. There is bias in anything a human being does. So I have
no doubt that these are not perfect instruments, but they will
get better over time, and the assessments that we have done in
the last 2 years are better than the assessments we did in the
first 2 years.
Senator Coburn. So, in other words, the programs set their
own outcome measurements.
Mr. Johnson. The program does. OMB and the program staff
have to agree that the performance measures are acceptable.
Senator Coburn. And then, they measure themselves against
it.
Mr. Johnson. They then determine the metrics they will use
to measure performance and how to collect the data, and how
often to collect that data.
Senator Coburn. And so, where is the bias in that?
Mr. Johnson. I do not know; just because----
Senator Coburn. If they are participating in setting the
goal, and they are participating in setting the metrics, and
they are the ones doing the measurement of the metrics, where
is the bias?
Mr. Johnson. I do not know where there is unusual bias. I
know that there is a bias in anything that human beings are
involved in. So I do not know what specifically Mr. Hughes is
talking about.
Senator Coburn. In teaching to the test, a problem across
agencies as they respond to PART questions, Mr. Hughes wrote in
his testimony that agency officials told him they gamed the
system to avoid negative scores and consequences. Do you think
that is true? Is there something in the program to help
alleviate? We know everybody games when they are being measured
to an extent. Are there things in the PART assessment system
that take that into account?
Mr. Johnson. I know that agencies like to be green. They
really like to be green, and they really like to have good PART
scores. And so, they do a lot of things to please OMB and to
get good scores and to look good on that scorecard.
Senator Coburn. Does that carry out into changed programs
and changed management to make the programs more effective to
deliver better process and therefore better response by the
government to the very people they are supposed to be helping?
Mr. Johnson. Well, teaching to the test, gaming the PART to
get a good score and providing only superficial analysis does
not help the program work better. One approach that we have to
improve the quality of program performance is shine a real big
light on it all, which is one of the primary reasons we took
all this assessment information, summarized it, and put it on
the Website ExpectMore.Gov for all the world to see and for
people to look up and say that is not the way I know the
program works. An employee can look at it, or someone served by
the program can look at it and say, well, that program does not
work very well; it is ineffective as far as I am concerned, and
they can complain to the agency or complain to OMB or complain
to their Senator or Congressman.
Shining a lot of light on how the program is assessed, on
what performance measures are used, and on what the performance
information says can drive improvements in the measures that
are used, the data that are used, and the quality assessment.
So that is why I believe it is so important to have No. 1 on
your list posted on this sign, which is transparency. You can
have all of this information, but unless we shine a really big
light on it, it will not get better over time. That is why we
took it with all of its warts, with all of its dimples, put it
out there. Now, let us begin the process with agencies and with
Congress, I would hope, to improve this program performance
information, to make these assessments better and to make our
plans to help program perform better more aggressive.
Senator Coburn. Let me follow up on that for a minute. So
Mr. Hughes can go to every PART assessment and via the
government Website can look at the goals, the metrics, the
measurement of the metrics, the response, and the rating.
Mr. Johnson. Right.
Senator Coburn. In other words, nothing is hidden.
Everything that comes to develop that, that can be accessed by
OMB Watch, so they can see all that.
Mr. Johnson. Right.
Senator Coburn. Right; OK.
Mr. Johnson. There is a one-page summary of every PART on
the ExpectMore.gov. There are links, at the bottom to the
detailed PART, which is multiple pages. It is written in OMB-
speak and has historical information and more detailed
information. That is the meat of the assessment. The summary
and all the details is available on ExpectMore.gov.
Senator Coburn. But they can get access----
Mr. Johnson. Yes, sir.
Senator Coburn [continuing]. If they need to as well.
And so, given your emphasis on transparency, are we to
assume that you are going to be very accepting of our OMB
transparency bill that Senator Carper and I have.
Mr. Johnson. That is on the contracting information.
Senator Coburn. Online grants, contracting, everything.
Mr. Johnson. We love transparency, and we are working very
effectively with your staff to figure out----
Senator Coburn. I understand that.
Mr. Johnson [continuing]. The best way to get there as soon
as possible. We are big on transparency and shining the light
of day on performance to a strengthen accountability.
Senator Coburn. Any other comments from any of our
panelists?
Mr. Hughes. If I could just respond to some of the stuff
that we have been talking about, the bias in the data and how
the data, which data is important and which data is not
important, there is a tension between outcomes and outputs in
any type of performance management initiative. The PART focuses
on outcomes, which is certainly a good goal. We think it is
more of a broad government-wide goal, maybe something that
should be included in something like the GPRA, the Government
Performance and Results Act.
A lot of times, you cannot judge the effectiveness of
programs based purely on outcomes, and I will give you a couple
of examples. One program that is run out of, I believe it is
the National Park Service, is an office that works as a
consultant with local communities to transform the neglected or
unused areas into public space: Parks, playgrounds, those sorts
of things. They have collected, long before PART came along.
And another thing that you should know is that agencies have
collected performance data before PART came along. It is not
like they were not doing this to begin with, and then, all of a
sudden, OMB says you have to do it.
Senator Coburn. Some were not.
Mr. Hughes. Some were not; that is correct; not all of
them.
They had a couple of standards by which they judged whether
they are doing a good job in this program that acts as a
consultant. One is through surveys with the local communities
that they consult with: Were you satisfied? Did people use the
parks? Did you like the services we provided? Another way is
they used to collect data about based on the amount of money
that they were given, how many square acres of parks did they
create? How many miles of jogging trails, those sorts of
things.
Those are outputs, the second part. The survey part could
be both. OMB, in the PART process, wanted them to focus on
outcomes. And one of the things that they said should be an
outcome was, Are the people living in the community healthier?
And I think that is a perfectly good goal. I think people
should be healthier. But the program in the National Park
Service has no way to force people to go and jog in the park.
All they can do is say this is the money we got to create parks
for communities. These are the parks we created. These are the
people we worked with and what they thought of what we did.
That is one instance, one example of multiple examples of
the difference between outcomes and outputs and how certain
programs are not necessarily structured to focus completely on
outcomes, or maybe the outcomes are beyond their control. And
one other example I will share about bias within the PART is
the Appalachian Regional Commission. This is a program that
Congress decided to be a patchwork, to cover the holes between
other programs that were working in similar issue areas.
Senator Coburn. I have been trying to get rid of it for 10
years.
Mr. Hughes. I am aware of that. [Laughter.]
And I do not have a personal perspective on the Regional
Commission myself, but Congress designed this program to fill
in the blanks, in the holes between programs, and the PART
assessment said that this is not a unique program, which of
course, it is not, because Congress designed it not to be
unique. It was designed to be duplicative, because the evidence
that Congress had seen at the time said that there are things
that are being missed.
And we can talk more about, maybe we should have just
pulled all of the programs together and redesigned them so that
the holes are not missed, and that is certainly something that
OMB is trying to do.
Senator Coburn. The whole point is you raise the question
about what Congress has not done so that they will do it
better. And to say that it is a blunt--it is a blunt tool, but
it raises it up to a level so that somebody has to now--let's
address this, and we have not addressed the Appalachian
Regional Commission. What we have done is we have let it
continue to do exactly what it does, and the danger with that
is: One, we are not efficient; two, we could design a program
that helps a whole lot more people with the same dollars; or
three is we could help the same amount of people with a whole
lot less dollars, which gives us dollars to help somebody else
somewhere.
So the point is that is a commission that I am very well
informed on, and I believe even the blunt tool will show that
we could be much wiser as Congress to make the goals of that
program more effective. I believe outcomes is the measure. I
believe the American people want outcomes as the measure. But
part of it is laziness on our part. When we write a program,
and this is something I am critical of Congress, we ought to be
very specific about what our intentions are, and we are not. We
ought to be very specific about what we expect, and we are not.
We ought to be very specific on how we want to measure whether
we got what we expected, and we are not.
So a lot of the problems do not have anything to do with
you all in front of us; they have to do with Congress not being
good legislators so that we design a system that can be looked
at later and say did we go after what we intended to go after?
Did we accomplish what we intended? And did we do it in a way
or within the cost parameters that we thought it would?
And so, the real criticism is not at OMB. They are dealing
with what we have dealt them. The real criticism is for us in
not being specific enough in terms of--and you can ask staff:
When I write a bill, I want it all the way down to the T. I
want limited discretion, because if we are going to write a
bill and do not know enough about it, we should not be writing
the bill until we get the information to write a bill
correctly. And I do not know many people who would disagree
with that. It is just easier to write it loose and let somebody
else worry with the details, and that is called lazy
legislating.
Mr. Hughes. That has been our experience working with you
on the transparency bill as well. I think, though, that it is
not necessarily as easy as you might make it out to seem. There
is another example: The Consumer Product Safety Commission was
ruled down on the PART review for not using cost-benefit
analysis in its regulatory rulemaking. It is actually
prohibited by Supreme Court decision from using cost-benefit
changes like that in their rulemaking.
That is not something that the program can control. And I
agree with you that it is good that even with the Appalachian
Regional Commission example that these things are brought up to
Congress. But as you have said many times, with the lack of the
kind of investigatory role that Congress is playing now into
how programs are made, we are concerned that the information
that we all admit has some biases and those sorts of things
will be taken as a snapshot, and the investigation will not be
done to get underneath what the rating is.
So the ineffective for the Consumer Product Safety
Commission, it should not be said we should get rid of it. But
it is the bias in the tool that gives you the ineffective.
Senator Coburn. But experience tells us that is not
happening, because Congress is not paying attention to PART,
and they are not paying attention to their own. They are
ignoring them both. So your fear is unfounded, because we are
not using it.
Mr. Hughes. Well, we would like to be vigilant.
Senator Coburn. We should do both, though. We should be
using theirs plus our own, and that is the point. Outcomes, to
me, is the measurement, not outputs. And outcomes, if we design
something to have an outcome, then, we ought to know what that
outcome measurement is, and then, we ought to hold agencies
accountable to be to that outcome. And I will just give you a
great example: How about the incidence of HIV reduction in this
country, which has not happened, and then, we spend money on
flirting classes? And there is no connection between the two.
In other words, if somebody is going to measure outcome, we
ought to be asking why, with all of the money we are spending
on HIV that we are not seeing a reduction in the incidence of
new HIV cases in this country.
And yet, nobody is measuring the performance against that
outcome, and that is an outcome that makes a difference in
lives. It is not outputs; yes, we are spending a lot of money,
but we are not measuring outcomes, and therefore, we are not
getting the ability to make the programmatic changes that need
to be made on the congressional side to accomplish that.
If the court prohibits a program from operating well, OK,
if it prohibits a program from operating well, that tells us we
have a problem in the design of the program.
Mr. Hughes. Well, I would disagree with that
classification. I do not think it is prohibiting it from
operating well. I think it is saying that the program needs to
take certain considerations into account when it does operate.
It needs to say there are certain things, equity issues within
programs in the Federal Government that are important to take a
look at. It is not necessarily that the Supreme Court is
putting up a roadblock in front of them getting the job done.
The Supreme Court is making a value judgment about how the
program should operate.
Senator Coburn. Which is not the Supreme Court's job. The
Supreme Court's job is to interpret the laws and the
Constitution and the treaties, not to tell Congress how to run
the budget of the country, and that is----
Mr. Hughes. And I would also say to you, too, that it is
not OMB's job to tell you how to run the budget of the country
either.
Senator Coburn. No, we agree.
Mr. Hughes. Yes.
Senator Coburn. You will not disagree with me on that at
all. I believe we have abdicated our responsibility, and the
reason OMB is having to do this is because we have not. But I
have no heartburn with somebody doing it somewhere. At least we
have some information with which to make a decision.
I want to thank each of you for being here. I would like a
little more formal response from your organization on specifics
on how you would definitely change an assessment tool program
for agencies and what we might be able to accomplish that would
limit, and I want that as justifiable constructive criticism so
that when we look at PART, we can have your thoughts in detail
on how we can assess that and maybe make recommendations.
Mr. Hughes. We look forward to that opportunity.
Senator Coburn. All right.
Thank you all so much for being here. The hearing is
adjourned.
[Whereupon, at 3:43 p.m., the Subcommittee adjourned.]
A P P E N D I X
----------
[GRAPHICS NOT AVAILABLE IN TIFF FORMAT]
<all>