|
Giving Hope and Support to America's Children |
Child Support Enforcement
Management Information Systems
Lessons Learned by State Officials
Office of Inspector General
Office of Evaluation and Inspections
December 1996
OEI-04-96-00011
TABLE OF CONTENTS
PAGE
INTRODUCTION
Child Support Data Systems
State Survey
LESSONS LEARNED
Working with State Government
Working with Federal Policies, Procedures, and Staff
Working with Private Contractors
INTRODUCTION
CHILD SUPPORT DATA SYSTEMS
In November 1996, the Office of Inspector General (OIG) released a draft
report on implementation of State child support certified data systems
(OEI-04-96-00010). That report describes experiences of State IV-D child
support agencies in implementing the automated data systems for child support
required by the 1988 Family Support Act. The Act required States to implement
certified automated data systems by October 1, 1996. However, only one State
met the deadline.
Delays resulted from problems with various program elements, such as ACF's
requirements that States share their technology, short timeframe for developing
and implementing the systems, and ineffective State and contractor working
relationships. Each of the major players in implementing the automated data
systems (Federal and State agencies, and private contractors) shared
responsibility for the delays.
Importantly, however, State child support enforcement agency staffs learned
what they believe were valuable lessons for implementing future required
automated data systems to support State welfare systems. This paper provides
a listing of the lessons learned by State program offices. We are providing
it to the Office of Child Support Enforcement for their use in planning and
implementing future required automated data systems.
STATE SURVEY
This paper is a spin-off of the work we did for our formal report on State
child support certified data systems. For that report, we used a standardized
questionnaire to survey all 50 States, the District of Columbia, Puerto Rico,
Guam, and the Virgin Islands. As part of that effort we obtained information
on State experiences and "lessons learned" in developing and implementing
automated data systems for child support enforcement. However, we did not
verify, validate, or interpret the significance of the lessons learned. Hence,
we did not include that information in our report.
During our exit conference, however, some members of OCSE suggested that
the opinions expressed by State program officers would be valuable in planning
and implementing future automated systems. We are happy to provide this raw
data for their use. We compiled from our survey instruments all the suggestions
made by State child support enforcement officials, and present their advice
exactly the way we received it. The "lessons" generally fall into three
categories--working with (1) State Government, (2) Federal Government, and
(3) State contractors.
LESSONS LEARNED
WORKING WITH STATE GOVERNMENT
-
Take the time necessary to find the right solution versus one that satisfies
the "Feds", but isn't useful to the front line worker.
-
Develop an effective structure for project management and problem solving.
-
Maintain total project oversight.
-
Obtain sign-offs on all products from top management to project leader.
-
Keep your scope of work in check. Remember that requirements are more important
than extras. The more complex the system, the harder they are to implement.
Keep it simple and enhance later.
-
Be prepared to test deliverables before acceptance.
-
Take adequate time to review and test deliverables.
-
Determine expectations for deliverables up front.
-
Give specific deliverables to vendor and insist on delivery. Monitor vendors
(development) workplan as well as their progress.
-
Establish measurable criteria for quality deliverables. Refuse to accept
any deliverable that does not meet criteria.
-
Review work with contractors on day-to-day basis.
-
Be prepared to properly manage and monitor contractor.
-
Hire a project coordinator, project monitor, and quality assurance contractor
at earliest opportunity.
-
Include more structured contract incentives and penalties, rather than sanctions.
-
Try to keep the same project team members throughout the project.
-
Start on conversion tasks as soon as possible.
-
Do not generate letters to employers and postmasters from converted data.
Wait until locate interfaces have run.
-
Manually convert database. Do not attempt an automated database conversion.
-
Data conversion is of utmost importance. Careful attention and planning must
be given to this activity to ensure proper, accurate, and timely
conversion--whether it be manual or automated conversion.
-
Monitor implementation contractor closely.
-
Formulate contingency plans.
-
Require greater involvement with project staff so they may better understand
the system impact with their performance on daily tasks. More sessions were
needed in bringing administration and regional caseworkers together to understand
what was going to direct their work.
-
Make it a priority to reconcile the accounting processes on a daily basis,
then weekly, monthly, etc.
-
Plan for a mass reconciliation with participants, cases, and program types
in the IV-A system. Problems with these interfaces mean adjustments to accounts
which frustrate caseworkers and is very time consuming. Grant details need
to be properly linked so rebates can be issued timely.
-
Phase in ticklers/alerts when the locate interfaces run so workers are not
overwhelmed with responses and stagger the participants submitted to the
interfaces.
-
Get use to programmers changing jobs frequently. We felt and still feel our
project serves as a training ground for the contractor. The contractor then
includes these trained and now experienced programmers on bids for similar
projects. Turnover is high.
-
Assure that the management control process is clearly understood before
initiating project.
-
Minimize collateral changes to the organizational and legal environment during
development.
-
Recognize at the beginning of any project that the line staff are the pros
of the organization and they must be involved in its design, development,
and implementation.
-
Provide adequate "program side" State staff involvement in the review of
the external and internal system designs.
-
Have good detailed requirements and specifications.
-
Follow good system development guidelines.
-
Make evaluation of project a separate contract.
-
Plan for follow-up training.
-
Develop a strong contract with specifics (e.g., penalties for missed milestones,
progress payments).
-
Establish constant interaction between functional and technical staff.
-
Plan for operational changes that result from automation.
-
Develop and implement in discrete manageable modules. This significantly
reduces risk, gets the product to the user on a more timely basis and reduces
the impact on staff.
-
Assure system is kept technologically and programmatically up-to-date. This
provides a concrete foundation for systems development efforts.
-
Build a unified team.
-
Spend more time on the design than you think you can afford.
-
Pay a great deal of attention to database design.
-
Design the database first.
-
Use as much new technology as possible.
-
Cut out as much decision by committee as possible. We worked a lot of overtime.
-
Get started earlier on project.
-
Allow some flexibility in contracts so amendments can be made later.
-
Develop and utilize a model system that allows user participation in every
key area.
-
Take as much time as you need for design document approval.
-
Require proof of testing from contractors at all levels of development before
acceptance begins.
-
Keep focus of administrators on the deadline.
-
Gain support from the IV-D Director's office and the agency of the project
office. The user buy-in that was obtained through the early involvement of
the IV-D staff in the design phase continued through monthly meetings of
a User Acceptance Committee that reviewed the design of each module at various
steps in the development process. A User Acceptance Sub-Committee reviewed
the system test results of each module as it was completed, and made invaluable
comments and suggestions along the way. This enabled the project office to
deliver the customer driven system that the project's mission statement called
for.
-
Obtain up-front commitment from all levels involved in project.
-
Identify a team/group of staff that is dedicated to the system development
and accountable/responsible to one project manager.
-
Maintain project management of each task, from system requirements through
implementation.
-
Have adequate staff dedicated to project work only.
-
Acquire the necessary State and contractor staff with appropriate expertise
in large scale data system development.
-
Establish staffing availability and priority if project goes over deadline.
-
Provide adequate staff to clean up data converted to the new system.
-
Identify the total number of full-time people needed to complete the project.
Assure that the total number of full-time people is always maintained. If
replacements need to be made, do so within a short timeframe.
-
Develop a data unit that will be a part of child support.
-
Provide adequate State training staff for initial and ongoing training of
program staff.
-
Be sure administrative and project staff are committed to a successful project.
-
Develop a team approach during development and implementation phases of project.
-
Do not compartmentalize development teams.
-
Provide a forum for regular/ongoing communication between all players involved
in project.
-
Have more frequent questionnaires like "Implementation of State Child Support
Certified Data Systems" to promote creative analysis of project status and
factors which could enhance progress.
-
Set realistic project schedules and budgets.
-
Develop good project workplans.
-
Set deadline and deliverables at a "phase" level. Any project phase that
exceeds one year will normally fail. Reviews should be done at a "phase"
level. a. Approve planning b. Approve design c. Approve functionality d.
Approve implementation e. Approve post implementation
-
Develop positive, proactive media and PR contacts.
-
Involve county (front-line and administrators) in analysis and design.
-
Meet periodically with IV-D Director.
-
Obtain commitment to automation.
-
Obtain commitment of State leadership and senior management.
-
Maintain regular communication with and support from State legislature.
-
Co-locate State with contractor. Contractor staff worked almost full-time
on this project. (We let bidders know up front that we expected staff to
be in on-site, not flying in and out as needed).
-
Try to keep staff turnover at a minimal.
-
Be sure you have dedicated staff who do "whatever it takes" to get the job
done.
-
Obtain available Data Processing Resources (i.e., access to Data Center equipment
and personnel).
-
Make sure the RFP requires that current functionality continue to be provided
by the transfer system, thereby, ensuring the current level of automation
is met With such a requirement in our RFP, our State would have had only
the features/functions of the transfer system, which were not always as automated
as the current system.
-
Have all regulations/guidance in place before the clock starts ticking.
-
Management support is critical to the success of the project.
-
State technical data processing staff must review and approve all technical
deliverables and products produced by vendor.
-
Large scale system projects such as certified data systems require one single
responsible party leading an independent project team. This responsible party
should maintain budget control and have full support of Department leadership.
-
Project Director should have full authority to control scope of project by
freezing user agency requirements.
-
Each State has a unique environment, IV-D structure and needs. Although a
core of common IV-D denominators and regulations apply, unique factors and
business practices must be recognized. States must recognize similar differences
and needs at the local level.
-
Regarding CSENet -- plan to do a mass reconciliation of case numbers when
State goes full service with another State. Some of the biggest problems
experienced were with cases in common--those cases where communications have
existed for years using paper. Now a State will begin sending communications
using CSENet where there is no initial CSENet electronic referral. This seems
to be causing a mis-match and transactions error out. This requires a person
to research the electronic transaction much in the same way a person would
research a miscellaneous letter without enough information for easy
identification.
-
The "Design Freeze" is the key to why we were able to met the deadline.
-
The project team allowed the system to come up in phases and never expected
perfection. We wanted functions to work with reasonable consistency. When
testers evaluated a screen or function, an evaluation was made to move or
not move it to production. We determined if the associated problems were
of sufficient magnitude to stop implementation or if there were workarounds
we could live with it. By striving for a basis working system and delaying
some of the "nice-to-have" items from the contract, we were able to get the
work out and also meet the goal of having a certified system. The first phase
brought up those screens and functions needed to support basic child support
activities and distribute collections. The second phase brought up screens
and functions required for certification.
-
Contractual roles and responsibilities must include specific authority described
for Project Executive and contractor. Corporate sign-off must include President
of Board, etc.
-
A fixed price contract must be adhered to and the contractor should provide
written statement of understanding.
-
Front-line users should be the critical staff to ensure a relevant system
is in place.
-
States must be very specific in describing the level of detail in the RFP
to avoid contract disputes.
-
State staff must be involved in all development processes. You cannot let
the vendor take responsibility for this process.
-
Management users must take an active involvement in any major data processing
project.
-
State administration needs to involve system development staff early on in
the planning process to be able to provide a "big picture" of what is involved,
which impacts the entire planning/development process.
-
The more direct user involvement, the better the design.
-
The user's role cannot be minimized. Informing them, involving them in the
design, testing and implementation of system, and gaining their support and
buy-in as early as possible is critical.
-
States should insist on having the vendor on-site during the development
phase.
-
States should clearly delineate decision-making authority and keep it limited
to a few persons.
-
The consistent use of adequate schedule software from project beginning to
end provides a basis for monitoring the progress of the project.
-
Advanced planning for operational impacts/change is critical.
-
When the contractor has the need to say "trust me", don't because there is
some problem that makes them have to say this.
-
The Project Manager for the State needs experience in large projects.
-
IQA staff was invaluable. State staff do not have sufficient experience with
today's technology.
-
It is better to "grow" these systems in pieces rather than to have huge
comprehensive projects.
-
The child support program is very complex. In this light, there needs to
be a Steering Committee of State and contractor experts who make decisions
about how the system should perform specific functions. Both the State and
contractor need to recognize the decisions made by this Committee and accept
the resulting design.
-
External, private sector consultation should be sought when developing RFP.
-
Agency policy, even when that policy is current, is not written with the
specificity needed for system development.
-
Before you finish designing and implementing a project, it will change because
either Federal or State law is changed.
-
It is very difficult to transfer child support functional knowledge to State
contractors.
-
The people made this project successfully.
-
Weekly meetings with project staff. Weekly meetings involved: Contractor
staff (Project Director, Lead Programmer), State Staff (Project Manager,
Head of Program, Policy and Systems), State IRM Staff (Data Center Manager
of Application, Lead State CS Technical Staff). Weekly meetings held to review
work accomplished, work planned for the week and to discuss problems. This
was very helpful. Within a few months of the November 11, 1993 start date,
everyone was convinced that we could do this (achieve certification under
FSA-88 requirements).
-
Getting everyone in one room is key. It makes getting acceptance easier and
facilitates assignment of responsibility and resolution of programs. We also
had support from the Governor, Cabinet Secretary, Division Director, Legislature,
Budget Office, and Office of Information Systems.
-
User buy-in and commitment at all levels is essential.
-
Although people say they want to do things more efficiently through automation,
if it means their job will change, don't believe them.
-
It is most advantageous to co-locate State and contractor personnel assigned
to a functional area. This arrangement expedited resolution of design issues.
-
Highly trained full-time data processing staff, who are actually employees
of the Child Support Program.
-
The number of manhours a State anticipates for testing will always require
more time due to staff turnover and other IV-D program priorities.
-
You can never have too much training. Training should not just be concentrated
at the front end, but should continue on a regular ongoing basis.
-
The contractor should provide on-site technical training to State staff.
-
If States are to automate, they should look at issues after automation that
will be major factors in the budgeting process (i.e., maintenance agreements
and support).
-
State staff should be trained in project management.
-
System design takes more time and more money than you ever anticipated.
WORKING WITH FEDERAL POLICIES, PROCEDURES, AND STAFF
-
Perform a detailed analysis of the transfer system to determine both State's
processes are similar in nature and operations.
-
Dedicate a staff of individuals to specifically work on the transferred system.
-
Ask for compliance reviews from ACF as soon as possible after system development.
The compliance review is very helpful.
-
Allow more time to complete project.
-
Do not set requirements for States where they have to "jump through hoops"
(e.g., transfer systems).
-
Streamline review and approval process.
-
Simplify reporting process.
-
Improve timeliness of everything.
-
The transfer of another State's system was very ineffective. We had to modify
the system before it could be implemented.
-
The transfer concept helped us meet the new deadline, however, make sure
the transfer concept does not lock you into old technology.
-
Transfers do not work as well as new "ground-up" systems.
-
ACF should issue a more general statement of "what" the system should do
rather than this extensive laundry list of detailed requirements. ACF's best
attempts to force "standardization" across the county are futile.
-
There is nothing wrong with certification, as long as it is focused again,
on the "what" rather than a particular solution or approach as to "how" the
"what" is accomplished.
-
ACF's mission should be to help the States in any or all aspects of the process
that they can.
-
All policy interpretations from ACF should be in writing.
-
States should request from ACF an on-site review of the Certification Guide.
-
All Federal certification requirements need to be fully defined, along with
necessary State policy and procedures, prior to beginning the design and
development of a data system.
-
ACF's certification requirements must be more clearly delineated "up front"
and further in advance of any planning process.
-
ACF should establish solid requirements that States must meet, and these
requirements should not be changed while the project is in progress.
-
Corrective action visits are needed to emphasize Federal requirements and
to assist the project in the maintenance and acceleration of momentum.
-
ACF User Group meetings are very helpful.
WORKING WITH PRIVATE CONTRACTORS
-
Hire a qualified contractor.
-
Always include some type of penalty in your contract for noncompliance.
-
Remember, contractors are trying to make as much profit as possible.
-
Do not let contractor table important key issues. It is the State's
responsibility to make sure all points are covered and addressed.
-
Be prepared to manage your contractor.
-
Set standards for contractor performance that ACF will help to enforce.
-
Avoid using a contractor, if possible. It may take more time without this
outside assistance, but you will really own your system, know how to run
it, and change it when necessary.
-
Know the financial stability of the contractor.
-
Establish key contractor senior management commitment and responsibility.
-
Develop a close and cooperative working relationship between State and contractor
staff.
-
Develop integrated design teams with State and contract staff combined. This
method makes for the swift identification of any differences in opinions
on how to design a piece of software, and make issue resolution at the lowest
levels possible.
-
Do everything you can to foster open communication between State and contractor
staffs.
-
Set liquidated damages at $5,000/day to send a clear message to contractor
via late deliverables.
-
States should have the necessary expertise to select contractor staff.
-
Vendors must be made to adhere to all terms of the contract and RFP.
-
Contractor staff wanted to do a good job. There were several exceptional
programmers whose dedication saved this project. They went above and beyond
what was expected by their management.
-
Firm, fixed price contracts are usually underbidded and you always get what
you pay for.
-
Contractors should be located on-site.
-
System development should be performed on-site. It should be a 50-50 effort
between the State and the contractor.
-
The requirements and expectations for contractors and State staff alike should
be expressed in as much detail as possible and as early on in the process
as possible.
-
Much of the success to date can be contributed to the ability and willingness
of both the contractor and State to remain flexible and adjust throughout
the project.
-
The State knows itself. The contractor needs to listen to State staff.