Site Historian
Introduction
All demonstration and comparison group sites will designate an individual to serve as site historian. Information recorded by these site historians ultimately is expected to be useful to the organization as well as to the demonstration evaluation.
Purpose
In evaluation research, "history" can be defined as an historic event that occurs during the time of the project that can provide a possible explanation for a project outcome. It is critical that the effects of history be accounted for when evaluating the demonstration project. When a significant effect is found, the question critical to internal validity is: "Is this particular effect the results of the demonstration intervention, or is there an alternative explanation?"
Evaluation methodology will use time-series analysis with multiple baselines and non-equivalent "comparison" groups. The purpose of having a site historian is to capture "history" or intervening events which are important to the evaluation, but which are not normally recorded or kept in the usual databases. It is especially important to maintain a written history when the research extends over a long period and when there are multiple sites that may be subject to different influences. For example, one site may be affected by change in the local labor market that does not affect the other localities. Alternatively, a change in policy or innovation in operating procedures may be well known at the time of occurrence, but may not be remembered by researchers conducting the evaluation several years later.
The site historian is specifically charged with documenting "historic" events and changes in both the experimental and comparison sites that have, or may have, an impact on the results of the demonstration project. They are not required to make an immediate judgement about the importance or expected effect of an extraneous event. It is necessary merely to determine that an event might have an effect and should be noted. Some events will be clearly pivotal, others will be anecdotal. However, even anecdotal items may be useful when aggregated. Perhaps the most difficult job of the site historian is deciding what must be recorded and what may be ignored. It is better to err on the side of caution since unnecessary notations can be deleted, whereas omitted notations are difficult to reconstruct.
Site Historian Qualifications
Each site historian is chosen based on their general knowledge and understanding of issues which could affect demonstration project implementation, and awareness of the organizational "culture". Also, if at all possible, the site historian should be selected from those personnel expected to remain at that location for the duration of the demonstration project. There is no prescribed broadband level required for a site historian. However, personnel at broadband level III, for participants, and GS 9-12 for the comparison groups, provide the organizational and event perspective needed to fulfill this requirement.
Location of the Site Historian
Typically, each paypool in the demonstration project will have a site historian, rather than at each geographic location. However, each Service and Agency will determine the number and location of historians needed to provide the maximum coverage.
Duties and Responsibilities
Site historians are required to record and document changes in the demonstration project workforce at each site, key events in demonstration implementation, and changes to the environment during the entire five years of the demonstration project. The site historian will keep a journal of events, policies, procedures, and other things that may affect the major dependent or intervening variables in the project. Weeks may go by when nothing is recorded. Other times, there may be daily entries. The very nature of the events being recorded virtually guarantees that a rigid format will not be helpful. However, a standard protocol should be followed:
- Every entry should be dated.
- Names and titles of relevant parties should be included. This will assist in locating these individuals if follow-up is needed at a later date.
- Detailed description of the event(s) should be provided.
Some examples of extraneous changes which could affect the evaluation results that should be included in the site historian log are as follows:
- Environment - Examples include personnel turnover, awards received by the installation, organizational downsizing, or a significant change in the local labor market for skills found among demonstration project personnel.
- Mission - Changes in programmatic structure from basic research to development (the new skills required could lead to different recruitment patterns). The fielding of a weapon system could also have an affect on the mission of the organization.
- Procedures - Changes brought about by improved recruitment procedures, a change in work schedules, or other enhancements leading to employee satisfaction. Enhancements are not necessarily related to hiring practices; the establishment of day care facilities, or civilian fitness programs or facilities would also be documented.
- Resources - Examples include budget changes, hiring freezes, RIFs, and upgrade or changes to facilities and work sites.
Process for Submitting
Site historian logs must be systematically collected, entered into a database, and analyzed. Download a "clean" copy of the database each quarter by clicking below. You should save it on your hard drive under a new file name, such as your organization and date; for example, save it as RDAISA_July01.mdb.
the Site Historian Log Database
Within the database there are options to make new site historian log entries, view existing entries, delete old entries, and generate a final report. Note: the database is in Access 97 format.
Depending on the procedures established by the organization, the site historian log may be submitted through a Service/ Agency directed approval process, or may be sent directly to the external evaluators, Cubic Applications, Inc. (CAI), for further recording and processing. Completed logs should be sent to:
Be sure to attach the new database (such as RDAISA_July01.mdb) to your e-mail.
Frequency of Reports
All site historian logs are submitted quarterly. Logs should be sent to CAI by the 20th of the month following the end of the reported quarter: i.e., 20 January, 20 April, 20 July, and 20 October. Negative reports are required.