SME Content Analysis
From eotcd
Contents |
Initial Content Analysis
After the classification results of each URL were completed, the notes provided by both the SME's and Arbitrators were compiled and examined. The notes were then divided into four different categories according to their content. This was done by two independent people examining each note individually and then comparing the results and making final decisions.
Categories and Guidelines
| Category | Definition | Examples |
| Classfication Difficulty | Sites that posed some classification difficulty to either an SME or an arbitrator. |
|
| Classification Strategy | Text that indicated the strategies used to determine author(s) |
|
| Website Problems | Websites that could not be displayed at all or could not be evaluated because they did not sufficiently display |
|
| Interagency Sites | Sites that identify multiple authors or are "interagency" sites, whether by one or both SMEs or the arbitrator. |
|
Secondary Content Analysis
A secondary analysis was done of each of the four groups individually. Exactly like the initial content analysis process, each group was given a set of categories and guidelines that were analyzed by two independents who then compared their results and final decisions were made.
The following tables provide information on each classification grouping, guidelines, and examples. The last two columns provide the total number of URLs that were found for each group. The last column is the percentage of those groupings when compared to the total number of URLs used for the project; 1151.
Classification Difficulty Analysis
| Category | Definition | Examples | Total URLs | Percentage URLs
N=1151 |
| Subagency Not in List | States that the subagency was not in the list provided, and implies that it should be. |
|
59 | 5% |
| Interagency Author Not in List | States that the interagency author was not in the list | 5 | .4% | |
| Needs Class | For comments that declare there is a need for a new class/update. |
|
38 | 3% |
| Guesses | For comments that indicate the SME or Arbitrator guessed. |
|
58 | 5% |
| Live Site | Comment indicates that the SME or Arbitrator referred to the live website rather than the archived one | 3 | .3% | |
| Unable to Determine | For comments that state the website was unclear or there were too many agencies to determine. |
|
33 | 3% |
Classification Strategy Analysis
| Category | Definition | Examples | Total URLs | Percentage URLs
N=1151 |
| Website Information | SME or Arbitrator state that they made their decision based on informaitn they recieved from the website. |
|
108 | 9% |
| In CPG | Their strategy was based on publications they found in CPG | 25 | 2% | |
| Responsibilty Information | The note includes information on who is responsible for the program. |
|
76 | 7% |
| Live Site | The SME or Arbitrator used the live site rather than the archived version to make their classification decision. “From website” notes were checked to see if the information was from the archived site or the live site. |
|
31 | 3% |
| Personal Choice | The SME or Arbitrator made their decision based on personal knowledge or preference. Also includes judgement calls from general appearance of impression |
|
78 | 7% |
Website Problem Analysis
| Category | Definition | Examples | Total URLs | Percentage URLs
N=1151 |
| Page Error | The website was redirected or the page was not found |
|
7 | .6% |
| Link Error | Text indicates the link was inoperative |
|
7 | .6% |
| Content Error | The website did not load properly or there was no content available for analysis. |
|
10 | .9% |
| Protected Site | The website required a login, password, or was for internal use only. | 6 | .5% | |
| GPO Class not on Class Tool List | The class in which the SME or Arbitrator wished to place the website under was not available on the list provided for them. | 2 | .2% | |
| Duplicate Data | The website contained the same data as another URL | 1 | 0% |
Interagency Content Analysis
| Category | Definition | Total URLs | Percentage URLs
N=1151 |
| Federal Government - No Lead | Federal government agencies only; no lead agency identified. | 21 | 2% |
| Federal Government - Lead | Federal government agencies only; lead agency identified | 15 | 1% |
| Government Agencies | government agencies only; can include federal, state, regional, local | 4 | .3% |
| Federal Government and Non-Government | Federal government and non-government agencies included; could include adademic, industry, etc. | 6 | .5% |
| Agencies Not Identified | The agencies are not identified | 5 | .4% |
