Document Actions
Appendix 1 - Data concerns
Context, Considerations, Comparison, Cautions, and Confidence
Some questions and issues to consider when interpreting the Albuquerque Progress Report 2008
LOCAL TRENDS
1. Am I thinking about an indicator in the CONTEXT of what it is intended to measure (the Desired Community Condition)?
Many indicators contained in this document are used to shed light on very complex Desired Community Conditions (DCC) for which no single perfect measure exists. The Progress Report does not measure the daily weather (F or C) or the amount of the City budget ($). Some indicators measure part of the whole or an example of a larger group. Some indicators measure a condition related to the Desired Community Condition. Some indicators measure the exact opposite condition of the desired condition (e.g., uniform crime rate as an indicator of “people are safe.”) It is very important to put each indicator in the context of the condition it is meant to indicate and not confuse the indicator with the Desired Community Condition.
2. What is the source of the data and how was it collected?
Is the source unbiased and reliable? Are there policy agendas connected to the indicator? Are the data gathered consistently? Is the measurement methodology sound? The IPC considered all these questions and identified data sources used for each indicator in each Indicator analysis.
3. What geographic area is being reported (the City of Albuquerque, Bernalillo County, the Albuquerque Metropolitan Statistical Area, which stretches from below Belen to Cuba, NM (and includes Sandoval, Valencia, Torrance, and Bernalillo Counties), the State of New Mexico, another area)?
When considering the local trend the reader is cautioned to note that various data sets use differing definitions of “local.” For example, certain data sets are available only at the State of New Mexico level, while others include the entire Albuquerque Metropolitan Statistical Area (Albuquerque MSA), which includes a four-county area (Valencia, Torrance, Sandoval, and Bernalillo Counties). Other presentations of “local trend” include only the Albuquerque City limits, and are much more specific to the local Albuquerque community. Some break the Albuquerque City boundaries into subareas, called Community Planning Areas (see map). When Albuquerque is listed as "principal city" that means that the data is for Bernalillo County, and comparative data is for the counties in which each principal city resides. Note the differing geographies included in the "local trend" as depicted in each of the maps that follows this section.
NATIONAL/REGIONAL COMPARISONS
4. To what extent can I compare results from Albuquerque to other cities or areas, or to national results? How many variables influence the indicator from jurisdiction to jurisdiction?
| In this report the IPC presents comparative data from other jurisdictions only for context and to spur questions in our community about why Albuquerque is the way that it is. The IPC has meticulously stayed out of the policy arena (what is or should be done to improve a particular Desired Community Condition). That is the domain of policy leaders, advocates, and Albuquerque citizens concerned about their community. |
An example of caution in comparisons is provided by the FBI about comparing crime rates and ranking jurisdictions.
The International City County Management Association (ICMA) has also noted other issues that may impact comparative condition measurement and governmental performance measurement and urges caution in comparing jurisdictions for several reasons summarized below:
- Weather, governmental mandates, demographic differences, variations in local tax bases, differing budget priorities as well as other factors may impact performance and conditions;
- Data from a particular jurisdiction may not be collected in the exact same way as others;
- ICMA does not rank jurisdictions and does not set benchmarks or targets.
ICMA uses these standards and measures for data collected from cities and counties.
- Defines indicators of effective service delivery.
- Collects "apples-to-apples" comparative data from participating jurisdictions on these indicators and gives participants access to the full data sets.
- Ensures a consistent set of data definitions among participating jurisdictions (as developed by participating jurisdictions).
- Employs a rigorous data-cleaning process to ensure the integrity of the data and other information (using both computer models and ICMA staff review and oversight).
- Facilitates analysis and discussions among program participants to determine the management practices key to communities in which data indicate high performance. This is accomplished via Web-based discussions on the private portion of the CPM Web site as well as face-to-face discussions among participating jurisdictions.
- Collects "best practices/effective practices" as to the ways in which jurisdictions are using the data in their budgeting as well as financial/program planning processes.
5. What is the source of the comparative information, how reliable are the data, how consistently was the data collected from jurisdiction to jurisdiction?
CITIZEN PERCEPTION OF COMMUNITY CONDITIONS
6. Two surveys were conducted, breaking the DCCs of the eight Goals into two groups: social and economic goals and physical (development) and government goals. Each survey had a stratified random sample of about 3000 addresses (the city of Albuquerque’s geographic information system has an address file, but it does not contain names of households, only street addresses). Census Blocks with high proportions of minority households and lower income households were over sampled to compensate for an anticipated under response. This was done by randomly selecting addresses in those blocks.
These surveys have been used to assess the extent to which Albuquerque residents’ perceptions of DCC importance and progress correspond with what the indicator data say about the respective DCC. In each case, DCC importance was rated higher than DCC progress. The difference in the means between the DCC importance and progress can be viewed as an indicator of priority (for example, a high importance mean with a low progress mean would produce a large difference).
It should be noted that several of the more complex community condition statements were simplified and modified to improve their understanding. More common words were substituted for technical terminology. For more information contact the Office of Management and Budget.
The chart in Appendix 2 compares the results of the 2007 IPC surveys of Albuquerque residents.