President Obama's new open data initiative could help cities help themselves

If you can inspire people to make tools that help communities understand and improve themselves by releasing the data they need, they will. That, at least, is what the White House is hoping with the launch of The Opportunity Project, a new initiative from the Obama administration that builds upon nearly seven years of effort to release government data to the public, for the public. The White House is hoping to inspire developers to build tools using federal and city data that will make it easier for decision makers to access the information they need.

That's far from new: The White House has been trying to inspire the nation's software developers to build tools using open government data since May 2009, when the former US chief information officer Vivek Kundra first launched Data.gov, appealing to people's patriotism and profit motives. To paraphrase President John F. Kennedy, they've challenged the nation to ask not what your country can code for you, but ask what you can code for your country. Taking inspiration from how weather and GPS data both informed the public and enabled third parties to build services using it, the Obama administration has focused on unlocking innovation by publishing massive amounts of government data online.

While those goals have advanced in fits and starts — held back by quality issues, problems with Data.gov, and a lack of political will to publish data sets in demand by industry and the media — the administration has learned a few things. Perhaps President Barack Obama has, too.

According to US chief data scientist D.J. Patil, as quoted in WIRED, the idea for The Opportunity Project came from the president, whose experience as a community organizer informed his perspective on the world.

This past week, President Obama focused on the "spirit of innovation" in his weekly address, hailing American innovators who created the lightbulb, the internet, and the telegraph.

"...our founders trusted us with the keys to this system of self-government because it's the best tool we've got to settle our differences and solve our collective challenges," he said, "and it's only as good as we make it."

What's new about this initiative?

What I saw at the launch of The Opportunity Project yesterday suggested a shift in an approach that has promise. Instead of simply dumping a data set onto Data.gov and challenging people to use it, the White House worked with over 30 tech companies and nonprofits to develop prototypes of new tools or add features to existing platforms. There's a Slack channel to keep the conversations started at the event going, a commitment from educational institutions to improve data literacy, and a set of user scenarios to focus upon.

Those tools are powered by new data sets released by the US Department of Housing and Urban Development that were derived from US Census Bureau data, including the American Community Survey, the 2010 Decennial Census, and the Longitudinal Employer-Household Dynamics survey, the Jobs Proximity Index, the Labor Market Index, the Low Poverty Index, racially/ethnically concentrated areas of poverty (R/ECAP), the Low Transportation Cost Index, and the Location Affordability Index.

Last year, the Census Bureau released an open source City Developer Kit to make it easier for developers to use its data. There's considerable demand for it: Since the Census launched its API in 2012, it has had 1,743,989,207 calls, with 13,204 keys provisioned for use.

Instead of starting with data and looking for problems to solve, the eight US cities (Baltimore, Detroit, Kansas City, MO, New Orleans, New York, Philadelphia, San Francisco, and Washington, D.C.) participating in The Opportunity Project looked at the challenges their residents face, from affordable housing to education to public health to transit, and then looked for the data needed to build tools that can help solve those issues. By creating a set of shared facts about where problems exist, policymakers and residents are on more equal footing. Eventually, the thinking goes, cities may even be able to use data to then compare the efficacy of different policies that seek to address structural racism or urban blight.

Releasing data and tools like this enables a kind of "participatory development," suggested Aden van Noppen, a special advisor within the White House Office of Science and Technology Policy, speaking at yesterday's launch, where the public can be more directly involved in identifying issues, holding their governments to account, and working towards improvements.

In the information age, limited access to data can represent both power and privilege. Communities that don't have data about themselves or the capacity to access it or literacy to understand and apply it suffer from data poverty.

"In the 21st Century, equal access to data is critical to the functioning of our democracy," said Seema Diyer, an urban planner at the launch of The Opportunity Project.

Shiny new apps aren't the big idea

The dozen digital tools at Opportunity.Census.gov could make a differences in people's lives: They're all oriented at reducing the asynchronies of information that persist in American cities between people making policy and those who live under its consequences. The challenge for that theory of change is that members of the community not only have to know that these shiny new web services and apps exist but also find them useful.

Will a "National Equity Atlas" lead to inclusive growth if few people follow its map of disparities in a city? Will inequities in opportunity revealed by DiversityDataKids.org lead to changes in those communities?

"We developed the Atlas with the goal to be a user-friendly tool that provides difficult-to-find disaggregated data and visualizations for community leaders and advocates who are working on policy change in cities and regions," said Sarah Treuhaft, director of Equitable Growth Initiatives at PolicyLink, in an email. "This is a quite different, and much smaller, universe than the user base of tools such as Zillow and Redfin."

PolicyLink and our partners at the USC Program for Environmental and Regional Equity (PERE) are both intermediary organizations that provide relevant research to such community leaders and work with them directly to advance policy change. PolicyLink has a strong network of community leaders — CBOs, policy organizations, government leaders, philanthropists, elected officials, some private employers — who are working to advance equity and inclusion locally. The Atlas was created to serve the demand that we heard from them for data on demographic change and indicators of inclusive growth broken down by race/ethnicity, income, gender, and other demographics.

We've found that people are using the data to understand conditions in their communities, inform strategy and policy, incorporate into funding proposals, engage their own networks, and build broader coalitions with shared narratives about the importance of equity and inclusion to their future growth and prosperity and strengthen the base of support for equitable solutions.

The recent history of civic apps and open government platforms suggests that discovery and utility are major issues, particularly in the context of consumer "app fatigue." That's why the incorporation of open government data into platforms and services that people already use matters.

When cities release standardized transit data, Google Maps can show you when the next train or bus will arrive — or won't. When Yelp incorporates data about hospital wait times and nursing homes inspections, consumers can make more informed choices.

What's most likely to endure are the civic features in thriving platforms created and operated by tech companies that use open data, like real estate companies Zillow and Redfin, both of which feature in The Opportunity Project.

Redfin combined new data sets from the Department of Commerce with Redfin's existing Walk Score and Transit Score and data on properties for sale and rent, and created a new Opportunity Score that can help home buyers think through the biggest purchases of their lives.

"Affordability is a growing challenge, one we see firsthand as a real estate brokerage helping people buy and sell homes," a Redfin spokesperson told me, "but the price of a home or the cost of rent is only part of the equation. What kind of life can you have in your new home? Homebuyers consider what their commute will be like, what social communities they'll be part of, and where their kids will go to school, and the opportunities their family will have for upward mobility."

Beyond the tech, what's most likely to matter long after President Obama leaves the Oval Office is the big idea behind this effort. This isn't just about a "new website" or the notion that civic infrastructure created or supported by its government should also be the public's; it's the fundamental principle that public data is the public's to use and re-use for commerce and social benefit.

This is not a new idea in US history, as US Census Bureau Director John Thompson reminded the audience at the White House yesterday, but its expression in our digitally infused present remains full of potential.

An enduring legacy of the Obama administration will be the idea that a government of the people, by the people, should work with the people to apply data to lift everyone up.

Also see

• How the White House 'Opportunity Project' wants to use open data to improve communities (TechRepublic)
• Welcome to data transparency in real estate, where possibilities and challenges await (TechRepublic)
• More than economics: The social impact of open data (TechRepublic)
• Open data, open mind: Why you should share your company data with the world (ZDNet)