There is a nice article about computational journalism in the current issue of Miller-McCune. It mentions lots of good use of government data and Sunlight in particular.

Deep Throat Meets Data Mining by John Mecklin. Miller-McCune (January/February 2009)

(email from our google group)

Clay Johnson, Sunlight’s Labs Director, has put together an introductory post and wiki page on how anyone can help participate in the labs work.

In that same spirit, I’ve got three rather simple things that anyone could do to help open the House and Senate.  If editing a wiki is a challenge, don’t worry!  Still-preliminary formatting is among the best of all problems to have, and the editors of Congresspedia are always happy to help, too.

Assemble a clear view of what’s available.  When we first started the Open House Project, I did an inventory of the information House information I could find from official sources.  That wiki page has been cleaned up, and is now available on Congresspedia here.  I started a Senate version, too.  These two pages will help enormously, since we can’t advocate for more openness without fully understanding what’s available.  If anything is missing, please feel free to add it!  The Senate version of the page will have many of the same resources as the House version, but they haven’t been added yet.  We should also consider adding such an information ecology for executive branch data, although I think this work may have been started elsewhere already.  Has it?  Should we start a third (executive) page, and link it to the other two?

Share Events.  If you know of any upcoming events related to transparency in government, add them to our wiki page.  No single news source does a comprehensive job collecting events resources, despite exhorbitant costs, so that’s something that issue advocates should probably take upon themselves to collect.

Share Reform Ideas.  As the Open House Project moves along, and as the Open Senate Project catches up, we’ve got a ton of issues to address.  If something strikes you as under-addressed, or you’ve got a question about how to find some piece of information or data, please ask.  I prefer that we err on the side of too much (relevant) email, rather than not enough.

thanks!

John

If you’re like me, you’ll watch this straight through.  Start with “NEW Behind the Scenes Video”, and see if you agree — this is compelling stuff.

I don’t know what it is that makes this so, but even though I’m not a resident of Representative Buck McKeon’s California House District, I can’t look away.

In the video, the Congressman wanders through his office, conducting impromptu unscripted interviews with his staff, soliciting messages for his constituents back home from unprepared staffers, who manage such awkard replies as “uh, Hi!”, or “There’s my desk.”

But that’s what makes this video a sleeper hit in the world of congressional office video.  One might expect to be put off by the video’s awkward moments, where the boss shows up with a new video device and asks for messages for “the folks back home,” but that awkwardness turns out to be a big asset.

As Rep. McKeon and his new media staffer (whose desk we get to see) wander the cubicled workspaces of their coworkers, they manage to portray their office as entirely normal, approachable, and welcoming, in exactly the way that most offices are, whether there’s a view of the US Capitol looming there or not.

If I were one of his constituents, I’d feel quite welcome calling or stopping by the district office.

CSPAN has come a very long way with their digital video services, evolving from an over-aggressive assertion of copyright to now offering embeddable video.

Here’s a clip of Chairman Brady discussing H. Con. Res. 307, a sense of the Congress resolution on preservation that passed:

OpenCongress has just released a votes comparison tool.

This strikes me as a big step forward. Several elements of well-coordinated data have been combined to produce something much more useful than the sum of its parts.

A well structured database of bills, votes, Members of Congress, and notable bills (see the “Hot” icon?) allows for a quick and dirty comparison of voting records on bills that are easier to evaluate. The comparison also highlights the differences between selected Members’ votes, allowing for a quick evaluation of differences.

While legislative intent is still hard to boil down to individual votes, tools like this one from OpenCongress go a long way toward turning a jumble of legislative information (and mis-information) into something comprehensible.

Many acts of Congress are passed with a reporting requirement.  Throughout the US Code are various requirements that agencies submit reports to Congress.  This is a good thing, since strong congressional oversight can create constructive incentives within agencies (and, by extension, the entities those agencies regulate).

Here’s the problem: agencies often ignore these reporting requirements entirely, turn in reports late, or don’t even know what reports are required of them.  Staff and Members on oversight committees are sometimes similarly delinquent, unaware of what reports may be pending or overdue.

Anytime the overseers and the overseen aren’t aware of the reporting they’re required to prepare, oversight has no chance of doing its job.  If this is sometimes the case within the executive and Congress, what chance does the public have of keeping track of what government is doing?  Pretty slim.

There’s no centralized list of upcoming reports, so congressional oversight staff, journalists, and agency staff have no menu of reporting responsibilities, and are stuck making assumptions based on what is often an incomplete view of statutorily mandated responsibilities.

I’m proposing a solution to this situation, see the end of this post for a summary.

A few months ago, I suggested a partial fix to this problem — a requirement that oversight plans, as prepared by congressional committees in the US House (and not the Senate) be given two additional requirements.  First, I suggested that after the oversight plans for each coming Congress are created, they should be made public by the Clerk of the House and the Oversight and Government Reform Committees, where they’re only required to be submitted now, not published.  (The House Rules Committee, for example, does this now.  I’d like to see a page or document with all committees’ oversight plans.)  This would provide a central place to see the collected oversight plans from across all House committees, not unlike the document now compiled that has all House committees’ rules.  That would give a centralized congressional plan for oversight, across committees, but wouldn’t ensure that committee staff, agency staff, or the public knew what reporting to expect from the agencies.

That’s the second part of my earlier suggestion: to require that House committees’ oversight plans contain a list of expected reporting within their jurisdiction.  (I’m ignoring the Senate here.)  It turns out that this requirement is suggested within the House Oversight Manual, under the section “Selected Oversight Techniques:”

1. Prepare a document, as needed, which outlines for each subcommittee of a standing committee the agencies, laws, programs activities, functions, advisory committees, and required agency reports that fall within its jurisdictional purview.

So that’s where my proposal stood a week ago: amend House Rules to require a list of expected reports from agencies, and provide a central place from which to access all of the oversight plans.

I still like that proposal, but further reflection reveals this as utterly insufficient.  How can effective oversight be mandated through self-imposed rules changes on Congress?  I still hold that those two changes would go a long way toward encouraging oversight, but without addressing the negligence or disorganization within the executive, Rules changes can only help so much.

After several conversations with subject matter experts, I have two additional suggestions for strengthening Congressional (and public) oversight of the Executive Branch.

First, the first step toward any legislative or rules changes will probably be to appreciate the scope of the reporting requirements.  CRS should be instructed to do a review of all of the pending agency report expected over, say, the next year.  If this were organized both chronologically and by agency, that document would immediately be immensely valuable to all committee staff preparing the next Congress’s oversight plan, and also to the agency staff responsible for prepring the reports.  It would tell them what’s expected from them, and what they can expect to have delivered to them, and when.

This CRS review of pending agency reports should also prove the value of having an aggregated reporting list to agency officials and Members of Congress both, regardless of political affiliation.  If you’re inclined to be critical of the next administration, then a menu of report cards should look like opportunity for criticism.  If you’re inclined to defend the next administration, then accountable and effective agencies should know when their next reports to Congress and the public are due.

Second, ongoing publication of expected (and overdue) reports (by chronology and agency) should be a responsibility taken on by the administration.  Two candidates for this responsibility:

OIRA - The Office of Information and Regulatory Affairs OIRA, part of OMB, seems like a natural fit, since congressional reporting is agency-created information.  OMB seems wary of taking on enforcement responsibility, so this could be a hard sell.

OLC - The Office of Legal Counsel OLC, part of the Department of Justice, is responsible for interpreting the law on behalf of the administration.  Who better to remind the agencies of just what their legally mandated responsibilities are?

I’d just like to add that there’s some precedent for this sort of thing, in the Unified Agenda, or “Semiannual Regulatory Agenda” which gives a prospective view of the next six months of rulemaking activity.

A “Semiannual Reporting Prospectus” should be similarly compliled, submitted to the Congress itself, and published in the Federal register.

If financial reports, political polls, annual job reviews, and students’ report cards can all be organized and coordinated so that we live in expectation of what they’ll say, shouldn’t we be able to do as well with our government?  The US Code may be as splintered and complex as the committee jurisdictions that oversee them, but our public view of their performance doesn’t have to suffer from their inherited discord.

A few simple adjustments can clarify and strengthen oversight of executive agencies:

• Committees should write oversight plans, which include what reports they expect to see from agencies, with dates (as required by law).
• Congress should publish these reports in a central document or location.
• Congress should request a CRS survey of all required agency reports for the coming year.
• The executive branch should assign responsibility for oversight reporting to a specific agency, and submit a menu of upcoming reports to Congress and publish it in the Federal Register.

Additional info:

Here’s a few reporting requirements I picked out of the US Code:

strategic plans

responsibilities of the Secretary of Agriculture

and here is a reporting requirement of the Secretary of Education

We’ve come a long way since I wrote the following in the project’s first press release:
“Since the value of our recommendations increases along with the expertise of those contributing to our project, we’re looking to the greater community to help identify attainable reforms,” said John Wonderlich, who is organizing the project with Matt Stoller. “By inviting input from stakeholders normally outside the legislative process, the House leadership has taken a first step in promoting an informed citizenry.”

As we prepare for some of our next steps, it’s time to review some of the progress we’ve seen since releasing our reform recommendations as laid out in the Open House Project report:

Legislation Database:

(Report Chapter )

We’ve seen two large developments on this recommendation. First, the proposed report language for the Legislative Branch Appropriations has language directing the Library of Congress to explore solutions to providing data level access, as described in this email to the Open House Project Google Group. Second, the Library of Congress has published instructions for creating permanent links to some legislative documents, as described in this blog post.

Preserving Congressional Information:

(Report Chapter )

The recommendation that Congress preserve digital information is among the most complicated, since so many different bodies are involved in this work, and because the challenges presented by the proliferation of digital information are so vast.

First, we started researching and attending meetings of the Advisory Committee on the Records of Congress, which serves as a perfect entry point into all of the work being done on congressional preservation. This quicky led me to dig up an old document, S-Pub 102-20, which surveyed all congressional information and its archival circumstances.

We’ve weighed in on NARA’s role in preserving digital content, watched the development of the Electronic Records Archive, and publicized the existing digital archives. The National Archives (and its Center for Legislative Archives) face a huge challenge in dealing with electronic records preservation, and we’re continuing to closely follow GPO’s new FDSys program, potential evolving roles for the FDLP, public private partnerships, the new Library of Congress initiative, and non-profits like public.resource.org that can help fulfill government’s role in preservation.

Congressional Committees:

(Report Chapter )

To get congressional committees to post more information online, we’ve created several ad-hoc recommendations to the community at large, and also as advice to individual committees. Shortly after we created an xml standard for committee feeds, we discovered that the Senate had begun posting committee feeds (in XML on this page ), and that the House Rules committee had started posting a feed of Special Rules and Amendments as they’re posted.

CRS Reports:

(Report Chapter )

As we’ve watched the CRS report bill get introduced and wend its way through the Senate, I’ve posted a few comprehensive updates, and we continue to promote the valuable work of CRS employees and service provided by OpenCRS.com. While convincing congressional staff to release CRS reports can be daunting, we’re getting access to most recent reports through OpenCRS, whose database recently topped 14,000 reports. A solution may be in hand soon, as the Senate considers a web-based channel that would grant constituents access to the reports.

Franking:

(Report Chapter )

In what we’ve described as an unqualified victory, the House and Senate have both seen comprehensive reform to their Web Use Restrictions. A review is available here. We also clarified the degree to which the Speech or Debate Clause applies to congressional staffers working online, here. Now comes the fun part: seeing what staffers come up with.

Citizen Journalism Access:

(Report Chapter )

I don’t have any updates regarding citizen journalism credentialing. Hopefully this will be addressed to some degree in the upcoming Congress!

Clerk of the House:

(Report Chapter )

The Clerk of the House has been busy getting up to speed with the new requirements of the Honest Leadership and Open Government Act, with its many new requirements placed on the clerk. This chapter is immensely complex, and could use additional work with the offices of the Clerk and the Legislative Resource Center to coordinate between their implementation and the needs of comsumers of this information.

On the upside, many of the new resources are going online, such as lobbying disclosure records, and broader public access is being pursued with the GPO, and with votes, as evidenced by Senator Lieberman’s remarks about access to votes data.

Congressional Record:

(Report Chapter )

After identifying the problem of an occasionally inaccurate Congressional Record, we identified that enforcement lies with the Ethics Committee, itself recently reconsidered. Inaccuracies could be better tracked, and a call for better enforcement of the accuracy (”typographical or grammatical”) requirements could be taken up.

Congressional Video:

(Report Chapter )

While bulk access to congressional floor and committee video hasn’t yet been established, the franking reforms at least allow Members to post their own videos.

Another strong candidate for reform in the 111th Congress, bulk access to congressional video will take a few small but well coordinated moves from House leadership and the CAO.

Coordinating Web Standards:

(Report Chapter )

After identifying a shortfall in technology coordination efforts within Congress, we suggested more effective authority, perhaps through an Office of Technology and Transparency. While this hasn’t happened, we’ve become much better acquainted with the bodies that oversee congressional technology — the Committee on House Administration, the Speaker and Minority Leaders, the Joint Committee on Printing, the House Systems Administrators Association, House Information Resources, the Library of Congress, and many more.

The same pending Appropriations Report language referenced earlier contains a request that congressional resource managers (those listed above) report on the current state of IT management across Congress.

This is an ideal first step towards coordinated technology planning, and we hope the language is included in an eventual Appropriations bill.

So that’s where we’ve come in terms of the Report’s recommendations.

One of the most important developments to arise in the wake of the OHP report, however, is the sustained community of activists and insiders who are constantly working to open up government. The Open House Project has quickly grown relevant outside its defined scope, and shows the real need for a collaborative space outside government.

We’ve had a large number of other successes that have arisen from the Open House Project community:

Helping pass starter funding for the OTA, advocating for release of the data behind CONAN, taking stands on executive branch issues (like OLC memos and IT authority ), picking fights with the GAO when appropriate, and working with the various individuals and organizations that make up the open government movement.

While I can’t give a comprehensive list of what the community has acheived, it’s important to recognize that there’s something important and valuable going on, and that a public email list with even a little bit of institutional approval can go a long long way.

Fulfilling one of the recommendations of the Open House Project report, The Library of Congress has published on their THOMAS web page directions for creating permanent links.

From our report:

We also recommend that:

THOMAS provide permanent links to all documents, in an obvious way, to enable Web researchers to directly refer to these documents

From THOMAS:

Legislative Handles are a new persistent URL service for creating links to legislative documents from the THOMAS web site (http://thomas.loc.gov). With a simple syntax, Legislative Handles make it easy to type in legislative links to bibliographies, reference guides, emails, blogs, or web pages. Legislative Handles, for instance, http://hdl.loc.gov/loc.uscongress/legislation.110hconres196, are a convenient way to cite legislation.

This matters because linking is one of the most basic components of online dialog. Without citations and primary sources, commentary and analysis are just opinion and rumor.

The “Handles” allow a simple procedure for linking to specific legislation, without that link expiring, as often happens if you create a link to a bill’s page accessed through a search.

This is a great first step toward modernizing THOMAS. We’d love to see additional changes, like access to bulk legislative data, or permanent links automatically created within bill pages. As it stands now, however, it’s great that THOMAS is recognizing the needs of its users, and helping the public create permanent links to legislation is long-awaited first step towards doing just that.

This weekend I’ll be at the Free Culture Conference 2008 on the Berkeley campus. Anyone out in that area want to meet up?

Yesterday, after months of negotiations and proposals, the House joined the Senate in updating the arcane guidelines that govern how Members of Congress use the Internet.

In May of 2007, the Sunlight Foundation released the Open House Project report, which included an entire chapter on the issue of Franking Reform.  That chapter, prepared by David All and Paul Blumental, has guided our advocacy and discussions of web use restrictions since then.

Those discussions simmered until earlier this summer, when tensions between Members of the Franking Commission  briefly escalated (the part of the Committee on House Administration that handles Web restrictions).  This summer’s discussion caught some media attention, and unsettled some web-savvy Representatives, and ultimately engaged both parties’ leaders in the House.

The Sunlight Foundation capitalized on the chaos, creating the first twitter-based petition in the site letourcongresstweet.org, which amassed twitter-based signatures, and displayed vigorous support for updated rules from online communities across the political spectrum.

While House officials maneuvered publicly, the Senate passed similar reforms with a bit less fanfare.  As recently as last week, agreement looked unlikely from the House committee, with Roll Call reporting that an attempt at negotiations ended in “an emotionally charged hearing and a breakdown in negotiations.”

That’s why we were suprised and delighted to get word from the Committee on House Administration that a new agreement had been reached.  This measure wasn’t just a slight rewrite, however.  The new guidelines represent an enormous change, one which has new media staff from both parties glowing.

Speaker Pelosi’s statement calls the revisions a “significant step forward toward bringing the House rules into the multimedia age and allowing for members to effectively communicate with their constituents online…  I also thank citizen initiatives such as the Open House Project for their thoughtful recommendations and continued efforts to encourage Members to engage their constituents through internet technologies.”

Ranking Member Vern Ehlers was similarly laudatory of the new rules, and of Chairman Brady’s leadership: “Mr. Brady recognized the need to allow enhanced constituent communication, and demonstrated outstanding leadership that enabled this Committee to adopt a long-overdue change,” Ehlers stated. “It is imperative that Members have the ability to use whichever web services they feel will best inform their constituents about the important issues facing this country.”

The new rules, as written, make a very important distinction, and one we’re delighted to see considered: Member web use will be evaluated based on the “official content,” and not the venue in which the materials are posted.  This puts new media communications on similar footing to traditional media, where Op-Eds and TV interviews are proximal to commercials without causing a conflict of interest.

The revisions should cause a renaissance in official political Web-use, with eager new media staff and savvy Members now able to confidently engage with their constituents.  We can’t wait to see what they come up with, and can only hope that all government reform arguments have such happy endings.