Last week the microformats.org community celebrated its 7th birthday at a gathering hosted by Mozilla in San Francisco and recognized accomplishments, challenges, and opportunities.

Humans First: Admin Emeriti & New Admins

The microformats tagline “humans first, machines second” forms the basis of many of our principles, and in that regard, we’d like to recognize a few people and thank them for their years of volunteer service as community admins:

• Dan Cederholm (co-founder)
• Ryan King (co-founder)
• Robert Bachmann
• Dimitri Glazkov
• Drew McLellan
• Scott Reynen

They’ve each been essential positive community contributors and guides over the years, and as admin emeriti are always welcome back should they decided to become admins again.

We’re also pleased to announce two new community admins:

• Ted O’Connor of Apple
• Christophe Ducamp

Both have similarly been consistent positive contributors to microformats for years, and we’re very happy they’ve stepped up as community admins.

Challenges & Opportunities

During microformats.org’s first seven years, we’ve seen many other attempts to promote structured data on the web. Some have since disappeared or been retired, one never got past an initial blog post, a couple have gained traction, and couple were just launched in the past two weeks:

• 2005-2009(?): StructuredBlogging
• 2005-2011: Google Base schema
• 2007-2011(?): Google Data API/Elements
• 2009-2009(?): Yahoo et al CommonTag.org
• 2010-2012+ Facebook OGP meta tags
• 2011-2012+ Google/MS/Y! Schema.org
• 2012-2012+ Twitter Cards meta tags
• 2012-2012+ OpenMetadata.org

Each of these is a signal that there are a few people (or a company) who want to do something with structured data on the web better than what they thought was already out there.

We can view each of these as a set of challenges and questions:

• What problems were these created to solve?
• Are they solving similar, overlapping, or different problems than microformats?
• Did their creators know about microformats?
• Did they try using microformats?
• Are they open efforts (or at least trying to be open), or are they vendor-specific?

We can also view each of these as feedback, whether intended or not, and opportunities to improve microformats. Open source teaches us that great ideas come from everywhere, thus we should analyze and document other efforts, seeking to answer the above questions.

If these alternative efforts are solving the same or similar problems as microformats, how can we improve microformats to meet their needs with established open standards?

If they’re addressing different problems, are those problems that microformats should be expanded to solve?

And if they’re open efforts, how can we best collaborate and produce even better solutions together?

microformats ~70% structured data domains

Even after seven years and the emergence of various alternatives, according to the Web Data Commons as of 2012, microformats still have the greatest adoption across different websites for publishing structured data in HTML:

Our continuing success is no indication that we should rest.

We should document the alternatives as they emerge, do our best to answer the questions posed, and reach out to other communities to find areas of overlap to collaborate. With greater collaboration comes greater interoperability.

Why microformats

As we continue to evolve and expand microformats, we should keep in mind the principles and values that brought us here. Here are a few of the core values that still distinguish microformats as a technology, effort, and community:

• Humans first (machines second). microformats are designed primarily for humans, and primarily for the greatest number of humans, whether authoring, or representing (e.g. the microformats community spearheaded and got the most inclusive “gender” property ever designed into the vCard 4 (RFC 6350) – which hCard 1.1 and 2 uses).

  Mozilla front end developer / UX engineer Gordon Brander recently remarked to me, “microformats work within existing web designer/developer workflow, which makes them easy and convenient. Other solutions require learning new attributes, which is enough of a barrier to just not bother.”

  When I related this to Sam Weinig of the Safari team, he made an astute observation and comparison: “essentially what you’re saying is that bolt-on solutions, even just attributes, whether for accessibility like ‘longdesc’, or for semantics like RDFa/microdata, just don’t work as well.”

• Very openly licensed standards. By virtue of being Creative Commons licensed from the start, and public domain / CC0 compatible since late 2007, microformats are the most openly available standards developed world-wide. CC0 provides the maximum freedom to re-use, publish, and if you’ve got a better idea, fork, experiment, and submit suggestions. Just like open source.

  This openness has shown to be particularly effective on the microformats wiki, which has 17 different translations in progress.

  These independent translators from around the world, who by virtue of committing their work to the public domain / CC0, perhaps know that not only are they sharing their work, but that the work they do cannot be taken from them. Once placed into the public domain, their work remains there, always reusable at any point in the future, by anyone.

  Contrast this with any form of writing/creating that is owned. If it’s owned it can be bought, and thus taken away from you. While that’s a perfectly reasonable trade to make for an income, for standards and longevity, it’s important that our work remain maximally public, as an open resource for generations to come.

  And finally, to date, no other standards organization has chosen to put all their research, examples, specifications etc. in the public domain / CC0. We invite every open standards organization to do so.

• Open spec history and editing.
  Two key distinguishing factors that contribute to our community openness are aspects of microformats specs:

  1. specs with open revision history – every microformats specification has an open revision history dating back to the launch of microformats.org seven years ago.

     An open revision history provides a level of transparency, accountability, and provenance second to none. In comparison, the other efforts listed above lack any kind of open revision history, e.g. clearly showing date-time, who edited, and how much changed in each spec. The microformats wiki revision histories are also easily browsable and delta-changes-viewable, far more usable/accessible than web views on revision control systems (e.g. W3C’s cvs and hg repositories).

  2. specs with edit buttons – also unprecedented and unmatched, every microformats specification is on a wiki page, editable by any account holder, should they find typos or other obvious errata / minor edits (spec editors handle larger spec edits, though anyone may copy a spec and demonstrate major edits for consideration in a copy).

     The community value of allowing such open editing cannot be understated. Many longtime contributors started interacting with the microformats community by jumping in and making minor edits or suggestions on the wiki (some, like Brian Suda, first participated by contributing edits to the original hCard specification, though we quickly followed up on IRC).

Latest in microformats

The past year has seen several interesting and useful developments in and related to microformats.

HTML5 enhanced time and data elements

In November 2011 there was a heated discussion in the W3C HTML WG and WHATWG about the HTML5 <time> element. It was first dropped, and then thanks to a lot of research contributions (from many in the microformats community) and persistence, not only was <time> saved but also enhanced in numerous ways useful for microformats. As such, every current use of anything date or time related (including durations) in microformats should use the HTML5 <time> element. For hCard and hCalendar, if you depend on H2VX for providing “add to address book” and “add to calendar” features, use “dev.h2vx.com” URLs which have support for HTML5 semantic elements, and follow @h2vx on Twitter for updates.

In addition, HTML5 now also has a new <data> element, for other types of data where the human representation is not easily/unambiguously machine-readable, or just uses a different format, e.g. geo latitude/longitude coordinates. The HTML5 <data> element is a nice upgrade from the Value Class Pattern ‘value-title’ feature we developed in microformats a few years ago.

Media Temple Server Hosting

For the first six years, the microformats.org server was sponsored by Commercenet, for which we’re very thankful. And for the past year Rohit Khare graciously supported our server hosting.

However from an infrastructure viewpoint, the server has been on the same virtual hosting, and in some cases original software, since launch in 2005.

We’re very happy to announce that Media Temple is providing new state of the art hosting for microformats.org. After working several hours over a few weekends, we transitioned everything* to the new server and flipped the DNS switches without any downtime. The new server is noticeably faster and more responsive, which has made wiki-editing even more seamless.

Microformats 2 – Start Publishing And Parsing

microformats 2 has been in development for the past couple of years, incorporating lessons learned from many years of using microformats in the real world as well as experience with various other semantic HTML data markup technologies (e.g. microdata, RDFa). It’s stabilized and ready for trying out in real world experiments, both in pages on the web, and the parser implementations.

There are already several examples in the wild publishing microformats 2 versions of hCard and hCalendar, including this very blog post. At least two parsers are in development as well, one in Ruby, and one in JS.

The time has come: if you write a tool that publishes or parses microformats, update it to support:

• Value Class Pattern
• HTML5 <time> and <data> elements
• microformats 2

Test your parsers with both the examples in the spec and the ever growing list of microformats 2 supporting sites.

Stay involved with IRC, wiki, and Twitter

While our blog and mailing lists were used a lot in early days, IRC and Twitter seem to have become the first place many ask questions about microformats, and thus we’ve adapted and are responding accordingly.

• Twitter: @microformats and #microformats
• IRC: #microformats on Freenode. IRC in particular has become the place for discussing new microformats developments (like microformats 2), or detailed microformats issues.

Join us and start using microformats 2 today. If you have any questions about microformats 2, whether how to use, publish, or parse, please feel free to ask on the IRC channel.

Today, Google launched a new search feature: Recipe View!

The new search category enables users to discover recipes that have been marked-up following the hRecipe specification from a variety of sources with a new level of accuracy. Google have made it easy for users to find recipes, because authors are now making it easy for them to locate their data from within their web pages.

Wired reports:

“Our intent is to make better user expereince to see if we can jumpstart this ecosystem,” Menzel said. “That way when someone thinks ‘Hey, I just invented a great recipe, let me put it on my blog,’ and that person’s recipe should be a candidate.”

But Menzel insists it’s got to be easy and that Google doesn’t want to push busy webmasters to do any work that won’t result in more traffic.

“This is really a pragmatic response to the dream of the semantic web,” Menzel said. “We would love if the XML world existed — it would make search awesome, but no one is going to to do it. But we need to start somewhere, and a lot of the internet is built manually by people and their time is valuable.”

ReadWriteWeb also notes:

Google didn’t indicate if it has plans to expand this sort of markup into other search efforts, but it’s a good reason – at the very least for recipe publishers – to mark up your websites.

We’re very excited about this new feature from Google, and are pleased to see the organisation continue to support and implement open standards that are simple for authors everywhere to use.

For more information on the specification, check out the hRecipe wiki pages.

As of today, Facebook has marked up all events with the hCalendar microformat including marking up their venues with hCard as well. According to a simple Google search, that’s millions of public events now with microformats (if anyone knows a more precise number for the total number of Facebook events including private ones, or how many events are created per day, please let us know!).

Here’s an upcoming example: the Great British Booze-up at SXSW 2011 (which I highly recommend if you’re into high fidelity markup, as a bunch of us will be there sharing drinks and post-future-panel microformats conversations).

Visit it in a browser that supports microformats (there are now plugins for Chrome, Firefox, Internet Explorer, and Safari). E.g. try Firefox 4 with the Operator Add-On, and you’ll see “Events (1)” in the toolbar. Just one click reveals “Great British Booze-up” which you can then choose to export to your iCal, 30 Boxes, Google, or Yahoo calendars.

And if you’re a user, that’s it. The ability to copy events to where you want them. If you publish hCalendar, Google will index and show rich snippets for your events. That’s what microformats empower you to do.

For the web authors/designers/developers out there, let’s take a closer look at Facebook’s markup. If you view source on the page and search for “vevent”, you see the following code:

<body class="vevent ...

Which indicates that this page is an event. Searching further in the source you’ll find that Facebook generates the rest of the from a script. While this isn’t ideal (in general it’s better to have the markup in markup), apparently there are sometimes performance reasons for script generated content. No matter, we can use Firefox’s “View Selection Source” context menu to view the generated markup.

I want to call out two specific aspects of Facebook’s implementation. Select the entire date time row: “Time Monday, March 14 · 6:00pm – 9:00pm” and right/control-click on it and choose “View Selection Source”. Here’s most of the markup you’ll see (white-space added for readability).

 <div>Monday, March 14 · 
  <span class="dtstart">
   <span class="value-title" title="2011-03-14T18:00:00"> </span>6:00pm</span> - 
  <span class="dtend">
   <span class="value-title" title="2011-03-14T21:00:00"> </span>9:00pm</span>
 </div>

Minor update 2011-02-18: The “-07:00″ timezone offsets were removed as they were not reliably accurate. Better to omit the timezone offset, use floating datetimes and let the consumer infer timezone from location as needed.

Encoding dates and times that work for humans and machines has been one of the biggest challenges in microformats, and what you’re seeing here is the result of years of community iteration (techniques, feedback, research) called the ‘value-title’ technique of the value class pattern. In short, by placing the machine readable ISO datetime into the title attribute of a harmless empty span element adjacent to the human readable date and time, we are able to achieve a pragmatic balance between user experience, content fidelity, and minimizing the effects of what is essentially duplicating the data (a DRY violation, something we avoid due to potential inconsistencies unless absolutely necessary for a greater principle, such as usability – humans first as it were).

This is the largest deployment of the ‘value-title’ technique known to date, and works great with the value class pattern support in Operator.

Let’s take a look at the venue. View Selection Source on the entire block from “Location Shakespeare’s Pub” to “78701″ and you’ll see (again with whitespace added)

 <div class="location vcard">
  <span class="fn org">Shakespeare's Pub </span>
  <div class="adr">
   <div class="street-address">314 East 6th Street</div>
   <div class="locality">Austin, TX 78701</div>
  </div>
 </div>

This is an excellent example of using a nested hCard for an hCalendar event venue, except for one thing:

   <div class="locality">Austin, TX 78701</div>

Which should be marked up more like:

   <span class="locality">Austin</span>, 
   <span class="region">TX</span>
   <span class="postal-code">78701</span>

I’m guessing what’s going on here is too coarse of an interface between a backend and frontend system, that is, for convenience the developers may be retrieving the entirety of the city, state, and zip from their backend as a single string, and thus the best the front end can do is to mark up the entire thing as the city (locality).

While not ideal, this isn’t horrible either. Using Operator again, choose “Export Contact” for Shakespeare’s Pub, and note that your address book displays it just fine (even if the fields aren’t exactly in the right spots). Copy and pasting that address to a map site also works. The markup isn’t ideal, but it’s usable and useful, and I for one am happy that Facebook chose to go ahead and make that pragmatic decision and ship now, while knowing they could iterate and improve data fidelity in a subsequent update.

Facebook’s deployment of hCalendar is just the latest in their series of slow but steadily increasing support for open standards and microformats in particular. Over two years ago Facebook added hCard support to their user profiles. Last year they announced support for OAuth 2.0, as well as adding XFN rel-me support to user profiles, thus interconnecting with the world wide distributed social web. They proudly documented their use of HTML5. And now, millions of hCalendar events with hCard venues. Looking forward to seeing what they support next.

Well done Facebook, and keep up the good work.

Inspired by the Wikipedia community, in the early days of microformats.org we had a bot, mfbot, setup by microformats.org co-founder and admin Ryan King, that reported wiki changes to the #microformats IRC channel. mfbot performed an invaluable role, both helping newcomers see what areas of the wiki were active, and helping admins quickly revert and block spammers. Unfortunately mfbot turned out to require just enough handholding that made it too much work to keep running over the long run (the cognitive surplus of our all volunteer admins tends to be quite limited, we’re busy folks).

Enter Loqi. In the process of setting up indiewebcamp.com (which all microformatters should check out) with Aaron Parecki today, he setup his IRC bot, Loqi to report IndieWebCamp wiki changes to #indiewebcamp. Turns out Loqi is running continuously and can handle watching changes across multiple wikis, notifying the appropriate IRC channels.

So now, Loqi is now watching the changes from the microformats wiki and reporting them to our our IRC channel (click to join).

Welcome Loqi!

7 years ago Kevin Marks and I presented “real world semantics” in an after-hours open sign-up slot at O’Reilly’s Emerging Technology (ETech) conference and first publicly introduced “microformats” to the world.

To put it in perspective, that very same ETech saw the launch of Flickr, which just celebrated its 7th birthday yesterday. Happy Birthday Flickr!

Since then we’ve had many ups and downs in microformats, learned many hard lessons in community management, and seen both billions of pages add microformats, as well as the adoption of new microformats like hRecipe, along with search engine support and widespread adoption by the SEO crowd (certainly quite a measure of having “made it”).

We’ve seen the emergence of competing syntax standards like RDFa and microdata, both of which have sought to solve general purpose problems beyond the common use cases addressed by microformats. We’ve also seen the rise of proprietary non-standard (AKA “snowflake”) APIs, and Facebook’s standards-based Open Graph Protocol (OGP).

From a cultural perspective, the launch of microformats.org in 2005 inspired and influenced the creation of numerous additional independent community-based standards efforts outside traditional organizations like W3C and IETF. From OAuth (which Twitter and others now depend on) to OEmbed, from the HTML Design Principles (many of which echoed microformats principles), to perhaps most recently ActivityStreams, and their real-world pragmatic microformats-process-like approach to introducing new object types and verbs.

Despite all these successes, there are still some longstanding issues affecting microformats, both the formats themselves overall, and the community. Upon reflecting on the past 7 years as well as learning from what’s worked (and not) in other open standards and open source organizations (W3C, IETF, Mozilla, WHATWG) it’s clear to me that these community and cross-format issues are what need to be addressed for microformats to continue advancing.

I’m not going to attempt to explore all the issues in a single blog post. Suffice it to say there are concrete issues around specification stages, simplifying microformats (use of, parsing, extending), and overall community participation models that need attention.

One aspect of microformats that has stood the test of time is the set of microformats principles. The first principle encourages us to solve a specific problem. That principle applies to processes as well as formats and as such the path forward is to solve specific, real world overall microformats problems, one at a time.

I’ve chosen document stages as the first such specific overall problem to solve, documented a real-world problem statement, and begun brainstorming more precise definitions for “draft”, “specification”, and a new one, “standard”. These definitions will update, expand, and clarify the microformats process and what we mean by microformats specifications. If you’re curious, you can take a look at the brainstorming in progress.

These new document stages are just one of many updates to microformats that we are working on and will be introducing and implementing in the coming months. Stay tuned: join the IRC channel, mailing lists, and follow @microformats on Twitter.

Mark your calendars: in just over a month’s time, myself and fellow admins Frances Berriman (Nature Publishing Group) and Ben Ward (Twitter), as well as Paul Tarjan (Facebook) will present a panel at SXSW Interactive 2011 on “The Future of Microformats“ where a lot of this and more will be discussed.

Join us and help shape the future.