Microformats and RDFa are not as far apart as people think

The BBC have caused a bit of a storm recently by announcing that they won't be using the hCalendar Microformat on their pages. The reason is the well-known problems with accessibility. One of the proposed solutions is to look at RDFa.

Confusing syntax with vocabulary

Whilst the BBC obviously understand their web-pages, and are conscious of the issues of accessibility, they may have demonstrated here a bit of a misunderstanding of the semantic nature of the web. And in my view, most of the follow-up discussions I've seen fall into the same trap.

The main point is to not confuse the syntax used to convey information, with the information itself--the vocabulary. The reason I think this is what is happening is that when the BBC say "we'll be looking at the possible use of RDFa", they may have missed the point that when you use RDFa, you still need a vocabulary, and if one doesn't exist, you'll need to create one.

It would be far better to look at the Microformat that is already in use (in this case, hCalendar), and see if it can be tweaked to make use of the generic nature of RDFa--but that requires a new approach to both Microformats and RDFa.

So before panic takes hold of the Microformats community, or smugness grips the RDFa one, let's try to get underneath what these two technologies are actually about.

Microformats are about vocabulary

Microformats were devised as a way to let authors add little pieces of semantics to their documents, in such a way that applications could make use of them. An application might be a search engine that can improve indexing and search, or a browser that can display extra information based on the embedded values.

There are three key advances that Microformats made. The first was to say that you can still do useful things, even if you only have a small amount of information. This was pretty radical, since up until that point the semantic web seemed to be an 'all or nothing' proposition. (For many people it still is.)

The second was to say that we should be able to publish semantic information in the same easy way that we publish web-pages--through content-management systems, blogs, and so on.

But the third advance is perhaps the most important; Microformats essentially said that we can teach end-users about a specific set of terms that help them do something useful, without having to teach them 'big picture' stuff. This meant that authors could be taught just enough mark-up to add contact details, events, licensing information, geo locations...and so on.

Problems with Microformats

Microformats opened the way for a new approach to the semantic web, but it does of course have its weaknesses.

The first is that it 'overloads' many of the HTML attributes to carry semantic information, in such a way that they can interfere with the normal use of the attribute. This is why people are now having accessibility problems, because some of the attributes are trying to play two or three roles.

The second weakness is that mixing vocabularies starts to get messy; because each Microformat is a combination of vocabulary and syntax, then it is actually quite specific, and they can therefore interfere with each other.

The third weakness if that because each vocabulary also requires 'syntax', then even if a perfectly usable vocabulary exists, it has to be 'converted' to be a Microformat.

RDFa is about syntax

RDFa started life at around the same time as Microformats, and holds with many of the same philosophies--that the semantic web is never going to happen unless metadata is as easy to publish as a web-page, and that authors should have an easy way to add information without having to understand the 'big picture'.

But RDFa set out to address a slightly different set of problems; instead of defining new vocabularies, it sought to create a generic syntax that can accommodate any vocabulary, allowed multiple vocabularies on a page, and did so in any mark-up language.

And whereas the Microformat community took as a fundamental principle that they wouldn't modify HTML at all, we took advantage of one of the key extension mechanisms of HTML which is that unrecognised attributes should simply be ignored by an HTML parser (i.e., they should not throw an error). So by adding a handful of new attributes to HTML and XHTML, RDFa doesn't interfere with the existing uses of attributes, such as for accessibility.

Microformats and RDFa

Ultimately, if people can put their prejudices to one side, there should be no reason why these two ground-breaking technologies can't work together.

To illustrate, take the rel-license Microformat. It's a nice, simple, self-contained document about how to use the license value in @rel and @rev. To specify the license of a current document, an author can add something like this:

<a href="http://creativecommons.org/licenses/by/2.0/" rel="license>cc by 2.0</a>

This is perfectly valid HTML, making use of @rel with @href, and it's nice and easy to explain--the key component of Microformats.

But it's not widely known that this is also perfectly valid RDFa.

However, RDFa takes this a step further, and provides authors with the ability to talk about other things, not just the current document.

Individual licenses for images

A common situation is to have a number of images on a page, and to want to indicate the license of each of them. With RDFa you now can.

Imagine we have this image in our page:

<img src="http://www.flickr.com/photos/detached/2529217704/" />

Our previous example of a license link needs only minor modification to make it refer to the image:

<a about="http://www.flickr.com/photos/detached/2529217704/"
 href="http://creativecommons.org/licenses/by-nc-sa/2.0/deed.en_GB" rel="license
>
  cc by 2.0
</a>

It's difficult to argue that this is complex, since it's only a minor change for authors.

Extending rel-license

But there is no reason why this formulation shouldn't also be included in rel-license; after all, the key purpose of a Microformat is to promote re-use and provide bite-size pieces that authors can quickly learn and put to work.

rel-license could remain 'the last word' on the use of license in @rel and @rev, but it could also provide information about how to use license in HTML, XHTML and RDFa. That way there is only one place that authors need to go.

Solving the BBC problem

I said at the beginning that the BBC still needed to create a vocabulary, even if it adopted RDFa. In other words, it may gain a solution to the accessibility problem by dropping hCalendar, but it loses the advantage of having a community-maintained syntax.

That doesn't mean we have the answer straight away, but I strongly suggest that we don't throw the baby out with the bathwater here.

The Microformats community is responsible for getting many people to look again at the semantic web, after having been put off by the more theoretical approach. But it needs to build on this success, and look at updating its formats to make use of the new attributes and generic parsing algorithm provided by RDFa. That way it can solve its technical problems, and continue to lead the way in defining 'agile' vocabularies for the semantic web.

And whilst the RDFa community has shown that it is possible to have a syntax that supports the 'full' semantic web, without it needing to exist in a separate space of complex mark-up and extra documents, it should remember that a generic syntax is nothing without vocabularies.

I've just started work on an exciting project to mark up job vacancies in the UK public sector, using RDFa. And although the use of RDFa will make it very easy for departments to publish the metadata, it's still going to require the creation of a vocabulary of terms. I'm going to be looking all over for suitable terms and vocabularies, and I'll certainly be looking at how Microformats might fit in.

Openness and Innovation Presentation at Media Futures 2008

Yesterday I had a lot of fun at the Media Futures 2008 conference at Alexandra Palace. The panel I was on was called Openness and Innovation, and included Robert Cailliau, Ian Forrester, Matt Webb and myself. The chair was Bill Thompson.

My slides are here:

| View | Upload your own

SlideShare Link

@role values for SVG

Something which was part of the early design concepts from the XHTML Role Attribute Module, which has got a little lost, is that elements from any language can provide a handy source of role values too.

To reconstruct the logic:

A role value is simply a URI, or resource. The reason for this is so that the extensibility hook that we're creating puts us straight into the world of RDF.

Now, some values of @role will need to be invented. This might be because they simply don't exist, or because we want the values to be 'cross-cutting', and apply to many different mark-up languages.

But there are many values that already exist, that are suitable for use in a variety of situations. For example, XForms has a hint element, that can apply to its form controls:

<xf:input ref="surname">
  <xf:label>Surname:</xf:label>
  <xf:hint>Please enter your surname or family name</xf:hint>
</xf:input>

The semantics of 'XForms hint' and pretty well defined, so it should be straightforward to apply them to other situations. For example, an Ajax library could pick up a hint and do something with it in an (X)HTML document, even without XForms:

<input name="surname" />
<div role="xf:hint">
  Please enter your name
</div>

SVG

This whole topic came up recently because someone asked whether it would be possible to add some new values for role which would identify paragraphs, sections, headers, and so on, and that could e used in languages like SVG; but the answer is that if we use the XHTML p, section, h1, h2, etc., values then we don't need to invent new roles:

<svg:text role="xh:h1">Metadata</svg:text>
<svg:text role="xh:p">
  Metadata is data about data...which is also data...kind of
  turtles all the way down...
</svg:text>

As you can see, a role-aware voice system would be able to provide feedback to a user in any mark-up language, simply by knowing XHTML role values.

Upcoming talks on RDF, RDFa and XForms

May is going to be pretty busy with talks about XForms, RDF and RDFa coming up.

First up is my talk XForms, REST, XQuery...and skimming at XTech 2008. The talk embraces themes I've been pursuing for a couple of years now; that as we put more functionality into the client, and servers get 'cleverer', it becomes much easier to build sophisticated web applications. Of course server technologies are moving so fast now that this whole approach is making more and more sense, so I'm looking forward to taking in recent developments in my talk. For example, both Amazon and Google effectively have 'databases in the cloud' that can be used to store data and query it, via APIs, with literally no configuration.

A few weeks later I'm going to be giving a tutorial on RDF at SemTech. This is an interesting development for me, because RDF and the semantic web were always my first interests--before XForms and before XHTML 2. (As well as writing RDF parsers, and designing applications, I also contributed chapters on RDF and RDFS to a couple of books on metadata and XML.)

But one problem I always had when trying to build semantic-web applications was that defining the user interface was pretty hairy. This was partly because RDF Schema is tricky to process, but also because HTML was insufficiently powerful in its core feature-set, so the translation from RDF to HTML involved a lot of work.

The need for a user interface language that was much richer than HTML was therefore why I got involved in the XForms standard (and worked with a team of people to produce the first fully conforming XForms processor, formsPlayer). So although it may not seem directly connected to the semantic web, I believe that in the coming period XForms will start to become a key part of the semantic web's architecture.

Another problem I kept coming up against whilst developing for the semantic web was the difficulty in actually publishing metadata. In particular I always found it frustrating that there was a lot of really useful metadata just sitting in ordinary web pages, and no-one could get at it. Attempting to resolve this problem gave rise to RDFa, and I'm excited that the RDFa in XHTML working draft is extremely close to becoming a stable recommendation. And as interest in RDFa grows, I'm pleased to say that some of my other presentations in May will be 'tech talks' on RDFa at Yahoo!, eBay and Google. (I'm really excited that I might be getting to meet some of the guys behind Yahoo!'s SearchMonkey.)

My final talk of the month will be at the excitingly-named Kings of Code, and I'm looking forward to talking about XHTML, XHTML 2, HTML 5, XAML, and anything else I can think of in relation to web languages.

So how about using RDFa in Microformats?

Yes, I know...everyone seems to think that RDFa and Microformats are at war. And maybe some would prefer it to be that way. But whatever way you look at it, the work of the Microformats community has been key in getting people fired up about what they might do with metadata that is placed in HTML and XHTML pages. And RDFa is benefiting from the vibrant atmosphere which they have created.

That doesn't mean I'm saying we can ignore the problems and limitations that Microformats have, such as the difficulty in mixing different formats in one document, the work involved in creating new formats (often wastefully duplicating work that has already been done, since specialist formats invariably already exist), or perhaps most significantly, the inability to refer to things that are not 'the current document'.

So I have a suggestion.

Why don't we emphasise the 'micro' in Microformats?

Emphasise the 'micro'

One of the original motivations for Microformats was that they were small, self-contained sets of rules that authors could apply to their documents, which would give the author some kind of benefit. Nothing in that broad definition says 'so therefore steer clear of all other formats or you'll catch the plague'.

So why not use RDFa features within microformats, as appropriate? But Microformats could still retain the compactness of a particular format.

I'll use rel-license as an example.

rel-license

The rel-license microformat is simply the use of the value "license" in the @rel attribute, in some mark-up. For example, if we want to say that the current document is licensed under one of the Creative Commons licenses, we might use the following mark-up in our document:

<a rel="license" href="http://creativecommons.org/licenses/by/2.0/">cc by 2.0</a>

However, what if we have a search page that returns lots of images or videos? What if each image or video is available under a different license to other images or videos on the page, or to the license for the page itself? Trying to solve this problem within the framework set by Microformats is proving quite difficult (see the open issues on the rel-license issues page), so at some point we need something more than rel-license.

Reusing @src attribute

The mark-up we gave earlier to indicate that 'this document' is available under a certain license, amounts to two attributes, @rel and href:

<a rel="license" href="http://creativecommons.org/licenses/by/2.0/">cc by 2.0</a>

RDFa lets us take that little 'package' (or 'microformat') and use it anywhere that we can set a 'subject'. Of course normally the subject is the current document, which is why the rel-license microformat looks just the same to an RDFa parser as it does to a Microformats parser. But RDFa allows this 'package' to be used with the @src attribute, giving us the following possibilities:

<img src="my-picture.png" rel="license" href="http://creativecommons.org/licenses/by/2.0/" />

<object src="my-video.mov" rel="license" href="http://creativecommons.org/licenses/by/2.0/" /></object>

As you can see, this straightforwardly solves the problem that rel-license is presenting to those who want to put many items in a page, so all that would be required to take this solution into the Microformats world would be to slightly extend the rel-license microformat to allow the src attribute as a subject.

RDFa or Microformats?

This example illustrates that if we see RDFa and Microformats as playing slightly different roles, then there is no need to take an 'either/or' approach to the two techniques.

Since RDFa is a general-purpose syntax that is capable of supporting any vocabulary that anyone comes up with, now or in the future, then we are often tempted to talk about the general rules. But for new users who are looking to achieve a specific goal, seeing documentation about a specific usage pattern (such as the one I've shown here with licensing), will almost certainly be more useful to begin with.

But that is what Microformats is all about--documenting common usage patterns in such a way that people can re-use them in their own mark-up. So what would be wrong with enhancing rel-license to allow the use of @src?

Note that people wanting to parse this microformat could either create a specific parser in the way that they need to do now for other microformats, or they could use a general-purpose RDFa parser.

The key thing is that the mark-up is exactly the same, whether you arrived at the use of @rel="license" via the Microformats page, or an RDFa page.

And that seems to me to be a good result all round, for the authors, for the programmers, for the search engines, and especially for those of us who want to see a more dynamic, usable, and semantic, web.

First steps in RDFa: Creating a FOAF profile

Now that the RDFa syntax document is in last call, and people like Yahoo! are starting to index the data, it's worth putting more of your own data into your web-pages, using RDFa. A simple place to start is to modify your home-page or blog profile so that it includes FOAF information.

FOAF

If you're not familiar with FOAF, or Friend-of-a-friend, it's a set of terms that can be used to describe people, organisations, and their relationships to each other. For example, we can mark up our names, point to our home-pages, indicate the companies and projects we work on (and point to their home-pages), and so on.

Since this vocabulary is gaining in popularity, and since RDFa allows us to use any vocabulary we like without having to re-write it (or ask anyone), we'll use FOAF via RDFa to mark up our pages. We won't use every part the vocabulary, so if you want to find further properties, or more detail on the properties used below, look at the full FOAF specification.

Creating a person

The first thing we need to do is to create a person object that will hold our information. This is done using the RDFa typeof attribute, which is much like @class in HTML. The type of the object we want to add is a Person and since 'person' comes from the FOAF vocabulary, we write it like this:

<html xmlns:foaf="http://xmlns.com/foaf/0.1/">
  <head>
    <title>Mark Birbeck's profile</title>
  </head>
  <body>
    <div typeof="foaf:Person">
      ...
    </div>
  </body>
</html>

Now we're ready to add our personal information to this block.

Adding personal information

The FOAF vocabulary is packed with useful properties that we can set, so let's start with some basics such as our name and the URL for our blog.

We can add our name using the foaf:name property, which is set via the new RDFa property attribute:

    <div typeof="foaf:Person">
      <span property="foaf:name">Mark Birbeck</span>
    </div>

Our blog is indicated using the foaf:weblog property. However, unlike foaf:name which is simply a string of text, the item we're going to refer to is a URL, so we must use the HTML rel attribute instead of @property:

    <div typeof="foaf:Person">
      <span property="foaf:name">Mark Birbeck</span>
      <a rel="foaf:weblog" href="http://internet-apps.blogspot.com/">XForms and Internet Applications</a>
    </div>

Creating a profile

We now have a person with some properties about them, but in FOAF terms that's slightly different to having a profile about the person. If that seems a little subtle, it is, but what it amounts to is that the document that contains information about me, is not actually me. In many situations it won't appear to make any difference, but unfortunately it can cause a lot of problems. For example, if the document is used to represent both my profile and me at the same time, what would be the result of adding some information about when the document was created? How do we know whether the information is indicating when I was born, or when the document was?

FOAF allows us to prise these two things apart with the foaf:primaryTopic property, so we're going to use this to say that the main subject-matter of our profile (a web document) is me (a person):

<html xmlns:foaf="http://xmlns.com/foaf/0.1/">
  <head>
    <title>Mark Birbeck's profile</title>
    <link rel="foaf:primaryTopic" href="#me" />
  </head>
  <body>
    <div about="#me" typeof="foaf:Person">
      <span property="foaf:name">Mark Birbeck</span>
      <a rel="foaf:weblog" href="http://internet-apps.blogspot.com/">XForms and Internet Applications</a>
    </div>
  </body>
</html>

Whilst we're here, it might be useful to use the foaf:maker property to indicate who created the profile; in this case it's the same as the subject of the profile (i.e., the person who the profile is about), so it's easily set as follows:

<html xmlns:foaf="http://xmlns.com/foaf/0.1/">
  <head>
    <title>Mark Birbeck's profile</title>
    <link rel="foaf:primaryTopic foaf:maker" href="#me" />    
  </head>
  <body>
    <div about="#me" typeof="foaf:Person">
      <span property="foaf:name">Mark Birbeck</span>
      <a rel="foaf:weblog" href="http://internet-apps.blogspot.com/">XForms and Internet Applications</a>
    </div>
  </body>
</html>

Now we can read this whole thing as follows:

• we have a person that is identified as #me;
• this person has a name of "Mark Birbeck";
• this person has a blog at <http://internet-apps.blogspot.com/>;
• this person is the creator of the current document;
• this person is the main subject of the current document.

Adding friends and colleagues

Now that we have our basic framework in place, it's pretty easy to drop more and more FOAF properties in. Perhaps the most commonly used is foaf:knows which is used to indicate the people that you know. Since the target of this property is once again a URL, we'll need to use the HTML rel attribute again:

    <div about="#me" typeof="foaf:Person">
      <span property="foaf:name">Mark Birbeck</span>
      <a rel="foaf:weblog" href="http://internet-apps.blogspot.com/">XForms and Internet Applications</a>
      <a rel="foaf:knows" href="http://www.w3.org/People/Ivan/#me">Ivan Herman</a>
    </div>

Adding a picture

The FOAF vocabulary also allows us to indicate pictures that we appear in, and pictures that we might want others to use to represent us. To set a picture of yourself that other software might use to represent you, use the foaf:img property:

    <div about="#me" typeof="foaf:Person">
      <span property="foaf:name">Mark Birbeck</span>
      <a rel="foaf:weblog" href="http://internet-apps.blogspot.com/">XForms and Internet Applications</a>
      <a rel="foaf:knows" href="http://www.w3.org/People/Ivan/#me">Ivan Herman</a>
      <span rel="foaf:img">
        <img src="http://www.formsplayer.com/files/pictures/picture-11.jpg" alt="Picture of Mark Birbeck" />
      </span>
    </div>

Linking to a Twitter account

The final illustration we'll show is how to use the FOAF vocabulary to point to your Twitter account, which will make it easy to build tools that will allow people to follow you with one click. The first thing to do is create a relationship called foaf:holdsAccount, which will connect our 'person object' with an online account object. To connect two objects we use the HTML rel attribute again:

    <div about="#me" typeof="foaf:Person">
      <span property="foaf:name">Mark Birbeck</span>
      <a rel="foaf:weblog" href="http://internet-apps.blogspot.com/">XForms and Internet Applications</a>
      <a rel="foaf:knows" href="http://www.w3.org/People/Ivan/#me">Ivan Herman</a>
      <span rel="foaf:img">
        <img src="http://www.formsplayer.com/files/pictures/picture-11.jpg" alt="Picture of Mark Birbeck" />
      </span>
      <span rel="foaf:holdsAccount">
        ...
      </span>
    </div>

Next we create an object of type foaf:OnlineAccount, in exactly the same way that we did when creating a person earlier:

      <span rel="foaf:holdsAccount">
        <span typeof="foaf:OnlineAccount">
          ...
        </span>
      </span>

Finally, we indicate that the particular type of account we're dealing with is a Twitter account (using foaf:accountServiceHomepage), and also provide our account name (using foaf:accountName):

      <span rel="foaf:holdsAccount">
        <span typeof="foaf:OnlineAccount">
          <a rel="foaf:accountServiceHomepage" href="http://twitter.com/">Twitter</a>
          <span property="foaf:accountName">markbirbeck</span>
        </span>
      </span>

Human and machine-readable

Everything we've marked up so far is human and machine readable, but the layout is not great for a human. Although the links to the blog will work, and the text will show names and accounts correctly, there is no context information. However, additional mark-up can be placed in the document, and as long as it is outside of the scope of the RDFa attributes it won't be classed as metadata. For example:

<html xmlns:foaf="http://xmlns.com/foaf/0.1/">
  <head>
    <title>Mark Birbeck's profile</title>
    <link rel="foaf:primaryTopic foaf:maker" href="#me" />    
  </head>
  <body>
    <div about="#me" typeof="foaf:Person">
      <span property="foaf:name">Mark Birbeck</span> writes a blog called
      <a rel="foaf:weblog" href="http://internet-apps.blogspot.com/">XForms and Internet Applications</a>.
      He knows
      <a rel="foaf:knows" href="http://www.w3.org/People/Ivan/#me">Ivan Herman</a>.
      <span rel="foaf:img">
        <img src="http://www.formsplayer.com/files/pictures/picture-11.jpg" alt="Picture of Mark Birbeck" />
      </span>

      His inane comments are available on his
      <span rel="foaf:holdsAccount">
        <span typeof="foaf:OnlineAccount">
          <a rel="foaf:accountServiceHomepage" href="http://twitter.com/">Twitter</a>
          account. His ID is '
          <span property="foaf:accountName">markbirbeck</span>'.
        </span>
      </span>

    </div>
  </body>
</html>

The whole shebang

If you want to use the mark-up as a template, then here is everything that we've seen, above. Just replace the values in red with your own details, add any human-readable text you want around it, and you are off and running:

<html xmlns:foaf="http://xmlns.com/foaf/0.1/">
  <head>
    <title>Mark Birbeck's profile</title>
    <link rel="foaf:primaryTopic foaf:maker" href="#me" />    
  </head>
  <body>
    <div about="#me" typeof="foaf:Person">
      <span property="foaf:name">Mark Birbeck</span>
      <a rel="foaf:weblog" href="http://internet-apps.blogspot.com/">XForms and Internet Applications</a>
      <a rel="foaf:knows" href="http://www.w3.org/People/Ivan/#me">Ivan Herman</a>
      <span rel="foaf:img">
        <img src="http://www.formsplayer.com/files/pictures/picture-11.jpg" alt="Picture of Mark Birbeck" />
      </span>

      <span rel="foaf:holdsAccount">
        <span typeof="foaf:OnlineAccount">
          <a rel="foaf:accountServiceHomepage" href="http://twitter.com/">Twitter</a>
          <span property="foaf:accountName">markbirbeck</span>
        </span>
      </span>

    </div>
  </body>
</html>

Publishing your FOAF page

Since our FOAF page is embedded into an HTML page, then you can publish your FOAF profile pretty much anywhere that you are able to publish HTML or XHTML. Unfortunately, I was not able to update my Blogger profile to include RDFa, so instead I've created a new blog page, which contains my profile. You'll see a few minor changes to the structure described above, to take into account that we're not creating the entire page, but essentially it's the same.

My Profile

The following is a sample profile, created using the techniques described in the post First steps in RDFa: Creating a FOAF profile:

Mark Birbeck has spent the last 7 years designing, building and thinking about a framework that enables dynamic user interfaces, driven by data content. Such a framework can dramatically increase programming productivity, and open up the world of application-building to many more people. Since he believes that the framework should be built on open standards, Mark is heavily involved in the W3C, as an invited expert with the XForms and XHTML 2 Working Groups, and also as the designer of RDFa.

His companies created the formsPlayer XForms processor, and Sidewinder, an open source, next-generation semantic web browser.

His blog focuses on building a new generation of internet applications, and a number of entries relate to Ajax, XForms, the semantic web, and the use of declarative mark-up. You can find him on Twitter, where his ID is markbirbeck.

RDFa is now in last call

After a great deal of effort by some very dedicated people, the RDFa in XHTML: Syntax and Processing document has entered last call. The full story is on the front-page of the W3C site, or at this permalink. The status of 'last call' means that as far as the people who have been involved in creating the spec are concerned, any technical problems that might have existed have been resolved. Now it's the turn of the wider community to make comments about the document, and even try to implement RDFa processors.

If you want to read more about RDFa, I've blogged a lot about RDFa, written an Introduction to RDFa, and presented on it. Bob DuCharme's Introducing RDFa is, as we'd expect from him, a great introduction, and Manu Sporny uses his vocal and artistic talents to provide RDFa Basics in video form.