The Semantic Web landscape is changing

Frederick Giasson

 

Today I read two really interesting blog posts about RDF, ontologies and the Semantic Web. I’ll start with Daniel Lemire’s Do not ask me to be a keynote speaker on ontologies and inference engines. In his article, Daniel said:

 

“Before I become interested in anything that has to do with web ontologies, I need to be convinced that, at least, RDF is a useful idea. So, first take Tim Bray’s RDF challenge:

“To the first person or organization that presents me with an RDF-based app that I actually want to use on a regular basis (at least once per day), and which has the potential to spread virally, I hereby promise to sign over the domain name RDF.net.”

But see, the rdf.net domain name is still down. Tim Bray, who can be seen as one of the initiators and early promoters of RDF, is still waiting for a useful RDF application. So am I.”

 

The challenge is not an easy one, but I don’t think it is an impossible one. After all, “Things are only impossible until they’re not”. As we will see later in this article, things are changing and such an application could be possible in a somewhat near future. At least, me and many other people are working toward that goal.

 

Semantic Web based research isn’t working

After that, I read Ingrid Und Leo’s blog post: quote: semantic web based research isn’t working

 

“1. Researchers need to stop thinking of themselves as researchers and start thinking of themselves as implementors.

2. Research institutes need to join forces with emerging businesses looking to adopt semantic technology. This breaks the current model of business / research institute collaboration since startups do not have money to contribute to fund research, but tough noogies.

3. Researchers need to build their tools in real-world development environments, i.e. as modules for LAMP web-publishing tools such as Drupal and WordPress. They need to find more organizational partners to deploy their solutions. They need to do something other than build widgets.”

 

Update 18-09-2006: A much better answer from Harry Chen: Struggling with the Semantic Web

 

From researchers to implementers

At the best of my knowledge, I totally agree with both articles.

The proof that both RDF and web ontologies are useful is yet to be done.

The good news is that that RDF “researchers” are becoming more and more RDF “implementers”. If I base my observations on the SIOC ontology development team, I can see that all the improvements to the ontology are made after implementations of the ontology into blogging softwares like WordPress or in community portals like Talk Digger or ODS and also with its interaction with other ontologies like FOAF, DC, etc.

They also work hard to push SIOC’s adoption by content creators around the Web. Right now, they developed exporter plugins for blogging systems, I implemented the SIOC ontology in Talk Digger, OpenLinks implemented the SIOC ontology in their ODS solution, I created a pinging system to aggregate and export RDF files to crawlers and software agents, and much more stuff have been done as John Breslin enumerate on his blog.

So, as you can see, the biggest part of their work is not as researchers, but as implementers.

 

The landscape is changing.

The landscape of the Semantic Web is changing. We are currently at a crucial moment in the Semantic Web’s development: from an academic goodie to a commercial venture.

More and more web developers and companies start to see how all these technologies and techniques could increase the power of their software, their infrastructure, etc. The problem is that they hadn’t the data to play with. It was impossible for them to justify such an investment considering that there were not any structured data (RDF) available on the Web.

Everything is changing, and everything should explode… soon!

Everything is in place; we had to have a context in place before hoping to see the semantic web appearing on the Internet. We had to reach a critical mass of RDF data, so people would see the need to start developing applications using that structured data. But we also had to reach a critical mass of applications so people would see the need to export their content in RDF. You see the pattern? We need both to see both happening.

 

Early adopters

The only hope we have is to get a critical mass of early adopters, and it is what we are slowly reaching. The early adopters will create applications, ontologies and data. According to Swoogle, they already aggregated about 1.7 millions of RDF documents. Ping the Semantic Web reached 32 000 in less than a month, without crawling the Web as Swoogle does. All these stats and projects tell me that we can hope to see real semantic web applications in a near future.

 

The Semantic Web Revisited

After writing this article, I checked my list of things to read. An item I put in my list about two months ago has attracted my attention. Harry Chen wrote about the The Semantic Web Revisited article. Read this article to know at which milestone the Semantic Web currently is. It covers all the topics around the Semantic Web, and some of them, like the importance of early adopters to help the Semantic Web to be widely adopted, are in direct relation with what I said in that article.

Technorati: | | | | | |

New Talk Digger features: follow the activity of your social network

Frederick Giasson

 

I just released two new features that enhance the binding between a Talk Digger user and its friends. Users are now able to easily follow which conversation their friends are tracking and what they have to say about them.

 

 

If you would like to test this new feature, you only have to follow these five steps:

  1. Creating a user account (if it is not already done).
  2. Searching for friends on Talk Digger and including them in your friends list. A good way to search friends is to check who are tracking the conversations that interest you; to search for users that have the same interests as you; to check the friends of you friends by clicking on the “Friends” tab of your friend’s user page.
  3. Add them to your friends list.
  4. Check your profile page.
  5. Click on the “Friends’ Conversations” or the “Friends’ Comments” to see which conversations are tracked by your friends and what you friends have to say.

 

Bellow is the screenshot of the conversations tracked by my friends:

 

 

And bellow is the screenshot of the comments wrote by my friends on Talk Digger

 

 

The theory behind these two new features

The theory is simple: you will probably be interested in conversations tracked by your friends because you probably have the same interests in life.

 

Following conversations of interest in a social group

Now people can follow what a group of people are talking about, the conversations that are interesting them.

Personally I have interests in the Internet, the semantic web, books and writing. So I’ll search for people with the same interests, I’ll add them in my friends list, and then I’ll start to check what these people are tracking, and what they have to say vis-à-vis some conversations.

That way, I created a sort of “virtual” group of interests and then I’ll start to check what this group is interested in.

 

The next step: creating an infrastructure to handle these groups of interest

The next step will probably be to create features to handle these groups of interests. Talk Digger is not just about tracking conversations, it is also about linking people together and create an infrastructure to manage interest groups would be a natural add-on.

Technorati: | | | | | |

I had a dream for the Semantic Web

Frederick Giasson

 

A year ago I had big ambitions for Talk Digger. At that time, I was dreaming that Talk Digger could help the Semantic Web to develop, to exist, to be used by thousands of people without them even knowing it.       
      

You have to know that at that time I hadn’t the vision clear enough, the knowledge and the resources to make such a dream reality. But slowly I directed me efforts toward that vision, that goal, hoping it could lead to something interesting. Opportunities after opportunities my vision became clearer, my knowledge of the subject evolved and the resources increased. I developed a new version of Talk Digger that broadcast its content in RDF using some specialized ontologies such as SIOC and FOAF. I developed a service called Ping the Semantic Web that aggregates and exports lists of semantic web documents (RDF) to anyone who request them.

Today I came one step closer to reach my goal: I make Talk Digger pinging Ping the Semantic Web each time a new user is created or a new conversation is started or updated on Talk Digger.

It probably doesn’t mean much for most people; however it means much for me.

I created a service that generates content from many different sources (traditional search engines results, users’ interactions with the system [creating comments, following conversations, creating links with other users, etc.], etc). I created a service that aggregate semantic web documents from around the Web to export them to any developers that wish to do something with them. Finally I make these two services interacting together.

What it means? It means that I created a prototype infrastructure for what I consider to be the first step toward the semantic web: creating semantic web formatted documents and making them freely and easily accessible to other web services and software agents; and all that using live and real data from “traditional web” resources and normal web users.

So that is it. Talk Digger documents are now living in the same world as other semantic web documents from around the Web. One developer can have access to all these documents from the same source. Talk Digger’s documents, as long as all others, are multiplexed by Ping the Semantic Web and have the possibility to live in a full set of different incarnations (different user interfaces and different data manipulations).

This is the dream I had.

Technorati: | | | | | | | | | |

How Talk Digger should evolve in the next months

Frederick Giasson

 

Talk Digger will slowly roll out. What it needs if more content, more users and more interaction (relation between users, comments, conversation pages crawling by other search engines like Google, MSN Search, Yahoo! (it already started slowly). Then more patterns would emerge, conversations will build up, and people will get in contact.       

However critics of Talk Digger are right: it is not perfect; it lacks indexed data and few bad results popup from time to time into conversations.

 

Validating the vision

Critics are right, but at the same time people are slowly validating the vision I had, the seed idea underlying the current implementation of Talk Digger.

When I read other people articles about this new service, I find that they know what it is all about, how it could help them in their day-to-day work and ultimately how it could be enhanced to meet their specific needs.

 

Current state of the system

I started to check how people were using the new features of this new system. As expected I have far more page views with this new version than the previous one. People search, browse and track conversations.

One thing that surprised me is that far more people than I initially thought were using the “Related conversation” tab of each Conversation Page.

I thought that I was alone, but it seems that many people use it not only to find out the relations between conversations (so relations between web pages, and ultimately people) but also to find new and interesting stuff from a starting point. So the process of browsing conversations using their relations is becoming the process of finding new and interesting stuff.

 

Improving the system

Considering that the vision I had of the system is slowly validated by other people’s reviews, a lot of stuff will come in the next months.

 

Crawling for more data

One of the biggest update is also the less time consuming (in the point of view of development time): indexing more and more data into the system (so crawling more and more conversations). It will be done by itself, as time go, with users’ searches and tracks.

 

Parsing out bad results

Another big improvement of the system will be to parse out bad results (irrelevant links from a web page to another web page (conversation)). It will enable the system to check the context of each link, considering their releventness, rating them accordingly and ultimately refusing to index it into the system if it is too irrelevant.

 

Automatic conversation topics finder (auto-tagging)

Many users are searching for tags. The problem is that there are few considering that the system is young, so no many conversations are “tracked” by users (because tags came from tags term used by users when they start tracking a conversation).

So the system could eventually extract conversation topics. That way, the “search tag” feature could become a “search by topics”.

Ultimately, these topics could use to cluster search results and find new relationship between conversations (relationships via topics instead of document links).

 

The future

This is only a part of the future; other ideas are incubated in my mind. It is just the beginning as we know it. Much more is to come…

Technorati: | | | | | | | | |

New Talk Digger website now Online

Frederick Giasson

 

    I talked a lot about it in the past few months, now it comes: the new generation of Talk Digger is now publicly available. I just upgraded it from closed alpha testing to public beta testing. I will monitor it and fix bugs as they will come in the next days and weeks. This version is much more stable than one month ago, but is probably less than in a month. I choose to make it public because I needed more people using it, so there we are.

Use it, play with it and please report anything that doesn’t seem right here.

I don’t want to talk much about it on my blog right now. What I want is looking at how people will use it, what they have to say, etc. So I open this thread to get comments from users (via comments on this blog post, email or Skype), talk with them, knowing their feelings about it, etc.

Hope you like what you will see.

Technorati: | | | | | |