I had a dream for the Semantic Web

Frederick Giasson

 

A year ago I had big ambitions for Talk Digger. At that time, I was dreaming that Talk Digger could help the Semantic Web to develop, to exist, to be used by thousands of people without them even knowing it.       
      

You have to know that at that time I hadn’t the vision clear enough, the knowledge and the resources to make such a dream reality. But slowly I directed me efforts toward that vision, that goal, hoping it could lead to something interesting. Opportunities after opportunities my vision became clearer, my knowledge of the subject evolved and the resources increased. I developed a new version of Talk Digger that broadcast its content in RDF using some specialized ontologies such as SIOC and FOAF. I developed a service called Ping the Semantic Web that aggregates and exports lists of semantic web documents (RDF) to anyone who request them.

Today I came one step closer to reach my goal: I make Talk Digger pinging Ping the Semantic Web each time a new user is created or a new conversation is started or updated on Talk Digger.

It probably doesn’t mean much for most people; however it means much for me.

I created a service that generates content from many different sources (traditional search engines results, users’ interactions with the system [creating comments, following conversations, creating links with other users, etc.], etc). I created a service that aggregate semantic web documents from around the Web to export them to any developers that wish to do something with them. Finally I make these two services interacting together.

What it means? It means that I created a prototype infrastructure for what I consider to be the first step toward the semantic web: creating semantic web formatted documents and making them freely and easily accessible to other web services and software agents; and all that using live and real data from “traditional web” resources and normal web users.

So that is it. Talk Digger documents are now living in the same world as other semantic web documents from around the Web. One developer can have access to all these documents from the same source. Talk Digger’s documents, as long as all others, are multiplexed by Ping the Semantic Web and have the possibility to live in a full set of different incarnations (different user interfaces and different data manipulations).

This is the dream I had.

Technorati: | | | | | | | | | |

How Talk Digger should evolve in the next months

Frederick Giasson

 

Talk Digger will slowly roll out. What it needs if more content, more users and more interaction (relation between users, comments, conversation pages crawling by other search engines like Google, MSN Search, Yahoo! (it already started slowly). Then more patterns would emerge, conversations will build up, and people will get in contact.       

However critics of Talk Digger are right: it is not perfect; it lacks indexed data and few bad results popup from time to time into conversations.

 

Validating the vision

Critics are right, but at the same time people are slowly validating the vision I had, the seed idea underlying the current implementation of Talk Digger.

When I read other people articles about this new service, I find that they know what it is all about, how it could help them in their day-to-day work and ultimately how it could be enhanced to meet their specific needs.

 

Current state of the system

I started to check how people were using the new features of this new system. As expected I have far more page views with this new version than the previous one. People search, browse and track conversations.

One thing that surprised me is that far more people than I initially thought were using the “Related conversation” tab of each Conversation Page.

I thought that I was alone, but it seems that many people use it not only to find out the relations between conversations (so relations between web pages, and ultimately people) but also to find new and interesting stuff from a starting point. So the process of browsing conversations using their relations is becoming the process of finding new and interesting stuff.

 

Improving the system

Considering that the vision I had of the system is slowly validated by other people’s reviews, a lot of stuff will come in the next months.

 

Crawling for more data

One of the biggest update is also the less time consuming (in the point of view of development time): indexing more and more data into the system (so crawling more and more conversations). It will be done by itself, as time go, with users’ searches and tracks.

 

Parsing out bad results

Another big improvement of the system will be to parse out bad results (irrelevant links from a web page to another web page (conversation)). It will enable the system to check the context of each link, considering their releventness, rating them accordingly and ultimately refusing to index it into the system if it is too irrelevant.

 

Automatic conversation topics finder (auto-tagging)

Many users are searching for tags. The problem is that there are few considering that the system is young, so no many conversations are “tracked” by users (because tags came from tags term used by users when they start tracking a conversation).

So the system could eventually extract conversation topics. That way, the “search tag” feature could become a “search by topics”.

Ultimately, these topics could use to cluster search results and find new relationship between conversations (relationships via topics instead of document links).

 

The future

This is only a part of the future; other ideas are incubated in my mind. It is just the beginning as we know it. Much more is to come…

Technorati: | | | | | | | | |

New Talk Digger website now Online

Frederick Giasson

 

    I talked a lot about it in the past few months, now it comes: the new generation of Talk Digger is now publicly available. I just upgraded it from closed alpha testing to public beta testing. I will monitor it and fix bugs as they will come in the next days and weeks. This version is much more stable than one month ago, but is probably less than in a month. I choose to make it public because I needed more people using it, so there we are.

Use it, play with it and please report anything that doesn’t seem right here.

I don’t want to talk much about it on my blog right now. What I want is looking at how people will use it, what they have to say, etc. So I open this thread to get comments from users (via comments on this blog post, email or Skype), talk with them, knowing their feelings about it, etc.

Hope you like what you will see.

Technorati: | | | | | |

Visualizing Web conversations using Talk Digger

Frederick Giasson

In this article, I will talk about the recent developments with the alpha version of Talk Digger and how it could be use to visualize the interactions between conversations tracked by it.

 

Recent developments

Yesterday I started to crawl most of the URLs submitted to Talk Digger in the past 6 months and indexing all the results in its new database.

Right now Talk Digger is tracking about 2500 URLs (so it has about 2500 conversations), and it indexed about 80 000 sources (other web pages linking to these 2500 conversations).

These numbers are not big, but the preliminary results are quite impressive (in my humble opinion). In fact, each time new URLs were tracked, new conversations was created, new sources was indexed, I discovered new ways to use it, to discover new stuff, to visualize relations between the data, etc.: the patterns were starting to emerge.

 

Visualizing interactions between Web conversations

In only 30 minutes of conversation browsing, I noticed 7 interesting use cases (patterns) in the system. I will present all of them by describing what is happening with each of them.

I added two visualization tools in the right sidebar of each conversation page.

 

 

The first tool

The first tool will help the users to answer to these two questions:

  • What are the other conversations that are talking about the current one?
  • What are the conversations the current one is talking about?

 

 

The current conversation is the one in light-blue, in the middle of the panel: “Talk Digger: find, follow and join discussions evolving on the Internet”.

From there, I know that the “Talk Digger: find, follow and join discussions evolving on the Internet” conversation is talking (in relation) with the other conversation “Frédérick Giasson – Computer scientist, software developer and consultant”.

It makes sense considering that I am the creator of Talk Digger and that the conversation “Frédérick Giasson – Computer scientist, software developer and consultant” is created by the URL of my personal web page.

I can also see that the conversations: “3spots”, “Library clips”, “ Digg Tools”, “decor8”, are also in relation with the current one.

That way, I can easily visualize the relationship between the conversations tracked by Talk Digger.

 

The second tool

The second tool will help the users to know what are the different conversations tracked by Talk Digger that came from the same source (URL).

 

 

From this panel, I know that Talk Digger is tracking two other conversations closely related to the current one: “Talk Digger Tools: Bookmarklet” and “Talk Digger Tour: Use the bookmarklet”.

In reality, these two other conversations are two different pages from a same domain name: talkdigger.com

Okay, now it is the time to check at the use cases to understand how these two tools can be used.

 

Use case #1: A normal blog or personal webpage.

This is the case of a conversation evolving around a single blog (or personal web page) and its interactions with other conversations:

 

 

In this example, the current conversation is the one of my personal web page.

What is interesting here is that we can see how it relates to itself. We can see that from my main page, I link to two other pages that have their conversations tracked by Talk Digger.

Also, I see that “jotsheet – blog o’ tom Sherman” also has a relation with me. In fact, tom Sherman is a old user of Talk Digger and talked about it in many of his blog posts.

 

 

I can also see other pages, from the same domain name, which has a conversation tracked by Talk Digger.

The difference between these results and the above ones is that they are not necessarily linking together (in opposition to the above relations).

 

Use case #2: Discovering the relation between a web page and its blog

 

In this example, I found the relation between a normal website (Library Law) and its blog (LibraryLaw Blog). What is interesting is that if you go on the Library Law’s web site, its blog is not clearly displayed. However, the relation between the two is clearly apparent.

 

Use case #3: Topic specific blogs and web sites.

Another interesting pattern is the one created by topic-specific blogs and web sites.

 

 

In this example, I used the Micro Persuasion blog wrote by Steve Rubel. This blog is focus on Web 2.0 news. As you can see, the “Micro Persuasion” conversation is relation (talk about) the conversations of other Web 2.0 services like “del.icio.us”, “Rollyo” and “Netvibes”.

So the relations here are topic centered.

 

Use case #4: Egocentric blogger.

This use case is fascinating because it shows how someone can relate to itself.

 

 

In this example, Robert Sanzalone, the writer behind the Pacific IT blog started track conversations for many of his blog posts. That way, we can easily visualize of one post is relating with the others.

 

Use case #5: Who cares about my photos?

Some people also care about what people say about their photo.

 

 

If we check the conversation evolving around nattu’s Flikr photo album, we will see two things:

  1. That the conversation created by this photo album is in relation with another conversation tracked by Talk Digger.
  2. That there is many other people that care about the conversation evolving around other people’s photo album.

 

Use case #6: In the news

Other people like to know what is the conversation evolving around specific pieces of news.

 

 

This is really interesting. We have a piece of news from ZDNet called “The new meaning of programming”. We instantly know that it relate with another conversation called “SocialNets & The Power of The URL”.

We also know that later, other pieces of news talked about it: “Mark Cuban is Wrong”, etc.

This is really interesting to find out how news could relate one between the other.

 

Use case #7: Online communities’ users.

Other people like to know the conversation evolving around their online persona present on online communities’ web site like MySpaces and LiveJournal.

 

 

In this example, we want to see the conversation about the user “2220s” on MySpaces. As we can see, 22-20s’s LiveJournal is talking about him.

We can also see a list of conversations evolving around many other MySpaces users’ page.

 

Conclusion

As we saw, depending on the source (URL), many different relationship patterns can emerge from Talk Digger’s conversations.

These preliminary results are quite exciting considering that I just started to crawl URLs since yesterday. I think the current infrastructure I developed in the past months is promising, the next steps is to continue crawling URLs and to get users using it.

 

Subscribe to the Alpha version of Talk Digger

If you would like to test these features, you can always subscribe for a user account. The next round of account created is planned for mid-August.

Technorati: | | | | | | | |

Talk Digger Alpha’s start

Frederick Giasson
Yesterday I created the first 50 accounts of the alpha phase of the next Generation of Talk Digger. 50 dedicated users started to use it, test it, report bugs and enhancements.

      

As expected, some bugs have been discovered; some parts of the user interface had to be re-worked, etc.

However, this is what is great with the crowd: I enhanced Talk Digger, on all aspects, more in 2 days than I had alone in 4 weeks. I am slowly changing all the little things that could possibly irritate users. It is a chance that I had a good starting set of dedicated users to help me.

Really, I am more than happy with the advancements!

In any case, I have to take care not to put my head in the sand because there are so many things to do: it is far than perfect (could it be? I doubt… nothing is perfect in this World!)

I just opened the subscription form for the second round of alpha user accounts. So if you haven’t been able to subscribe for the first round, you only have to subscribe for the second one that should take place by mid-August.

Technorati: | | | | | | | | | |