Well, Talk Digger is not a traditional search engine even if you can search for keywords. In fact, the first goal of this search engine is to find who link to a specific webpage (your blog?).

What I would suggest you if you don’t find any malayalam blogs would be to put the url of one of these in talkdigger and then checking who links to them. Then, starting to browse them from a see blog.

for example, there are the people linking to your blog:

http://www.talkdigger.com/conversations/mallu-ungle.blogspot.com

I hope you will find what you are searching for.

Take care,

Fred

PLease let me know why a large majority of asian languages (wich includes 9 indian languages) are not part of your search results.

Cheers

Wow this is great! It has been fast

This is not a problem for the current list of profiles. In fact, the simple way for me is to get a list of web page, and then crawling them (PTSW already try to get elements from HTML document to RDF documents.

So if it would be possible for you to generate me this list in a list of URL separated by return carrier, I could start to crawl them tomorrow or over the weekend. That way I would only have to feed it with them. (in fact it would be a small agent that would read the list and then pinging it with them, anybody could do that).

Thanks!

Salutations,

Fred

A full dump into foaf format would be a bit more coding work, as the files are programatically generated at this time, not real on-disk files. I did however add tag in each profile page to aid with discoverability of the foaf files, so they could pretty easily be scraped/spidered starting with this directory page.

Yeah I was talking about the public semantic bank. I will take a deeper look to it later.

Yeah well, putting longwell over PTSW could be a good idea, but I have other plans that will roll out later in february (so keep checking this blog ). In fact, longwell is nice, but my mother, my friends, etc don’t like it: too complex, need to much knowledge to use it, etc. So this is the reason why I have other plans.

Thanks,

Take care,

Fred

Geting the RDF from the banks is trivial, just follow the alternate link. I don’t know about pinging PTSW on data change. Ask the SIMILE developers about that.

It might be very interesting if you would set up a semantic bank yourself. And promote publishing Piggy Bank data to your bank. This way you might get a lot of RDF data scraped over the Internet by Piggy Bank users for your project.

Even better would be if all the data indexed by PTSW would be accessible through Longwell faceted browser interface. This would be a semantic web killer app. But this is more of a dream than a near future project, I guess.

I am not sure that I will support OpenID with that version of the blog since I would have too much time to put in that and that there is no plugin for opened available for that version of b2Evolution

Vaclav Synacek: Yeah well, it is sure that if it is wrote in JS, then it couln’t be that useful to such a project

By the way, I was wondering without having the time to investigate further, is the data available in the Semantic Bank available to public? If so, it would be great if the data could be indexed by PTSW and if the semantic bank could ping it each time a new/updated file is indexed into the bank.

Danny: yeah you are right. But there are specialized database of information (normally in university) that filter all that information for them. It is sure that if you try to find all you information on Google, you will have to spend a lot of time filtering out all the crap

Take care all,

Salutations,

Fred

To Fred: I’m the original author of LinkedIn scraper. Orkut scraper was done by Ben Hyde. I’m not Orkut user and thus I have never seen Orkut scraper in action.

Answer to your question if these scrapers can be adapted to work outside of Piggy Bank is a bit more difficult. Generally it should be possible – they are standard JavaScripts with dependencies on Firefox XPath processor and some Piggy Bank specific RDF processing calls (not really hard to replace), it is open source and anybody is free to do it. However both scripts are meant to be personal tools and to work inside your browser, they only work after you log in to the specific social network. They only can scrape what you as a user are allowed to see. So in the case of LinkedIn: every user can see as far as his friends’ friends profiles, not further, so Piggy Bank can scrape also only this far. The users of LinkedIn agreed to share their profiles with direct friends and their friends, but nobody else – this policy is hard coded into the web interface of LinkedIn and Piggy Bank being a browser plugin cannot break out of these policy rules. No magic here. This is what I meant by while respecting contacts’ privacy.

Conclusion: while porting some scrapers outside of Piggy Bank environment might be possible and interesting thing to do, I don’t see a point in porting these particular two scrapers as they rely on logging in the social networks and thus will remain personal tools anyway.

To Dan Libby: videntity is an interesting beginning of a project. I’m not sure I got the whole idea, but I think of it as an ‘OpenID provider with XFN/FOAF file hosting and web hosting the myspece way’. This might be a more open alternative to myspace and the like, but I don’t see much to offer to the people having a proper web hosting where they can get OpenID, can make their own FOAF or XFN files. So I can’t wait for the ‘future plans’ being implemented.