[quote post="833"]Not yet. We’re working on a new record structure that will have many more fields. You can see the almost finished schema at[/quote]

Nice! I just took a quick look at it and it seems that almost everything can be described using BIBO. Except for a couple of really specialized fields, everything is in place. In fact, the exercise we should do if getting some records using this new schema, and trying to describe them in rdf using bibo.

One thing I was surprised to see if the “contributions” field. are the “kind” of contributions limited, or it is an open field where people can write “anything” (any terms). These kind of things will have to be sorted out. But all in all everything is looking really good.

[quote post="833"]The upshot is that this is a multi-part activity, with some interlocking parts.[/quote]

Yeah sure. How we have to see that, I think, is as 3 sources of information; and we will be able to inter-link each of these sources together.

When do you think we could get some records with this new schema? And when are you planning to implement this new schema in OpenLibrary?

Thanks,

Take care,

Fred

Not yet. We’re working on a new record structure that will have many more fields. You can see the almost finished schema at http://demo.openlibrary.org:9021/file/tip/catalog/schema.py. Some of those fields are for internal processing only, however.

Also, we are talking here about at least three different things. The files that Simon downloaded are subject authority records. Those do not represent books, they represent subject headings. Then there is the LC classification schedule, which is also subject-oriented, but entirely different from the subjects in the subject authority records. And these two are very different from the bibliographic data in OL or in library catalogs. Those might have subject headings from the subject heading file and classifications from the LC classification schedules.

The upshot is that this is a multi-part activity, with some interlocking parts.

[quote post="833"]Way cool Ed!

So the obvious question is, what next? How are we going to get this up, with nice stable URIs? Is this something you are planning to host at the loc?[/quote]

Bruce: I talked a lot about that with Edd over IRC. I think that Edd first have to get his hands on the data and then to get the rights to publish it. Once he is able to do this; we should check if we can help him archive his goal.

In mean time, many things can be done with bibo openlibrary.

In fact, if I check meta-data related to the openlibrary, I only see a BISAC subject field. It is why I proposed to describe bisac in rdf in this blog post.

Once Edd would be ready, we could eventually link the bisca structure to the LCSH one. (normal linking of two datasets). And we could check with openlibrary if we could do something with that.

[quote post="833"]would it make sense to start a semweb for libraries discussion list where we could talk about stuff like this? Or is bibont ok? I wasn’t sure if we could discuss issues that didn’t relate specifically to that ontology there.[/quote]

Well Edd, I think that the BIBO mailing list is there for this In fact, bibo is not only a place where to develop a new ontology for bibliographic things; but a place to check for best practices and to develop such initiative!

So I would encourage you to start the conversation on the list!

[quote post="833"]As I said, the Open Library is thinking of doing both faceting and browsing, although at this point we’re only thinking of using the upper two levels (mostly single and double letters).[/quote]

Karen: great! However the facething and browsing is not only do over subject properties of the document resources. In fact, I really think that the first step is to map the openlibrary in rdf according to bibo. Once it is done, many projects will be able to spawn from this initiative: browsing interfaces; faceted interfaces; linkage with other datasets; extension of subject backbones (like lchs), etc.

So, would you be willing to start this project?

Taka care,

Fred

We do need a place for this discussion, because we are duplicating work . There’s a tab-delimited version of the LC outline at:

http://www.archive.org/details/LcClassificationA-z

and someone on code4lib has created it in a database format (I don’t know if it differs at all from this).

As you know, the real meat, and overwhelming complication of the “divide like” instructions, is in the full classification, which does not seem to be readily available in machine-readable form. I’m also not sure what we would do with that level of detail, so the next discussion I think has to be about how we see ourselves using this data. As I said, the Open Library is thinking of doing both faceting and browsing, although at this point we’re only thinking of using the upper two levels (mostly single and double letters). Those can’t be extracted algorithmically because there are some inconsistencies between notation and levels, but I create an un-authoritative version here:

http://www.archive.org/details/LcClassificationLevels1-2

I realize there are already several discussion lists where we talk about these matters, but I wonder…would it make sense to start a semweb for libraries discussion list where we could talk about stuff like this? Or is bibont ok? I wasn’t sure if we could discuss issues that didn’t relate specifically to that ontology there.

So the obvious question is, what next? How are we going to get this up, with nice stable URIs? Is this something you are planning to host at the loc?

It still could use some work. Primarily, it needs to use proper URIs. Since I worked on this I’ve become an employee of the Library of Congress, and I’ve run into other people there who are interested in making data sets like LCCO/SKOS available to the public. So I think the reports that LC will be able to participate in the emerging web of linked data aka the semantic-web. At least that is my sincere hope, and plan

[quote post="833"]It would be ideal if we could start with data that we already have, such as the data that Simon captured. Even a not very sophisticated transform would allow us to work with the data in an experimental way.[/quote]

All projects, even the more ambitious ones, have to start somewhere. Doing a first mapping between some of your data and bibo is certainly a good start. From there, we will be able to check if created graphs can be easily queried and browsed. For the browsing purposes, we will create a new bibo template within the Zitgist Browser (browser.zitgist.com). That way, converted data should be easily browsable (this new template will come along with a new version of the browser that will be released soon).

About the queriability of the generated rdf graphs, we will be able to put online triples stores with this information to the community so that people can query the data and test things.

[quote post="833"]The catch, as always, is that we at least want the identifiers to be stable and somewhat durable. I agree with Fred, however, that we have to accept that there may be more than one identifier for a resource,[/quote]

Well, are the openlibrary URLs unique? Then I think they do great URIs From there, we will be able to dereference URIs once we are set and that the OpenLibrary data will be available in rdf. This is a pure technical consideration.

[quote post="833"]especially over time, and that just means that we will have to manage equivalences and even “fuzzy” equivalences (”x” *might* be equivalent to “y”). That shouldn’t be a barrier.[/quote]

Exactly. It is why I personally dislike the use of the property owl:sameAs for that task (to use use of that property say that a resource is exactly the same as the other (considering some set theory consideration). Fuzzy relations can be described using other techniques, but I think we shouldn’t think about that stuff for now. Many other considerations on the table first.

[quote post="833"]Would it help to have a brief description of the characteristics of, say, the subject or name authorities data in MARC21 format? Or is that already obvious?[/quote]

Well, it could certainly help. In fact, I think that bibo can already, easily be used to map openlibrary’s schema in RDF. One easy way for me to proceed is to get a snapshot of the openlibrary db (postgre). That way, I would load it on our servers, I would create a rdf view (a rdf view is a view that takes relational data and map it, on-the-fly, into RDF according to some ontologies). The server used would be virtuoso (available in open-source if you like to try it). So, data migration and rdf view creation would be easy to do. This view enable us to query the relation data like if it would be native RDF data. So you can send SPARQL queries, and such. And then, the data can easily be converted in native rdf (xml or n3).

This could be an easy and somewhat quick way to start using an openlibrary (or whatever the document dataset) mapping into rdf according to bibo.

Take care,

Fred

There are various reasons why this is, but not the least of which is that as a US Federal agency LoC is required to do “cost recovery” for any service they provide to others. Some of us are trying to get grant money to start the process of converting some of the library data to a semantic web format since it seems obvious that LoC will not be able to do so on its own. It’s not been an easy sell to granters since they tend not to understand the underlying value of the semantic web formats. We’re still waiting to hear if we can proceed, but have had some disappointments.

It would be ideal if we could start with data that we already have, such as the data that Simon captured. Even a not very sophisticated transform would allow us to work with the data in an experimental way. The catch, as always, is that we at least want the identifiers to be stable and somewhat durable. I agree with Fred, however, that we have to accept that there may be more than one identifier for a resource, especially over time, and that just means that we will have to manage equivalences and even “fuzzy” equivalences (“x” *might* be equivalent to “y”). That shouldn’t be a barrier.

Would it help to have a brief description of the characteristics of, say, the subject or name authorities data in MARC21 format? Or is that already obvious?