{"id":1170,"date":"2010-11-29T14:06:36","date_gmt":"2010-11-29T18:06:36","guid":{"rendered":"http:\/\/fgiasson.com\/blog\/?p=1170"},"modified":"2011-09-17T10:04:37","modified_gmt":"2011-09-17T14:04:37","slug":"getting-data-out-of-mypeg-ca-using-structwsf-endpoints","status":"publish","type":"post","link":"https:\/\/fgiasson.com\/blog\/index.php\/2010\/11\/29\/getting-data-out-of-mypeg-ca-using-structwsf-endpoints\/","title":{"rendered":"Getting Data Out of MyPeg.ca using structWSF Endpoints"},"content":{"rendered":"\n\n\n
A few weeks ago I presented the new MyPeg.ca community indicators web portal for Winnipeg\u00e2\u20ac\u2122s citizens<\/a>. I explained how in MyPeg.ca<\/a> we leverage Structured Dynamics<\/a>\u00e2\u20ac\u2122 semantic technologies stack (aka<\/em>The Semantic Muffin). Today\u00e2\u20ac\u2122s blog post explains one facet of the project that shows how external agents (people, services, software, etc.) can interact with the system\u00e2\u20ac\u2122s indicator datasets using the structWSF web service endpoints.Since this post focuses only on data export, I suggest you read the structWSF Web Services Tutorial<\/a> for a complete overview of how the endpoints architecture works.<\/td>\n\n

\"Merging<\/a><\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

Two Main structWSF Characteristics: Accessibility & Management<\/h3>\n

structWSF<\/a> is a set of 22 web service endpoints that lets you integrate data from different sources, manage that integrated data, and publish it via different communication channels such as web pages, software applications, etc.<\/p>\n

Obviously, the main characteristic of this framework is that everything is a web service. This means that all functionality of the system can be accessed from anywhere on the Internet. However, this doesn\u00e2\u20ac\u2122t mean that everything is open like a snack-bar. In fact, there are two levels of accessibility: (1) access to the web service endpoint\u00e2\u20ac\u2122s URL, and (2) access to the content of datasets hosted on structWSF. Depending on the usecase, people could restrict the direct access to the web service endpoint(s) by properly configuring their web server, others could choose to let anyone access the endpoints, but would restrict the access to the dataset(s) hosted by structWSF. In case of MyPeg.ca, the sponsor chose to open the access to their web service endpoints and datasets.<\/p>\n

Just by surfing on the MyPeg.ca portal, you are already leveraging these endpoints in multiple different ways. First, each time you generate a browse<\/a> or a search<\/a> Web page, you are telling the web server to send multiple queries to different endpoints; then the web page\u00e2\u20ac\u2122s content will be populated with that information and presented to you. But, each time you click on an explorer<\/a> node, your web browser is also sending queries to exactly the same web service endpoints. So, in one case a PHP script acts to query the endpoints; and, in other cases, a Flash Semantic Component<\/a> does. Depending, all structWSF data can be accessed from quite different environments.<\/p>\n

The other main characteristic of structWSF is that any kind of data can be imported in, and exported out, of the system. structWSF leverages RDF (Resource Description Framework) as the canonical data format that can be used to express any other formats. It is because of the usage of RDF that structWSF can act as an effective ETL<\/a> (Extract, transform, load) system. Depending on the web service endpoint, the output formats currently supported by most of the endpoints are:<\/p>\n