Winnipeg City’s NOW [Data] Portal

The Winnipeg City’s NOW (Neighbourhoods Of Winnipeg) Portal is an initiative to create a complete neighbourhood web portal for its citizens. At the core of the project we have a set of about 47 fully linked, integrated and structured datasets of things of interests to Winnipegers. The focal point of the portal is Winnipeg’s 236 neighbourhoods, which define the main structure of the portal. The portal has six main sections: topics of interests, maps, history, census, images and economic development. The portal is meant to be used by citizens to find things of interest in their neibourhood, to learn their history, to see the images of the things of interest, to find tools to help economic development, etc.

The NOW portal is not new; Structured Dynamics was also its main technical contractor for its first release in 2013. However we just finished to help Winnipeg City’s NOW team to migrate their older NOW portal from OSF 1.x to OSF 3.x and from Drupal 6 to Drupal 7; we also trained them on the new system. Major improvements accompany this upgrade, but the user interface design is essentially the same.

The first thing I will do is to introduce each major section of the portal and I will explain the main features of each. Then I will discuss the new improvements of the portal.

[extoc]

Continue reading “Winnipeg City’s NOW [Data] Portal”

Open Semantic Framework 3.1 Released

Structured Dynamics is happy to announce the immediate availability of the Open Semantic Framework version 3.1. This new version includes a set of fixes to different components of the framework in the last few months. The biggest change is deployment of OSF using Virtuoso Open Source version 7.1.0. triple_120

We also created a new API for Clojure developers called: clj-osf. Finally we created a new Open Semantic Framework web portal that better describes the project and is hopefully easier to use and more modern.

Quick Introduction to the Open Semantic Framework

What is the Open Semantic Framework?

The Open Semantic Framework (OSF) is an integrated software stack using semantic technologies for knowledge management. It has a layered architecture that combines existing open source software with additional open source components. OSF is designed as an integrated content platform accessible via the Web, which provides needed knowledge management capabilities to enterprises. OSF is made available under the Apache 2 license.

OSF can integrate and manage all types of content – unstructured documents, semi-structured files, spreadsheets, and structured databases – using a variety of best-of-breed data indexing and management engines. All external content is converted to the canonical RDF data model, enabling common tools and methods for tagging and managing all content. Ontologies provide the schema and common vocabularies for integrating across diverse datasets. These capabilities can be layered over existing information assets for unprecedented levels of integration and connectivity. All information within OSF may be powerfully searched and faceted, with results datasets available for export in a variety of formats and as linked data.

A new Open Semantic Framework website

The OSF 3.1 release also triggered the creation of a new website for the project. We wanted something leaner and more modern and that is what I think we delivered. We also reworked the content, we wrote about a series of usecases 1 2 3 4 5 6 and we better aggregated and presented information for each web service endpoint.

A new OSF sandbox

We also created an OSF sandbox where people can test each web service endpoint and test how each functionality works. All of the web services are open to users. The sandbox is not meant to be stable considering that everybody have access to all endpoints. However, the sandbox server will be recreated on a periodic basis. If the sandbox is totally broken and users experiment issues, they can always request a re-creation of the server directly on the OSF mailing list.

Each of the web service pages on the new OSF portal has a Sandbox section where you see some code examples of how to use the endpoint and how to send requests to the sandbox. Here are the instructions to use the sandbox server.

A new OSF API for Clojure: clj-osf

The OSF release 3.1 also includes a new API for Clojure developers: clj-osf.

clj-osf is a Domain Specific Language (DSL) that should lower the threshold to use the Open Semantic Framework.

To use the DSL, you only have to configure your application to use a specific OSF endpoint. Here is an example of how to do this for the Sandbox server:

[cc lang=”lisp”]
[raw]
;; Define the OSF Sandbox credentials (or your own):
(require ‘[clj-osf.core :as osf])

(osf/defosf osf-test-endpoint {:protocol :http
:domain “sandbox.opensemanticframework.org”
:api-key “EDC33DA4D977CFDF7B90545565E07324”
:app-id “administer”})

(osf/defuser osf-test-user {:uri “http://sandbox.opensemanticframework.org/wsf/users/admin”})
[/raw]
[/cc]

Then you can send simple OSF web service queries. Here is an example that sends a search query to return records of type foaf:Person that also match the keyword “bob”:

[cc lang=”lisp”]
[raw]
(require ‘[clj-osf.search :as search])

(search/search
(search/query “bob”)
(search/type-filters [“http://xmlns.com/foaf/0.1/Person”]))
[/raw]
[/cc]

A complete set of clj-osf examples is available on the OSF wiki.

Finally the complete clj-osf DSL documentation is available here.

A community effort

This new release of the OSF Installer is another effort of the growing Open Semantic Framework community. The upgrade of the installer to deploy the OSF stack using Virtuoso Open Source version 7.1.0 has been created by William (Bill) Anderson.

Deploying a new OSF 3.1 Server

Using the OSF Installer

OSF 3.1 can easily be deployed on a Ubuntu 14.04 LTS server using the osf-installer application. It can easily be done by executing the following commands in your terminal:

[cc lang=”bash”]
[raw]
mkdir -p /usr/share/osf-installer/

cd /usr/share/osf-installer/

wget https://raw.github.com/structureddynamics/Open-Semantic-Framework-Installer/3.1/install.sh

chmod 755 install.sh

./install.sh

./osf-installer –install-osf -v
[/raw]
[/cc]

Using a Amazon AMI

If you are an Amazon AWS user, you also have access to a free AMI that you can use to create your own OSF instance. The full documentation for using the OSF AMI is available here.

Upgrading Existing Installations

Existing OSF installations can be upgraded using the OSF Installer. However, note that the upgrade won’t deploy Virtuoso Open Source 7.1.0 for you. All the code will be upgraded, but Virtuoso will remain the version you were last using on your instance. All the code of OSF 3.1 is compatible with previous versions of Virtuoso, but you won’t benefit the latest improvements to Virtuoso (in terms of performances) and its latest SPARQL 1.1 implementations. If you want to upgrade Virtuoso to version 7.1.0 on an existing OSF instance you will have to do this by hands.

To upgrade the OSF codebase, the first thing is to upgrade the installer itself:

[cc lang=”bash”]
[raw]
# Upgrade the OSF Installer
./usr/share/osf-installer/upgrade.sh
[/raw]
[/cc]

Then you can upgrade the components using the following commands:

[cc lang=”bash”]
[raw]
# Upgrade the OSF Web Services
./usr/share/osf-installer/osf –upgrade-osf-web-services=”3.1.0″

# Upgrade the OSF WS PHP API
./usr/share/osf-installer/osf –upgrade-osf-ws-php-api=”3.1.0″

# Upgrade the OSF Tests Suites
./usr/share/osf-installer/osf –upgrade-osf-tests-suites=”3.1.0″

# Upgrade the Datasets Management Tool
./usr/share/osf-installer/osf –upgrade-osf-datasets-management-tool=”3.1.0″

# Upgrade the Data Validator Tool
./usr/share/osf-installer/osf –upgrade-osf-data-validator-tool=”3.1.0″
[/raw]
[/cc]

Open Semantic Framework 3.0.1 Released

I am happy to announce the immediate availability of the Open Semantic Framework version 3.0.1. This new version includes a set of fixes to different components of the framework in the last few months.The biggest addition is the new OSF Installer which will deploy OSF on Ubuntu LTS 14.04 servers. triple_120

A Community Effort

This new release of the OSF Installer is an effort of the growing Open Semantic Framework community. The upgrade of the installer to deploy the OSF stack on the lastest Ubuntu Long Term Support (LTS) version 14.04 has been created by William (Bill) Anderson.

Samar Acharya also suggested to decouple the PHP5 Debian packages from the core OSF Installer repository to cope with the support of future version of Ubuntu or other Linux distributions. This led to the creation of the new OSF-Installer-Ext repository, which is only used to host these distribution specific files like the PHP5 Debian files.

Upgrading Existing Installations

Existing OSF installations can be upgraded using the OSF Installer. The first thing is to upgrade the installer itself:

[cc lang=”bash”]
[raw]
# Upgrade the OSF Installer
./usr/share/osf-installer/upgrade.sh
[/raw]
[/cc]

Then you can upgrade the components using the following commands:

[cc lang=”bash”]
[raw]
# Upgrade the OSF Web Services
./usr/share/osf-installer/osf –upgrade-osf-web-services=”3.0.1″

# Upgrade the OSF WS PHP API
./usr/share/osf-installer/osf –upgrade-osf-ws-php-api=”3.0.1″

# Upgrade the OSF Tests Suites
./usr/share/osf-installer/osf –upgrade-osf-tests-suites=”3.0.1″

# Upgrade the Datasets Management Tool
./usr/share/osf-installer/osf –upgrade-osf-datasets-management-tool=”3.0.1″

# Upgrade the Data Validator Tool
./usr/share/osf-installer/osf –upgrade-osf-data-validator-tool=”3.0.1″
[/raw]
[/cc]

Configuring and Using the OSF Search API (Screencast)

In this screencast, I will show you how you can leverage the semantic power of the OSF Search endpoint into Drupal using OSF for Drupal. You will see how you can configure the OSF SearchAPI module, how you can turn any property into a filtering facet and how you can display the facets into blocks.

Then I will briefly show you how you can create new search results templates and how the template selection works using type inference.

Finally I will show you how you can enable and disable inference in the search feature, and how you can leverage the semantic structure of your data to change the relevancy of the search results returned. You have all the leisure to boost different characteristics of your data to return more relevant results to your users.

 


tut_14_blog_400

Specifying Field Widgets for OSF Entities Fields (Screencast)

In this screencast, I will show you how you can use ontologies to specify the field types to use for the classes and properties we map into Drupal using OSF Entities mapping process. Once the field types will be configured for each Datatype property, I will run the mapping process to generate new fields that will use the configured field types. Once done, I will show you the impact of this configuration into the fields and fields instances that are being created into Drupal.

The second part of this screencast focus on the configuration of the field widgets that are being used by each field. Then I will update a few entities using the new forms. I will tell you how you can modify the form by re-ordering the fields, by changing their titles or other configuration options such as their cardinality.

OSF Entities supports the following 18 field types and 34 field widgets.

 


tut_13_blog_400