Header image

Freebase: Then & Now

Posted by admin in Uncategorized

 Freebase Logo optimised.svg

Freebase is an open database of the world’s information, built by a global community and free for anyone to query, contribute to, and build applications on top of.

In order to enable that, Freebase provides a set of HTTP web services that your applications can interact with. While you don’t need any special software to perform the required HTTP requests, it is often useful to have language-specific libraries that try to simplify the effort required to connect to such web services.

This project provides one such library for the Java language.

JSON

One of the characteristics of the Freebase APIs is that they heavily rely on JSON, both for input and for output. Even the query language used to query information against Freebase (what is known as MQL and pronounced so that it rhymes with “mickel”) is a specific form of JSON.

Unfortunately, Java’s strongly typed nature and unlike other languages like Javascript and Python where it belongs as a first class citizen, makes it very unnatural to use JSON as a data format with the API.

This is why this library is built around the concept of a telescopic JSON API that is designed to make it easier not only to parse and serialize data in JSON formats, but also to construct, modify and manipulate JSON data directly inside code with as little syntax overhead as possible.

We suggest you to familiarize yourself with our telescopic JSON API before reading the examples below.

MQL

The most important tool at our disposal when dealing with Freebase is the MQL language. To learn more about it, point your browser as the very comprehensive documentation on the Freebase web site and follow up from there. You are also highly encouraged to try it out on the MQL Query Editor, a highly interactive page on the Freebase web site that helps you familiarize yourself with the query language and with the schemas used in the various data domains that Freebase contains.

Reading Data from Freebase

Let’s start by asking who is the director of the movie “Blade Runner”. This is achieved with the following MQL query

{
  "id":   null,
  "type": "/film/film",
  "name": "Blade Runner",
  "directed_by": [{
    "id":   null,
    "name": null
  }]
}

how do we execute that query from Java with this library? First, you need to make the jar available to your app. This is beyond the scope of this document, but if you use Maven or Ant+Ivy, we strongly suggest you to read how to directly integrate this library with your POM.

Once the library is made available to your classloader, we need to import the required classes

import com.freebase.api.Freebase;
import com.freebase.json.JSON;

import static com.freebase.json.JSON.o;
import static com.freebase.json.JSON.a;

The first import is the class that contains all the Freebase APIs, while the second is the class that represents the telescopic JSON API.

The two following static imports don’t change any behavior but add useful syntax sugar to our class and allows us to use the short hand form of object and array creation, respectively o() and a() instead of JSON.o() and JSON.a() saving us a lot of typing and making it easier to read our hand-crafted JSON.

Now that we have all the tools we need, the first action is to retrieve a Freebase instance to work on. This is achieved by calling a static constructor on the Freebase class, like this:

     Freebase freebase = Freebase.getFreebase();

This connects you with the main Freebase site at http://www.freebase.com/. Alternatively, you can ask to get access to the Freebase Sandbox, a scratchpad replica of the entire Freebase system (along with all the data and its underlying infrastructure) that is very useful to test writes and data loads. That is done like this:

      Freebase sandbox = Freebase.getFreebaseSandbox();

Both main and sandbox instances present the exact same APIs and react in the exact same way, the only difference is that in sandbox you should not be concerned if you write something bogus because it is wiped out every well. If you’re doing read-only activity, it’s probably better to use Freebase since it’s a faster system, but if you’re doing read-write activity and you’re not 100% sure of the results, we strongly suggest to use the sandbox.

Now we need to come up with the MQL query. There are two ways to do this: by parsing a string that encodes JSON (but this makes the burden of keeping it well formed on you) or use the telescopic JSON API.

Here is how the first would be:

String query_str = "{" +
  "'id':   null," +
  "'type': '/film/film'," +
  "'name': 'Blade Runner'," +
  "'directed_by': [{" +
    "'id':   null," +
    "'name': null" +
  "}]" +
"}".replace('\'', '"');
JSON query = JSON.parse();

Note how both Java and JSON use double quotes and instead of doing escaping all over the place and heavily sacrifice readability, we preferred using a runtime approach with the last replace() call. This is by no means required (or safe in case you have escaped single quotes inside), but it makes life a lot easier in practice.

Still, the above is cumbersome and error prone, mostly because it’s very easy to destroy the syntax integrity of the JSON object while working it with piece-meal with concatenated strings. A better approach is to use the JSON API directly, like this:

JSON query = o(
  "id", null,
  "type", "/film/film",
  "name", "Blade Runner",
  "directed_by", a(o(
    "id", null,
    "name", null
  ))
);

where o() builds a JSON object and a() builds a JSON array and all the various casting issues have been eliminated by the use of a single polymorphic class that can represent all JSON content.

Now that we have the query, we need to execute it against Freebase, this is done by calling the “mqlread” service, like this:

JSON result = freebase.mqlread(query);

Like most Freebase web services, they behave as JSON-in/JSON-out services and also wrap the result with an envelope that provides additional information from the HTTP protocol on how the request has been executed. To get to the meat of our result, we need to ask for the “result” object, then dig our way into the JSON result to find the name of the director, like this:

String director = result.get("result").get("directed_by").get(0).get("name").string();

NOTE: since movies can have more than one director (even if it’s not the case on this one), we have encapsulated the ‘directed_by’ clause with an array. This is why we have to have the extra “get(0)” in there to get the first director).

Writing Data to Freebase

The other side of the equation is obviously how to write data in Freebase or modify data that is already there. In order to achieve this, first you have to have a valid Freebase account, so go get one before you can continue.

After that the only difference between reading and writing (besides the shape of the query, of course) is that you have to authenticate yourself against the site. This is done by invoking sign-on methods on the Freebase class like this:

Freebase sandbox = Freebase.getFreebaseSandbox();
sandbox.sign_in(username, password);

which fills your current Freebase instance with authorization credentials that will be used later when invoking the commands that require authorization which are the “mqlwrite” (used to add relational data) and the “upload” (used to add binary and textual data) services.

When we publicly launched Freebase back in 2007, we thought of it as a “Wikipedia for structured data.” So it shouldn’t be surprising that we’ve been closely watching the Wikimedia Foundation’s project Wikidata[1] since it launched about two years ago. We believe strongly in a robust community-driven effort to collect and curate structured knowledge about the world, but we now think we can serve that goal best by supporting Wikidata — they’re growing fast, have an active community, and are better-suited to lead an open collaborative knowledge base.

So we’ve decided to help transfer the data in Freebase to Wikidata, and in mid-2015 we’ll wind down the Freebase service as a standalone project. Freebase has also supported developer access to the data, so before we retire it, we’ll launch a new API for entity search powered by Google’s Knowledge Graph.

Loading Freebase into Wikidata as-is wouldn’t meet the Wikidata community’s guidelines for citation and sourcing of facts — while a significant portion of the facts in Freebase came from Wikipedia itself, those facts were attributed to Wikipedia and not the actual original non-Wikipedia sources. So we’ll be launching a tool for Wikidata community members to match Freebase assertions to potential citations from either Google Search or our Knowledge Vault[2], so these individual facts can then be properly loaded to Wikidata. We believe this is the best first step we can take toward becoming a constructive participant in the Wikidata community, but we’ll look to continually evolve our role to support the goal of a comprehensive open database of common knowledge that anyone can use.

Here are the important dates to know:

Before the end of March 2015
- We’ll launch a Wikidata import review tool
- We’ll announce a transition plan for the Freebase Search API & Suggest Widget to a Knowledge Graph-based solution

March 31, 2015
- Freebase as a service will become read-only
- The website will no longer accept edits
- We’ll retire the MQL write API

June 30, 2015
- We’ll retire the Freebase website and APIs[3]
- The last Freebase data dump will remain available, but developers should check out the Wikidata dump[4]

The Knowledge Graph team at Google

[1] http://wikidata.org
[2] http://www.cs.cmu.edu/~nlao/publication/2014.kdd.pdf
[3] https://developers.google.com/freebase/v1/
[4] http://dumps.wikimedia.org/wikidatawiki/


About Author:  LissaCoffey

As a Google Apps for Education Trainer, Google Apps Certified Administrator, Allison is an international Google Summit featured speaker and global trainer with Apps Events. She is also a Virtual Instructor for NH’s Virtual Learning Academy, with MIT Technology in Education . She works with school by developing customized approached to deploying and leveraging Google Apps for Education in ‘their environment.’ For over a decade she has specialized in technology in education and her professional skills focus on creating rich and engaging [student centered] 21st Century Learning Environments that integrate innovative teaching strategies designed for digital learners at .

Years in Education:11+ ,Services Provided:Professional Development and Teacher Training,Consulting,Speaking, Engagements, Educator-to-Educator Connections

Website

Sky customer services


You can follow any responses to this entry through the RSS 2.0 Both comments and pings are currently closed.