Skip to content

Using Apache Hive to directly evaluate SPARQL queries.

License

Notifications You must be signed in to change notification settings

tyrex-team/rdfhive

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

20 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

RDFHive

Using Apache Hive to directly evaluate SPARQL queries.

Overview: SPARQL is the W3C standard query language for querying data expressed in RDF (Resource Description Framework). The increasing amounts of RDF data available raise a major need and research interest in building efficient and scalable distributed SPARQL query evaluators.

In this context, we propose and share RDFHive: a simple implementation of a distributed RDF datastore benefiting from Apache Hive. RDFHive is designed to leverage existing Hadoop infrastructures for evaluating SPARQL queries. RDFHive relies on a translation of SPARQL queries into SQL queries that Hive is able to evaluate.

Technically, RDFHive directly evaluates SPARQL queries i.e. there is no preprocessing step, indeed an RDF triple file is seen by Hive as a three-column table. Thus, the bash translator simply translates SPARQL queries according to this scheme. This method has two advantages: first, creating a database is very fast; second, since the upfront investment is light, RDFHive is an interesting tool to evaluate a few SPARQL queries at once.

Version: 1.0

Requirements

How to use it?

In this package, we provide sources to load and query RDF datasets with RDFHive. We also present a simple test-suite based on the popular RDF/SPARQL benchmark: LUBM. For space reasons, this dataset only contains a few hundred of thousand RDF triples.

Get the sources.

git clone github.com/tyrex-team/rdfhive.git ;
cd rdfhive/ ;

Load an RDF dataset.

RDFHive can only load RDF data written according to the N-Triples format. This file has to be uploaded first on the HDFS.

hadoop fs -copyFromLocal local_file.nt hdfs_file.nt ;
bash bin/load.sh dbName hdfs_file.nt ;

Query Evaluation.

To execute a SPARQL query over a loaded RDF dataset, RDFHive first translates it into SQL and then evaluates the generated query. If --debug is specified, RDFHive will be more verbose.

bash bin/eval.sh dbName LocalQueryFile ;

Remove a Database.

An already created database can also be removed.

bash bin/remove.sh dbName ;

Test Suite.

Finally, a very basic test suite is included in this repository to demonstrate RDFHive.

cd tests/ ;
bash preliminaries.sh ;
bash run-benchmarks.sh ;
bash clean-all.sh ;

Additional Scripts.

Moreover, two scripts are also part of the project (in bin/): lubmqueries.sh and watdivqueries.sh which already contain translation of LUBM and WatDiv SPARQL queries.

Supported SPARQL Fragment

digit := [1-9]
alphanum := [a-z|A-Z|1-9]
prefix := PREFIX (alphanum)*: <(alphanum)*>
var := ("?"|"$")(alphanum)*
tp := (var|(alphanum)*) (var|(alphanum)*) (var|(alphanum)*)
selectQuery :=
   (prefix)*
   SELECT (REDUCED|DISTINCT)? ("*"|(var)+)
   WHERE { (tp) (" . "tp)* }
   (LIMIT (digit)*)? (OFFSET (digit)*)?

License

This project is under the CeCILL license.

Authors

Damien Graux
damien.graux@inria.fr

Pierre Genevès
Nabil Layaïda

Tyrex Team, Inria (France), 2016

About

Using Apache Hive to directly evaluate SPARQL queries.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages