Assessment

Purpose of this Work

As part of assessing Vitro, we need to:

1. test the scalability of Vitro (and, by extension, VIVO) for RIALTO type data and RIALTO phase 1 amounts of data.
2. ease [and cost time-wise] to work, develop, maintain Vitro
   • Take any info from what we’ve learned in the scaling questions
   • Highlight any open questions about Vitro [VIVO] usage
3. If Vitro scales adequately, Vitro vs VIVO
   • Driven by data modeling
   • Driven by the ease of usage / cost of usage

Vitro Performance Assessment Matrix

Tests for assessing performance of Vitro (or other identified future options) as the canonical RIALTO data store. Options listed in priority order, starred tests being the preferences for testing. Standard loading of 4,596,065 triples (3,158,059 unique) from 73,479 n-triples files.

• inf == inferencing (within Vitro)
• index == indexing (within Vitro, default mappings to Solr)
• ⭐ == primary foci

Vitro Ingest	no inf & no index	inf & no index	no inf & index ⭐	inf & index ⭐
1 SPARQL Update API (Vitro)	Unable to run w/Vitro	Unable to run w/Vitro	Docs, Metrics	Dismissed
2 Jena tdbloader (cmd line)	Dismissed	Dismissed	Dismissed	Dismissed
3 TDB Java API (Vitro)	Dismissed	Dismissed	Dismissed	Dismissed
4 Jena SPARQL Update (ARQ)	Dismissed	Dismissed	Dismissed	Dismissed
5 Fuseki SPARQL Update API	n/a	Dismissed	Docs, Metrics	Dismissed

Metrics gathered for each case:

• Sample Data Load Times
  • Total time to load all sample data
  • Time per RDF record / file
  • Time per request (if multiple requests, i.e., HTTP)
  • Scaling graph based on the above
• Sample Data Index Times (if indexing)
• Ops Metrics: (taken from https://sulstats.stanford.edu/dashboard/db/servers)
  • CPU usage
  • memory usage
  • overall load
  • swap usage
  • perhaps an I/O-related metric would be good too