Object import for Hyrax. |
In your project's Gemfile
, add: gem 'zizia'
, then run bundle install
.
- Require 'zizia' in your
config/application.rb
file:
module MyApplication
class Application < Rails::Application
require 'zizia'
- Add the engine to
routes.rb
:
mount Zizia::Engine => '/'
- Add the helpers to the
ApplicationController
helper Zizia::Engine.helpers
- Give admin users permission to import in your
Ability.custom_permissions
:
can :manage, Zizia::CsvImport if current_user.admin?
can :manage, Zizia::CsvImportDetail if current_user.admin?
-
Add links to
/csv_imports/new
and/importer_documentation/csv
in the Hyrax dashboard. -
In your Rails application's
application.css
andapplication.js
include Zizia's assets:
*= require zizia/application
- Run
rake db:migrate
The spec/dummy
folder in this application is a complete Hyrax application with Zizia installed.
You can use that as an example for adding this to your current Hyrax application or copy that
to create a new application with Zizia installed.
- Add a deduplication_key to your default work type's metadata:
property :deduplication_key, predicate: ::RDF::Vocab::BF2::identifiedBy, multiple: false do |index|
index.as :stored_searchable
end
- If you are using the default Hyrax metadata profile aka
Hyrax::BasicMetadata
, you are ready to download a sample CSV and start importing.
If you aren't using Hyrax::BasicMetadata
you'll need to create a custom importer
and mapper
class. First ensure that a work type is registered
with your Hyrax
application. Then write a class like this:
require 'zizia'
class MyImporter
def initialize(csv_file)
@csv_file = csv_file
raise "Cannot find expected input file #{csv_file}" unless File.exist?(csv_file)
end
def import
attrs = {
collection_id: collection_id, # pass a collection id to the record importer and all records will be added to that collection
depositor_id: depositor_id, # pass a Hyrax user_key here and that Hyrax user will own all objects created during this import
deduplication_field: 'identifier' # pass a field with a persistent identifier (e.g., ARK) and it will check to see if a record with that identifier already
} # exists, update its metadata if so, and only if it doesn't find a record with that identifier will it make a new object.
file = File.open(@csv_file)
parser = Zizia::CsvParser.new(file: file)
record_importer = Zizia::HyraxRecordImporter.new(attributes: attrs)
Zizia::Importer.new(parser: parser, record_importer: record_importer).import
file.close # unless a block is passed to File.open, the file must be explicitly closed
end
end
You can find an example csv file for import to Hyrax in the fixtures directory. Files for attachment should have the filename in a column
with a heading of files
, and the location of the files should be specified via an
environment variables called IMPORT_PATH
. If IMPORT_PATH
is not set, HyraxRecordImporter
will look in /opt/data
by default.
git clone https://github.com/curationexperts/zizia
cd zizia
bundle install
To run Solr and Fedora for testing purposes, open a new terminal session for each of the following commands and run each in it's own terminal session:
solr_wrapper --config spec/dummy/config/solr_wrapper_test.yml
fcrepo_wrapper --config spec/dummy/config/fcrepo_wrapper_test.yml
Please also ensure you have an appropriate version of ChromeDriver installed and are running Redis in order to run the test suite:
redis-cli ping
should return "PONG"
chromedriver -v
should return a version matching your installed version of Chrome
After this you can run the whole suite:
bundle exec rspec spec
System specs are located in the spec/dummy/spec/system
folder:
bundle exec rspec spec/dummy/spec/system/csv_import_details_page_spec.rb
- Set the environment variable
NOCOV=true
to run the tests without Coveralls - You can run individual tests by giving the line number of the test, e.g.
bundle exec rspec spec/dummy/spec/system/csv_import_details_page_spec.rb:23
To input any kind of file other than CSV, you need to provide a Parser
(out of the box, we support simple CSV import with CsvParser
). We will be writing guides about
how to support custom mappers (for metadata outside of Hyrax's core and basic metadata fields).
To make a new release:
- Increase the version number in
lib/zizia/version.rb
- Increase the same version number in
.github_changelog_generator
- Update CHANGELOG.md by running this command:
github_changelog_generator --user curationexperts --project zizia --token YOUR_GITHUB_TOKEN_HERE
- Commit these changes to the master branch
- Run
rake release