Skip to content

Add a new Datasource

Davide Macchia edited this page Sep 5, 2022 · 10 revisions

Procedures

If you are a Data Provider and you want to upload the model of your data, follow this guide:

A new Model

  1. Add a new unique DatasourceCode on descriptor.enum (e.g. THEGUARDIAN = "THEGUARDIAN").
  2. Create a new folder under src/source (e.g. /src/source/theguardian).
  3. In the new folder, create a Model (model.theguardian.ts) containing all the interfaces that describes the datasource's information.
  4. Add the new file to the index.ts under /source/ folder (export * from './theguardian/model.theguardian').

A new Validator

  1. If the Datasource contains multiple files (e.g Google datasource) add a new Enumeration into descriptor.enum with a RegEx for each file of the new datasource
  2. Add the new enumeration to the FileCode type
  3. Add a description for each new FileCode into descriptor.json, skip this step otherwise.
  4. if the Datasource can be exported, describe the procedure into the descriptor.json.
  5. Create a new Validator file under the new folder (e.g. /src/source/theguradian/validator.theguardian.ts). The Validator must have a default list of FileCodes, must extends the ValidatorDatasource class and overwrite the functions to let the FileCodes' RegEx match the files' pathnames.
  6. Update all the functions related functions under Selector.
  7. Update the index.ts

A new Service

  1. Create a new Service file under the new folder (e.g. /src/source/theguradian/service.theguardian.ts).
  2. The Service must parse each file of the Datasource and map them into a specific interface of the Model, must have a parseFile function.
  3. Update all the functions related functions under Selector.
  4. Update the index.ts

A new Processor

  1. Create a new Processor file under the new folder (e.g. /src/source/theguradian/processor.theguardian.ts), it must implements ProcessorDatasource using the decorator @staticImplements<ProcessorDatasource>().
@staticImplements<ProcessorDatasource>()
export class ProcessorTheguardian {
...
}
  1. Update all the functions related functions under Selector.
  2. Update the index.ts
Clone this wiki locally