Skip to content
This repository has been archived by the owner on Jun 3, 2024. It is now read-only.

Latest commit

 

History

History
182 lines (142 loc) · 4.67 KB

developer.md

File metadata and controls

182 lines (142 loc) · 4.67 KB
Raw

Developer Documentation

Design

Software Architecture

Introduction

Bragi

Contributing

Release

Note: Some of the links below might not be accessible by everyone, since the release is only handled by the owners of the repository. Don't worry, if you don't have access, it's probably because you don't have to create a release!

Here is a brief description of the sequence of actions that should happen:

  • Jenkins job mimir_build_release (see .jenkins.release)
    • Launch a new job manually (selecting Minor or Major)
    • the job will bump Cargo.toml
    • the job will create a git tag
    • the creation of the tag will trigger Github Actions
  • Github Actions
    • build the Docker images and publish them
    • build the Debian packages exporting artifacts
    • trigger publish_autocomplete_packages Jenkins job
  • Jenkins job publish_autocomplete_packages (see .jenkins/publish_packages)
    • the job will get the Debian package from artifacts created in Github Actions
    • the job will upload the Debian packages to various places (FTP, Nexus)

publish_autocomplete_packages should be automatically triggered by Github Actions but it might not. If this is the case, launch this job manually once Github Actions has finished its actions.

Dependencies

Crates

This is a list of (some of) the main crates the projects depends on:

Domain Crate Motivation Alternatives
logging tracing
  • Same team as tokio, warp, …
  • Support opentelemetry
  • Support tracing, logs
error handling snafu
web framework warp
commandline clap
elasticsearch elasticsearch

Development Process

Testing

You will find information about tests in general here.

This section is meant for developing your own tests.

Tools

import2mimir

import2mimir.sh is a bash script used to easily create and manipulate a test environment. It contains functions to

  • download dataset from remote sources.
  • create and initialize a docker-based elasticsearch environment.
  • index admins, streets, addresses, stops, pois.

import2mimir2.sh is configured by a small rc file. Here is an annotated example:

# The directory in which data is stored
# DATA_DIR="data"

# Elasticsearch host. If you use docker, it should probably stay at localhost.
ES_HOST="localhost"

# The port number for elasticsearch host (9200 + offset, 9300 + offset)
ES_PORT_OFFSET=3

# The name of the dataset
ES_DATASET="idf"

# The name of the image
ES_IMAGE="docker.elastic.co/elasticsearch/elasticsearch:7.13.0"

# The name of the container
ES_NAME="elasticsearch"

This first part of the configuration says that we will start an elasticsearch based on the ${ES_IMAGE} image. It will publish ports 9203 and 9303. The name of this docker container will be ${ES_NAME} (elasticsearch).

The next part is to configure what data is downloaded and indexed.

# The departement (French administrative regions) for BANO
# Base d'Adresses Nationale Ouverte data.gouv.fr
BANO_REGION="75 77 78 92 93 94 95"

# NTFS Region to download
NTFS_REGION="fr-idf"

# OSM Region
OSM_REGION="ile-de-france"

# Base directory for cosmogony
COSMO_DIR="../cosmogony"

The script takes a configuration file (like the one we just described), and displays its progress on stdout, as well as record a log file.

cd mimirsbrunn
./script/import2mimir.sh -c ./scripts/idf.rc

WARNING This script will destroy the docker container named ${ES_NAME} prior to recreating it. Make sure this is not a container you want to keep.

autocomplete