Repo Check

Service suite for strengthening the world of software documentation and becoming a prolific open source contributer in the process.

Python 3, Flask, Docker, bash

Components

Repo Check operates as a suite of Dockerized applications. Exactly one Queueing and at least one Inspector are required for core functionality. A Finder instance allows for functionality without the manual addition of git repositories.

All components use a run_loop() rate-limited by an idle() method and enrivonement variables.

• IDLE_INTERVAL determines the time to wait when a component has no job before checking the queue again.
• CYCLE_INTERVALS set a minimum time between run_cycle iterations. If an iteration completes in less time, the service will idle for the difference.

Queueing

The main function of the Queueing service is to keep the inspection queue filled with inspection jobs.

Inspector

The main function of the Inspector service is to check repository documentation for errors and generate reports on their findings.

Finder

The main function of the Finder service is to find new repositories to add to the database.

Data Structures and Schemas

Report

{
  "metadata": {
    "repo_url": string,                 # URL of the associated git repo
    "created": integer,                 # Linux epoch timestamp as integer
    "version": string,                  # Version of Repo Check Inspector
    "job": Job,                         # The initiating inspection job
    "inspected_files": list[string],    # Files analyzed by Inspector
    "status": string,                   # enum("closed", "new", "open")
    "last_viewed": integer              # Linux epoch timestamp as integer
  },
  "data": {
    "summary": {
      "count_typos": integer,           # Number of typos found
      "typos": list[tuple[]],           # List of (typo, correction) tuples
      "count_bad_links": integer,       # Number of links found with non-2XX status
      "bad_links": list[tuple[]],       # List of (URL, status code) tuples
    },
    "details": {
      $document1: {                     # 
        "typos": [],                    #
        "bad_links": []                 #
      },
      $document2: {},
      $document3: {},
      ...
    }
  }
}

Inspection Job

{
  "created": integer,
  "repo_url": string,
  "file_list": list[string],            # <Optional> Generated by LocalRepo if None
  "force": bool                         # Force processing of jobs that fail prerun()  # Not implemented
}

MySQL Database

Repo table

CREATE TABLE 'repo' (
  'id' INT(11) NOT NULL AUTO INCREMENT,
  'url' VARCHAR(160) NOT NULL,
  'repo_type' ENUM('docs', 'web') DEFAULT NULL, 
  'added' INT(10) UNSIGNED DEFAULT NULL,
  'last_inspected' INT(10) UNSIGNED DEFAULT NULL,
  'last_result' ENUM('error', 'none', 'report'),
  'check_out' TINYINT(1) DEFAULT 0 NOT NULL,
  PRIMARY_KEY ('id')
) ENGINE=InnoDB;

Watch table

CREATE TABLE watch (
  id INT(11) NOT NULL AUTO INCREMENT,
  repo INT(11) NOT NULL,
  watched_since INT(10) UNSIGNED NOT NULL,
  PRIMARY_KEY ('id'),
  FOREIGN KEY ('repo') REFERENCES repo ('id')
) ENGINE=InnoDB;