OSWatcher

Tracking the evolution of operating systems over time

Table of Contents

• Overview
• Requirements
• Install
• Usage
• Troubleshooting
• Maintainers
• Contributing
• License

Overview

OSWatcher is an ambitious project that aims to track the evolution of operating systems by making diffs between recognizable characteristics.

The core of OSWatcher is to build a reference database about every OS releases, that is to be populated by an extractor in charge of capturing the various information that can be extracted from an installed operating system, both online and offline, in a reproducible way.

Offline:

• filesystem hierarchy
• setuid binaries
• executable properties
• library graph dependencies
• statistics around perl/sh/python scripts
• syscall tables
• kernel configuration
• cronjobs
• /etc configuration

Online:

• IDLE memory consumption
• default processes running
• mapped libraries
• listening ports and associated services
• DNS requests sent
• unix sockets
• dbus traffic
• iptables rules
• loaded drivers

Requirements

• python >= 3.7
• virtualenv
• libguestfs
• Docker (optional)

Install

1. Clone repo and submodules

git clone https://github.com/Wenzel/oswatcher.git
cd oswatcher
git submodule update --init

2. Install system dependencies

On Ubuntu 18.04

sudo apt-get install virtualenv python3-virtualenv libguestfs0 libguestfs-dev python3-guestfs python3-dev pkg-config libvirt-dev

3. Create a Python3 virtualenv

virtualenv --system-site-packages -p python3 venv
source venv/bin/activate
pip install .

Note: We have to use --system-site-packages because libguestfs is not available on pip.

VM setup

OSWatcher works on VMs stored in libvirt, either via qemu:///session or qemu:///system.

Note: qemu:///session is recommended as it requires less permission and should work without further configuration.

Example Usage: Filesystem capture in Git

Hooks configuration

Open hooks.json and edit /path/to/repo to an empty git repository (outside of oswatcher's git repo).

        {
            "name": "hooks.filesystem.GitFilesystemHook",
            "configuration":
            {
                "repo": "/home/user/test/git_fs"
            }
        }

Start the capture tool on a VM and specify the hooks configuration to start capturing the VM's filesystem in the previously configured git repository.

(venv) $ oswatcher [options] <vm_name> hooks.json

Demo

Capturing Windows XP Filesystem in a git repository (high-quality)

Advanced Usage

Neo4j

Some of OSWatcher's plugins are using neo4j as a database.

• system.OperatingSystemHook
• filesystem.Neo4jFilesystemHook
• security.SecurityHook

Follow the instructions in the db directory to run a Neo4j inside a docker container.

Modify your hooks.json to include a neo4j dictionary in the general configuration section.

You will also need to include the:

• OperatingSystemHook at least.

The rest is optional.

To visualize the filesystem in Neo4j, include the FilesystemHook and the Neo4jFilesystemHook, like the example below:

{
    "configuration":
    {
        "neo4j": {
            "enabled": true,
            "delete": false,
            "replace": false
        },
        "desktop_ready_delay": 90
    },
    "hooks":
    [
        {
            "name": "hooks.filesystem.LibguestfsHook"
        },
        {
            "name": "hooks.filesystem.FilesystemHook",
            "configuration":
            {
                "enumerate": true,
                "log_progress": true,
                "log_progress_delay": 10
            }
        },
        {
            "name": "hooks.filesystem.Neo4jFilesystemHook"
        }
    ]
}

Access Neo4j web interface at http://localhost:7474

Troubleshooting

libguestfs

If libguestfs fails to initialize, you can use the libguestfs-test-tool to quickly understand the root cause of the failure.

Maintainers

@Wenzel

Contributing

PRs accepted.

Small note: If editing the Readme, please conform to the standard-readme specification.

License

GNU General Public License v3.0