Zeek Documentation

The documentation repo at zeek-docs contains version-specific Zeek documentation source files that are ultimately used as the basis for content hosted at https://docs.zeek.org.

Markup Format, Style, and Conventions

For general guidance on the basics of how the documentation is written, consult this Zeek wiki:

https://github.com/zeek/zeek/wiki/Documentation-Style-and-Conventions

Source-Tree Organization

The zeek-docs repo containing this README file is the root of a Sphinx source tree and can be modified to add more documentation, style sheets, JavaScript, etc. The Sphinx config file is conf.py. The typical way new documents get integrated is from them being referenced directly in index.rst or indirectly from something in the toctree (Table of Contents Tree) specified in that main index.

There is also a custom Sphinx domain implemented in ext/zeek.py which adds some reStructureText (reST) directives and roles that aid in generating useful index entries and cross-references. This primarily supports integration with the script-reference sections, some of which are auto-generated by Zeek's Doxygen-like feature, named "Zeekygen". The bulk of auto-generated content lives under the scripts/ directory or has a file name starting with "autogenerated", so if you find yourself wanting to change those, you should actually look at at doing those changes within the Zeek repo itself rather than here, so see the next section for how Zeekygen docs can be (re)generated.

Generating Zeekygen Reference Docs

All Zeekygen-generated docs get committed into Git, so if you don't have to perform any changes on it and just want to preview what's already existing, you can skip down to the next :ref:`Local Previewing <local-doc-preview>` section.

The Zeekygen documentation-generation feature is a part of Zeek itself, so you'll want to obtain the Zeek repo from Git, read the :doc:`INSTALL </install>` file directions to install required dependencies, and build Zeek:

git clone --recursive https://github.com/zeek/zeek
cd zeek
# Read INSTALL file and get dependencies here
./configure && make -j $(nproc)
# Make desired edits to scripts/, src/, etc.
./ci/update-zeekygen-docs.sh

The last command runs a script to generate documentation, which will end up in the doc/ subdirectory. Note that doc/ is just a Git submodule of this this zeek-docs repository, so you can run git status there to find exactly what changed.

Also note that the documentation-generation script is run automatically on a daily basis to incorporate up any documentation changes that people make in Zeek itself without them having to necessarily be aware of the full documentation process. The GitHub Action that does that daily task is located in the Zeek repo's .github/workflows/generate-docs.yml file.

Local Previewing (How To Build)

First make sure you have the required dependencies used for building docs:

• Python interpreter >= 3.7
• Sphinx: https://www.sphinx-doc.org/en/master/
• Read the Docs Sphinx Theme: https://github.com/rtfd/sphinx_rtd_theme
• GitPython: https://github.com/gitpython-developers/GitPython

If you have pip, you may just use the command pip3 install -r requirements.txt to install all the dependencies using the requirements.txt from zeek-docs.

Now run make within the zeek-docs repository's top-level to locally render its reST files into HTML. After the build completes, HTML documentation is symlinked in build/html and you can open the index.html found there in your web browser.

There's also a make livehtml (requires pip3 install sphinx-autobuild) target in the top-level Makefile that is useful for editing the reST files and seeing changes rendered out live to a separate browser.

Hosting

Documentation is hosted by Read the Docs (RTD), so you can generally read about how it works there. The web-interface is accessible via https://readthedocs.org/projects/zeek-docs.

How zeek-docs is configured to use RTD is a combination of some custom settings in its .readthedocs.yml file and others only accessible through RTD's web-interface (e.g. domain and subproject settings). Most config settings are likely understandable just by browsing the web-interface and RTD's guides, but a few particular points to mention:

• There is an associated, always-failing project at https://readthedocs.org/projects/zeek. It's always-failing because RTD redirects only activate when pages 404 and this project exists so that all attempts to use https://zeek.rtfd.io or https://zeek.readthedocs.io get redirected to https://docs.zeek.org. Those would have been the project URLs if ownership of the RTD 'zeek' project was had from the start, but it was only obtained later, after documentation already started development in the 'zeek-docs' RTD project slug.
• Over time, page redirects have accrued into redirects.yml as a way to help document what they are and why they happened and also as a potential way to automate addition/reinstantiation of a large number of redirects, but typically redirects can be manually added via the RTD web interface first and then noted in redirects.yml
• There are RTD subprojects for things like Broker, Package Manager, and Spicy. The use of subprojects simply allows access to their RTD docs via the custom domain of https://docs.zeek.org
• RTD will auto-build any newly-pushed commits to zeek-docs (i.e. a webhook is configured), but if a tag is changed to point somewhere different, you'll typically have to go into the RTD web interface, "Edit" the associated version under "Versions", "wipe" the existing docs, and then manually trigger a rebuild of that version tag under "Builds".