Monitoring

So you'd like to generate alerts when something goes sideways in the system...

Using 3rd Party Tools

If monitoring CORE using 3rd party tools such as Nagios, Sensu, Zabbix, etc, reasonable things to check include:

• The server running Office is online and serving HTTP
• The server running Office's database is online and serving MySQL
• The lane machines are online and serving MySQL (the lanes do not need to allow remote HTTP)

These type of tools work great for checking uptime and other basics like server load, disk space, etc. They will likely always have more advanced alerting options than anything built into CORE.

Using Built-In Tools

First, put an email address into the Scheduled Task interface. This interface is just a thin wrapper around cron. Putting an address here will generate an email any time a scheduled task crashes (or writes to stderr). Make server running Office can actually send email.

Second, enable one or more Monitors on the Dashboard page as well as the Monitoring Scheduled task. This page also has a separate email configuration to generate notifications. Monitors are modules that assess some aspect or aspects of the system each time the Monitoring task runs. The Dashboard page updates after each run and alerts are generated if a critical situation is detected. A couple Monitors are provided out of the box but you can also write your own to check for other conditions.

CORE's Monitors do not necessarily do anything that can't also be done via 3rd party tools, but if you're more familiar with PHP than the scripting languages used by a given 3rd party tool this may be an easier path. All Monitor classes must inherit from COREPOS\Fannie\API\monitor\Monitor and define three methods:

• check()
  • This method examines the system state for whatever the monitor cares about. It returns a JSON-encoded string. It can format the decoded data however it wants.
• escalate(JSON string)
  • This method evaluates the system state to determine whether it's critical - meaning someone should be notified ASAP. The argument is whatever check returns.
• display(JSON string)
  • This method reformats the data in HTML for in-browser display.