README.md

Introduction

pdslogger provides a new class and associated functions that augment the functionality of the standard Python logging module.

pdslogger is a product of the PDS Ring-Moon Systems Node.

Installation

The pdslogger module is available via the rms-pdslogger package on PyPI and can be installed with:

pip install rms-pdslogger

Getting Started

The PdsLogger class provides a variety of enhancements to Python's logging.Logger class. Use the PdsLogger() constructor or method get_logger() to create a new logger or access an existing one by name. To add the capabilities of this class to an existing logging.Logger, use PdsLogger.as_pdslogger(). In this case, the behavior of the new PdsLogger will be identical to that of the given Logger, including all formatting, but the additional capabilities of this class will also become available.

PdsLogger supports hierarchical logs, in which the log generated by a running task can be cleanly separated into individual sub-tasks. When a sub-task is completed, the log contains a summary of the number of logged messages by category and, optionally, the elapsed time. You create a new level in the hierarchy with open() and close it with close(). Alternatively, you can write:

with logger.open(...):
    # Write log messages here

to close the newly-opened section of the log automatically. A specific handler can be assigned to the PdsLogger as part of the open() call and it will be removed upon closing. Use this feature, for example, to obtain separate log files for individual tasks or when processing a sequence of individual files.

The constructor allows the user to create additional error categories beyond the standard ones named "debug", "info", "warn", etc. Each category can be assigned its own numeric level, where DEBUG=10, INFO=20, WARNING=30, ERROR=40, and FATAL=50. A level of None or HIDDEN means that messages with this alias are always suppressed.

The following additional message categories are used widely by the RMS Node and are defined by default:

• "normal" (default level 20=INFO) is used for any normal outcome.
• "header" (default level 20=INFO) is used for message headers following open() and summary messages following close().
• "exception" (default level 50=FATAL) is used when an exception is encountered.
• "ds_store" (default level 10=DEBUG) is used when a task encounters a ".DS_Store" file as managed by the MacOS Finder.
• "dot_" (default level 40=ERROR) is used when a file beginning with "._" is encountered. These files are sometimes created by "tar" commands in MacOS.
• "invisible" (default level 30=WARN) is used if any other invisible file is encountered.

Of course, any of these default levels can be modified on a logger-by-logger basis.

Use set_limit() to specify a limit on the number of messages that can be associated with an alias. For example, if the limit on "info" messages is 100, then log messages after the hundredth will be suppressed, although the number of suppressed messages will still be tracked. At the end of the log, a tally of the messages associated with each alias is printed, including the number suppressed if the limit was exceeded.

PdsLogger supports a rich set of formatting options for the log records, which can be specified in the constructor or by using set_format(). By default, each log record automatically includes a time tag, log name, level, text message, and optional file path. You can also use add_root() and related methods to exercise more control over how file paths appear.

Use log() to log a message, or level-specific methods such as debug(), info(), warning(), error(), critical(), etc. Each of these methods receives a message string and arguments identical to the methods of the same name in logging.Logger. However, they also take an optional file path, which is automatically formatted to appear after the message text in the log.

Method exception() can be used inside a Python except clause to write an exception message into the log, including the stacktrace. If you desire greater control over how an exception is recorded in the log, you can use the LoggerException class or define your own subclass.

Simple tools are also provided to create handlers to assign to a PdsLogger using add_handler() and related methods:

• file_handler() is a function that provides a rich set of options for constructing logging.FileHandler objects, which allow logs to be written to a file. The options include version numbering, appending a date or time to the file name, and daily rotations. file_handler() also supports the RMS Node's rms-filecache module, which allows log files to be seamlessly saved into cloud storage; simply pass in a URI or FCPath object instead of a local file path.
• info_handler(), warning_handler(), and error_handler() are simpler versions of the above, in which the level of message logging is implied.
• stream_handler() is a function that creates handlers to write to an I/O stream such as sys.stdout or sys.stderr.
• STDOUT_HANDLER is a pre-defined handler that prints all output to the terminal.
• NULL_HANDLER is a pre-defined handler that suppresses all output.

Note that a PdsLogger will print message to the terminal if no handler has been assigned to it. As a result, if you really wish to not see any messages, you must assign it the NULL_HANDLER.

In the Macintosh Finder, log files are color-coded by the most severe message encountered within the file: green for info, yellow for warnings, red for errors, and violet for fatal or critical errors.

For extremely simple logging needs, four subclasses of PdsLogger are provided.

• EasyLogger prints all messages above a specified level of severity to the terminal.
• ErrorLogger only prints error messages.
• CriticalLogger only prints exceptions and other "fatal" messages.
• NullLogger suppresses all messages, including logged exceptions.

These four subclasses have the common trait that they cannot be assigned handlers.

Details of each function and class are available in the module documentation.

This simple example:

import pdslogger
logger = pdslogger.PdsLogger('sample', parent='test')
logger.warning('Warning message')
with logger.open('Sub-log'):
    logger.debug('Debug message level 2')
logger.close()

will yield:

2024-12-04 13:47:37.004203 | test.sample || WARNING | Warning message
2024-12-04 13:47:37.004224 | test.sample || HEADER | Sub-log
2024-12-04 13:47:37.004240 | test.sample |-| DEBUG | Debug message level 2
2024-12-04 13:47:37.004270 | test.sample || SUMMARY | Completed: Sub-log
2024-12-04 13:47:37.004276 | test.sample || SUMMARY | Elapsed time = 0:00:00.000016
2024-12-04 13:47:37.004280 | test.sample || SUMMARY | 1 DEBUG message

2024-12-04 13:47:37.004295 | test.sample || SUMMARY | Completed: test.sample
2024-12-04 13:47:37.004299 | test.sample || SUMMARY | Elapsed time = 0:00:00.000094
2024-12-04 13:47:37.004302 | test.sample || SUMMARY | 1 WARNING message
2024-12-04 13:47:37.004305 | test.sample || SUMMARY | 1 DEBUG message

Contributing

Information on contributing to this package can be found in the Contributing Guide.

Links

• Documentation
• Repository
• Issue tracker
• PyPi

Licensing

This code is licensed under the Apache License v2.0.