An implementation of regression testsuite for FreeBSD's audit subsystem. Made as part of GSoC'18 with FreeBSD.
For a complete overview and further updates on the project, checkout the Project Wiki.
| S. No | Test Suite | SLOC | Status |
|---|---|---|---|
| Proposed Work | |||
| 1 | audit(4) subsystem | 8785 | Merged |
| Stretch Goal | |||
| 1 | auditpipe(4) pseudo device | 293 | Merged |
| 2 | praudit(1) viewer utility | 136 | Merged |
| 3 | auditon(2) system call | 270 | In Progress |
FreeBSD is a rapidly developing operating system with an extreme focus on advanced security & networking features. For an OS with a widespread usage and development, testing and monitoring of security regressions becomes a critical measure. FreeBSD has an audit subsystem which is responsible for monitoring a variety of security-relevant system events, such as user-logins, configuration changes, file system & network access. Although the audit framework is indispensable for security conscious organizations running FreeBSD servers, currently there is no tool to test its reliability and the intended behaviour.
The project aims to develop a regression test-suite, which will evaluate the audit framework for proper logging of most auditable system calls classified in TCP/IP & UDP sockets, File I/O, process control and device management, along with the semantics of audit trail's BSM/XML/text output.
BSM tokens can be obtained via I/O multiplexing on a special clonable device /dev/auditpipe, by configuring various preselection parameters for local mode auditing with the provided IOCTLs. Several libbsm(3) APIs and functions within the FreeBSD kernel can be used to analyse syscall tokens in the audit record. Finally, kyua(7)'s run-time engine will be used to automate regression testing of entire operating system at once, audit(4) included.
For FreeBSD 12-CURRENT users, all verified tests in the Audit Test-Suite have already been merged in Head, starting with r334360. These tests can be run independently since Kyua enables standalone testing, or they can be automated to run along with the entire FreeBSD TestSuite.
- To explicitely test
audit(4)subsystem:
cd /usr/tests/sys/audit
kyua test [audit_class]:[audit_event]- To explicitely test
auditpipe(4)subsystem:
cd /usr/tests/sys/auditpipe
kyua test [auditpipe_test]- To test the audit viewer utility
praudit(1):
cd /usr/tests/usr.sbin/praudit
kyua test [praudit_test]A general report of a test-run can be found in TEST-RESULT. This is the state after r335791
Note: Port devel/kyua needs to be present in the base system along with the ATF (Automated Testing Framework) libraries (which come pre-installed with 12-CURRENT).
- Install Kyua with the FreeBSD package management tool,
pkg(7)
pkg install kyuaFor FreeBSD 12/11 STABLE, installation script is under development.
FreeBSD uses OpenBSM, an implementation of Sun's BSM event auditing file format and API. OpenBSM is made up of a plethora of tools including audit-viewer applications, e.g praudit(1) & auditreduce(1) as well as the libbsm(3) library to provide an interface to BSM audit record stream. The auditd(8) daemon is responsible for managing the kernel’s audit(4) mechanism and also rotates the log files whenever required.
FreeBSD already has userspace support for audit system. The audit daemon, auditd(8) can be enabled by adding the following line to /etc/rc.conf:
echo ‘auditd_enable=“YES”’ >> /etc/rc.confA safer way is to use sysrc(8)
sysrc auditd_enable=“YES”Then start (and stop) the audit daemon, with a new audit trail:
service auditd {one}start && audit -n;
service auditd {one}stop dir=‘/etc/security’Audit configuration is defined in ${dir}/audit_control. Various event selection expressions are defined here, also found in ${dir}/audit_class, which lets a user configure the events to be audited. For example,
- lo for login-logout
- nt for network communication
- pc for process control
- ad for administration
${dir}/audit_user specifies events to be audited for each system user, like login attempts. ${dir}/audit_event contains a list of all auditable system events.
The tests will span from simple permission check of the configuration files to exhaustive system call testing. The system calls in question will be categorized in TCP/UDP network sockets, process control, file & IO management.
In addition to the application program which triggers all processes, there will be a set of ATF test cases which will be created in accordance with FreeBSD Test Suite. A self-contained automation infrastructure will be developed which will enable the independent and ad-hoc testing of the audit system.
This application would consist of a set of test programs written in atf-c(3), containing independent test cases for triggering similar system calls, e.g open(2) & openat(2). Each program would test a wide variety of functionalities of corresponding system calls. Testing scenarios would ensure that the audit record contains all expected parameters, e.g the arguments, valid argument types, return values etc. The testing will be done for various success and failure modes, with cross-checking for appropriate errno codes in case of failure mode.
-
In a situation where the system does not have the audit system enabled, the tests would usually fail or skip. However, the audit daemon can be started (and stopped) by “service auditd one{start, stop}” without modifying
/etc/rc.conf. This eliminates the need for a setup script as described in the deprecated test plan. -
Create separate functions for the startup and termination of the audit daemon. This step also takes into account the possibility of
auditd(8)running beforehand. If so, the process termination is avoided to maintain the current state of the machine. The running status can be checked by “service auditd onestatus”. -
Audit system also allows live auditing via
auditpipe(4)for IDS and testing purpose. This enables us to open/dev/auditpipeand verify the proper audit of the system call in question. Usually in cases like these,poll(2)helps in confirming if the opened file descriptor is ready to perform I/O operations. -
The system may not have the proper settings in
/etc/security. This would fail the events not covered by “audit_control” configuration file. However,auditpipe(4)describes quite a fewioctl(2)requests in audit_ioctl.h for such scenarios.getauclassnam(3)can be used to obtain required information from theaudit_class(5)database.
These are the three pre-SoC reports outlining the initial approach on building and automating the tests.
- Report1: Network System Call testing
- Report2: File-read System Call testing
- Report3: atf-c(3) test program for mkdir(2)
3 Test Suites for various sections of FreeBSD kernel and base-system were developed as 3 consecutive GSoC projects.
| Year | Project | Author |
|---|---|---|
| 2016 | TCP-IP Test-Suite | Shivansh Rai |
| 2017 | Smoke Test-Suite | Shivansh Rai |
| 2018 | Audit Test-Suite | Aniket Pandey |
External repositories involved in the project apart from freebsd source
This project is licensed under the BSD-2-Clause License - see the LICENSE file for details.
I am grateful to my mentors, Alan Somers, George Neville-Neil and Robert Watson for the prompt-responses and the guidance they provided during the summer. It was an awesome feeling to be mentored by some of the best people of FreeBSD community. I would definitely be in touch with them and would be glad to receive their guidance in the future as well!
Also thanks to Shivansh for introducing me to FreeBSD! Looking at the work he did in the past two years (links above) gave me a definite edge in being able to adequately formulate my proposal and complete it in due time. Thanks to Mateusz Piotrowski and Domagoj Stolfa for valuable inputs at the various stages of my project whenever I was stuck. It has been a tremendous learning experience and definitely something way more than I could have expected for a summer! I look forward to continue the work on improving the audit(4) subsystem and also exploring various other sections of FreeBSD.
In the end, thanks to The FreeBSD Project for accepting my proposal and welcoming me into their community!
Aniket Pandey