A Native Windows to Redis/Elasticsearch Logstash Agent which runs as a service.
TimberWinR's development has been and in the short term will continue to be on a hiatus. The project's ownership has been recently transferred, and thus the project will be slow to pick up steam again. Currently a roadmap is being established, and a vision for the project is currently being formulated.
If you wish to be involved with TimberWinR in an on-going basis file an issue.
TimberWinR is a native .NET implementation utilizing Microsoft's LogParser. This means no JVM/JRuby is required, and LogParser does all the heavy lifting. TimberWinR collects the data from LogParser and ships it to Logstash via Redis (or can ship direcly to Elasticsearch)
TimberWinR uses a configuration file to control how the logs are collected, filtered and shipped off.
These are broken down into:
- Inputs (Collect data from different sources)
- Filters (Are applied to all Inputs)
- Outputs (e.g. Redis, Elasticsearch, Stdout, StatsD)
Please use the TimberWinR Google Group for discussion and support:
https://groups.google.com/forum/#!forum/timberwinr
Latest Build:
The current supported Input format sources are:
- Logs (Files, a.k.a Tailing a file)
- Tcp (listens on TCP port for JSON messages)
- IISW3C(Internet Information Services W3C Format)
- WindowsEvents (Windows Event Viewer)
- Stdin (Standard Input for Debugging)
- W3C(Internet Information Services W3C Advanced/Custom Format)
- Udp (listens for UDP on port for JSON messages)
- TailFiles (Tails log files efficiently)
- Generator (Generate logs for testing New)
The current list of supported codecs are:
The current list of supported filters are:
Note that there are now two syntaxes for filters, array and single, if you want more than one filter of the same type then you must use the array tag instead of the singular tag. i.e:
A single Json filter using the single tag (this is only provided as a convienience, the array syntax is preferred)
"Filters": [
{
"json": {
"type": "Win32-FileLog",
"condition": "\"[logSource]\" == \"dev\"",
"source": "Text",
"add_field": [
"_index",
"dev-%{yyyy.MM.dd}"
]
}
}
]Multiple Json filters must use the jsonFilters and array syntax, also mutateFilters, grokFilters, dateFilters, geoipFilters.
"Filters": [
{
"jsonFilters": [
{
"type": "Win32-FileLog",
"condition": "\"[logSource]\" == \"dev\"",
"source": "Text",
"add_field": [
"_index",
"dev-%{yyyy.MM.dd}"
]
},
{
"type": "Win32-FileLog",
"condition": "\"[logSource]\" == \"sta\"",
"source": "Text",
"add_field": [
"_index",
"sta-%{yyyy.MM.dd}"
]
}
]
}
]Since TimberWinR only ships to Redis and Elasticsearch, the format generated by TimberWinR is JSON. All fields referenced by TimberWinR can be represented as a JSON Property or Array.
TimberWinR reads a JSON configuration file, an example file is shown here:
{
"TimberWinR": {
"Inputs": {
"WindowsEvents": [
{
"source": "System,Application",
"binaryFormat": "PRINT",
"resolveSIDS": true
}
]
},
"Filters": [
{
"grok": {
"condition": "\"[type]\" == \"Win32-Eventlog\"",
"match": [
"Message",
""
],
"remove_field": [
"ComputerName"
]
}
}
],
"Outputs": {
"Redis": [
{
"_comment": "Shuffle these hosts",
"host": [
"server1.host.com",
"server2.host.com"
]
}
]
}
}This configuration:
- Inputs: Events from the Windows Event Logs (System, Application)
- Filters: Removes the ComputerName field
- Sends the event to Redis services (server1.host.com, server2.host.com) in a shuffling manner (balanced).
If installing from source, you must first install LogParser, then install TimberWinR. Install LogParser from here:
Install LogParser from Microsoft.
After installing, follow the remaining directions here. If you install from Chocolatey then LogParser will automatically be installed first.
You can run TimberWinR interactively when you are developing your JSON config file, to do so use the following options:
TimberWinR.ServiceHost.exe -configFile:myconfig.json -logLevel:Debug
TimberWinR uses TopShelf to install as a service, so all the documentation for installing and configuring the service is show here TopShelf Doc
Specifically the command line options are listed here in Topshelf Command-Line Reference guide.
Install and set to Automatically Start the service:
; Install Service (will autostart on reboot)
TimberWinR.ServiceHost.exe install --autostart
; Start the Service
TimberWinR.ServiceHost.exe start
To Start/Stop the Service from the Command Line
TimberWinR.ServiceHost.exe start
TimberWinR.ServiceHost.exe stop
Alternatively you can use the Services Control Panel.
TimberWinR.ServiceHost.exe [options]
Options:
-logDir: Specifies the directory where TimberWinR will write its log file TimberWinR.txt
Default is -logDir:"C:\logs"
-logLevel: Specifies the logging level for TimberWinR
Legal Values: Trace|Debug|Info|Warn|Error|Fatal|Off
Default is -logDir:Info
-configFile: Specifies the path to the JSON config file, or directory which contains .json file(s).
Default is -configFile:default.json
-diagnosticPort: Specifies the diagnostic port which can be used to get a health check of the service.
Default Port is 5141, A value of 0 will disable it. Open a browser
http://localhost:5141
This may be a single .json file or a directory containing .json file(s). If it is a directory, all files will be read and processed, the order in which the files will be processed will match the alphabetical order on disk.
If you really just want to try it out, grab the binary distribution, extract the .zip file into a directory, e.g. C:\TimberWinR
Grab the JSON example file and place it into C:\TimberWinR\default.json. Edit the default.json file and change the Redis instance to match yours, replace 'tstlexiceapp006.mycompany.svc' with the IP or DNS name of the machine running redis. Fire up the collector, enable the verbose debugging to see some Windows Events.
TimberWinR.ServiceHost.Exe -configFile:default.json -logLevel:Debug
You should see
To run it as a service
TimberWinR.ServiceHost.exe install --autostart
TimberWinR.ServiceHost.exe start
TimberWinR is distributed as an installable package via Chocolatey, and it is dependent on
After you install it via Chocolatey, you will have a folder:
C:\Program Files (x86)\TimberWinR
TimberWinR will be configured to read the file default.json located in the folder above. You can edit this file to customize your installation.
Use these commands to Stop/Start the service.
sc stop TimberWinR ; stop the service
sc start TimberWinR; start the service