Source of Flume NG for tailing files in a directory
| Property Name | Default | Description |
|---|---|---|
| Channels | - | |
| Type | - | com.jinoos.flume.DirectoryTailSource |
| Type | - | com.jinoos.flume.DirectoryTailSource |
| dirs | - | Nick of directories, it's such as list of what directories are monitored |
| parser | com.jinoos.flume.SingleLineParserModule | Implemented class path of DirectoryTailParserModulable. |
| dirs.NICK.path | - | Directory path |
| dirs.NICK.file-pattern | - | Pattern of target file(s), ex) '^(.*)$' is for all files |
For parser, there is two module classes what implemented DirectoryTailParserModulable
-
SingleLineParserModule
-
MultiLineParserModule
-
Example for SingleLineParserModule
agent.sources = dirMon
agent.sources.dirMon.type = com.jinoos.flume.DirectoryTailSource
agent.sources.dirMon.parser = com.jinoos.flume.SingleLineParserModule
agent.sources.dirMon.dirs = tmpDir varLogDir
agent.sources.dirMon.dirs.tmpDir.path = /tmp
agent.sources.dirMon.dirs.tmpDir.file-pattern = ^(message)$ # /var/log/message
agent.sources.dirMon.dirs.varLogDir.path = /var/log
agent.sources.dirMon.dirs.varLogDir.file-pattern = ^(.*)(\.log)$ # /tmp/*.log
- Example for MultiLineParserModule
agent.sources = dirMon
agent.sources.dirMon.type = com.jinoos.flume.DirectoryTailSource
agent.sources.dirMon.parser = com.jinoos.flume.MultiLineParserModule
# Regex pattern for first line, format "yyyy-MM-dd HH:mm:ss"
agent.sources.dirMon.first-line-pattern = ^([\d]{4})-([\d]{2})-([\d]{2}) ([\d]{2}):([\d]{2}):([\d]{2})
agent.sources.dirMon.dirs = tmpDir varLogDir
agent.sources.dirMon.dirs.tmpDir.path = /tmp
agent.sources.dirMon.dirs.tmpDir.file-pattern = ^(message)$ # /var/log/message
agent.sources.dirMon.dirs.varLogDir.path = /var/log
agent.sources.dirMon.dirs.varLogDir.file-pattern = ^(.*)(\.log)$ # /tmp/*.log