Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for CREATE watch event for config-directory #12264

Closed
neelayu opened this issue Nov 21, 2022 · 6 comments · Fixed by #15645
Closed

Support for CREATE watch event for config-directory #12264

neelayu opened this issue Nov 21, 2022 · 6 comments · Fixed by #15645
Assignees
Labels
area/agent feature request Requests for new plugin and for new features to existing plugins

Comments

@neelayu
Copy link
Contributor

neelayu commented Nov 21, 2022

Currently, when the telegraf starts with --config-directory flag and with --watch-config=<poll | inotify>, it expects *.conf files to be present in that directory. That means, the watch events for only those files are recorded which are already present at the time of telegraf execution.

In case, we add a new file to the directory, telegraf doesn't pick up that change. This is because, the tail api, which the telegraf uses, doesn't emit that event back to the agent. One workaround is to manually send a SIGHUP and the telegraf will detect that file creation and run as desired.

So I have done a small POC which makes the necessary changes in the influxdata/tail repo which allows the watcher to record "CREATE" event and emit it back. This solution will work well in case of inotify watcher type. For poll it requires more changes than anticipated.
Tasks done-

PS: I am creating this issue here as there is no way to create one in influxdata/tail.

cc @powersj

@Hipska Hipska added area/agent feature request Requests for new plugin and for new features to existing plugins labels Nov 30, 2022
@powersj
Copy link
Contributor

powersj commented Dec 8, 2022

next steps: get sven's review on the 2nd PR

overall I think we as of now we would be good with this addition in telegraf

@goswamisandeep
Copy link
Contributor

goswamisandeep commented Dec 14, 2022

additional ask , it should work within a docker container @neelayu

@powersj powersj self-assigned this May 29, 2024
powersj added a commit to powersj/telegraf that referenced this issue Jul 19, 2024
This allows for watching for new files in a configuraiton directory.
When a new file is found, Telegraf will reload with that file added to
the list of files to watch. Note this will reload when any file, a
telegraf config or not, is added to the directory.

TODO: notify type poll does not seem to work, only inotify
TODO: test on Windows

fixes: influxdata#12389
fixes: influxdata#12264
powersj added a commit to powersj/telegraf that referenced this issue Jul 23, 2024
This allows for watching for new files in a configuraiton directory.
When a new file is found, Telegraf will reload with that file added to
the list of files to watch. Note this will reload when any file, a
telegraf config or not, is added to the directory.

TODO: test on Windows

fixes: influxdata#12389
fixes: influxdata#12264
@powersj
Copy link
Contributor

powersj commented Jul 23, 2024

@neelayu,

Thanks to your tail PRs, I have a PR that adds the ability to watch directories for addition and removal of configuration files: #15645 could you download the artifacts in that PR and verify that this resolves your use-case?

Thanks!

@powersj powersj added the waiting for response waiting for response from contributor label Jul 23, 2024
@neelayu
Copy link
Contributor Author

neelayu commented Jul 23, 2024

@powersj Thanks for taking this up! I will test out and let you know.

@telegraf-tiger telegraf-tiger bot removed the waiting for response waiting for response from contributor label Jul 23, 2024
@neelayu
Copy link
Contributor Author

neelayu commented Jul 24, 2024

Working fine for me.

@powersj
Copy link
Contributor

powersj commented Jul 24, 2024

Fantastic, thank you for checking it out!

powersj added a commit to powersj/telegraf that referenced this issue Jul 24, 2024
This allows for watching for new files in a configuraiton directory.
When a new file is found, Telegraf will reload with that file added to
the list of files to watch. Note this will reload when any file, a
telegraf config or not, is added to the directory.

TODO: test on Windows

fixes: influxdata#12389
fixes: influxdata#12264
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/agent feature request Requests for new plugin and for new features to existing plugins
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants