Reporting other/custom/redis information #155
Comments
|
Hi @LukeHandle , at the moment there is no support for custom reports, plugins, third party scripts, etc. Mainly because there hasn't been any request about that. The current structure of And also this other issue is related #115 Let me dig into this and see how feasible this is. Iideally plugins/scripts could be written and called through recap when enabled through the configuration and produce the reports for each one of those. |
|
Seems like this will be possible, I've started migrating some of the core functions outside the I'll keep working on this and updating. |
|
I've been having a mess around with some useful Redis commands (not sure if that function should be added to the core, or separate of the code in this project).
run off by default: I'm happy to write a draft module if that helps? |
|
The use of Since is not yet concrete how the non-core modules/plugins will be loaded (I'm thinking redis will be the first one), lets wait on the coding for now. But definitely love to have contributors for this project :) Thanks! |
|
Made some progress by fixing #115 it is important to avoid that dependency so the new reports can be dynamically archived/backed up/emailed/etc, I made some testing and documented in that issue: #115 (comment) |
|
Sorry I forgot to update last week that I made good progress in here, tonyskapunk/recap:issue155 has a new commit, in here the functionality to run plugins has been added, I also included an example plugin. The README has been updated and includes the requirements to write a new plugin. I tested the plugin and the creation of logs should work, I did not test the backup/snapshot features, but I'd expect them to work just fine per the previous commits. It still needs to test recaplog to ensure is archiving the old logs, I don't expect issues in there either. Once I've spend some time on testing I'll create the PR to get this going. |
|
TL;DR Had to make a small change to fix report names detection, after that the tests came successfully, I created PR #158 to get this one rolling. @LukeHandle could you please do some testing on your end making sure this is working as expected? The README should include the information needed around Plugins, specifically hereTest results recapRecap runExecution# recap
# ls -1 /var/log/recap/*20180601-105055*
/var/log/recap/fdisk_20180601-105055.log
/var/log/recap/kernel_cmd_20180601-105055.log
/var/log/recap/netstat_20180601-105055.log
/var/log/recap/ps_20180601-105055.log
/var/log/recap/pstree_20180601-105055.log
/var/log/recap/resources_20180601-105055.logLogsSnapshots:Execution# recap -S
# ls -1 /var/log/recap/snapshots/
fdisk_20180601-105156.log_snapshot
kernel_cmd_20180601-105156.log_snapshot
netstat_20180601-105156.log_snapshot
ps_20180601-105156.log_snapshot
pstree_20180601-105156.log_snapshot
resources_20180601-105156.log_snapshotLogsBackupExecution# recap -B
# ls -1 /var/log/recap/backups/
fdisk_20180601-105055.log
kernel_cmd_20180601-105055.log
netstat_20180601-105055.log
ps_20180601-105055.log
pstree_20180601-105055.log
resources_20180601-105055.logLogsrecaplogArchiving (and compressing)Archiving the previous day## Mimicking yesterday's logs:
# for f in *20180601*.log; do cp ${f} ${f/20180601/20180531}; done
# recaplog
# ls -1 /var/log/recap/*.gz
/var/log/recap/fdisk_daily_20180531.log.tar.gz
/var/log/recap/kernel_cmd_daily_20180531.log.tar.gz
/var/log/recap/netstat_daily_20180531.log.tar.gz
/var/log/recap/ps_daily_20180531.log.tar.gz
/var/log/recap/pstree_daily_20180531.log.tar.gz
/var/log/recap/resources_daily_20180531.log.tar.gzLogsDeleting older logsRemoving logs older than 15 days## Mimicking 17 days old logs
# age_17d=$( date -d 'now -17 day' '+%Y%m%d' ); echo ${age_17d}
20180515
# for f in *20180601*.log; do cp ${f} ${f/20180601/${age_17d}}; done
# for f in *${age_17d}*; do touch -m -d ${age_17d} $f; done
# recaplog
2018-06-01 11:09:21-05:00 [ERROR] Unable to archive unexisting reports.
# ls -1 /var/log/recap/*${age_17d}*
ls: cannot access '/var/log/recap/*20180515*': No such file or directoryThe ERROR above is expected, since we have already archived the logs from the day before. Also the fact there are no files for that date is a confirmation of recaplog working. Logs |
|
Hey @tonyskapunk, currently looking at this. I've written a basic Redis plugin for testing though will likely raise a PR for discussion on methods being used. Some of the |
|
I've opened tonyskapunk#2 over on your actual account as it doesn't make sense to PR against here until #158 is merged? I can rebase against master instead if preferred as well? === Please let me know any changes you would like me to make etc. I have tried to follow standard you have used elsewhere, but may have slipped etc. |
|
We keep moving on this one, I tested the PR tonyskapunk#2 and left some messages in there, I think is good approach try to include that plugin in the PR for this issue and that's the best way I can think of. I will remove one of the plugins in there
|
|
Sorry this is taking longer but since there are so many changes I want to ensure others review them. Since there are quite good amount of changes in |
|
The PR was up for quite a long time and I don't want to hold this more. I'm happy with the testing so far, the introduction of CI (through travis-ci) should help this project to do some automated validations so the person validating does not have to do everything on her own. I'm closing this since is now in development. Thanks so much for your help @LukeHandle !! |
Is there interest in adding in other reporting aspects - or ability for a custom reporting?
I'm particularly interested in Redis stats (eg.
redis-cli infoor just vianc), but the rotation and file organization thatrecapuses is great.Maybe a custom report option allowing command "X" to run as part of the report to file name "Y"? Is this of interest, or entirely out of scope 😄
The text was updated successfully, but these errors were encountered: