Configuring Butler for Qliksense Alert

[jpjustus]

Hi,
I am trying to configure Butler for Qliksense with the intention of receiving alert email only when reload tasks in QMC fails.
I have done the following steps:

1. Install node.js
2. Install Butler

Now I have qliksense certificate exported and template.yaml file (renamed to production.yaml) configured only for email alert portion. I have left untouched for slack, MS teams.
From the documentation I understand that I should copy the certificates and the yaml file to /src/config.
Can you please assist me where I should exactly copy the file where Butler is installed ?
I have already set NODE_ENV=production

Once the files are copied over, I assume that the next step is to start Butler.

Thanks,
John

[mountaindude]

It sounds like you will run Butler as a regular Node.js app and not as a Docker container - correct?

You can place the certificates anywhere, as long as Butler can access them.
Personally I usually export a set of certificates from the QMC, then place them in a central location (like d:\secret\<servername>) on the server where Butler will run. That way the certificates can be shared between Butler, Butler SOS, Butler Auth, Butler Sheet Icons etc.

You'd then refer to that location from your production.yaml file.

Looking at the standard directory structure of Butler it looks like this (some subdirectories and files removed for clarity).

├── README.md
├── changelog.md
├── docs
│   ├── grafana
│   ├── log4net_task-failed
│   ├── log4net_user-audit-event
│   ├── sense_apps
│   └── sense_script
├── icon.png
└── src
    ├── butler.js
    ├── config
    │   ├── email_templates
    │   │   ├── aborted-reload.handlebars
    │   │   └── failed-reload.handlebars
    │   ├── production.yaml
    │   ├── production_template.yaml
    │   ├── schedule.yaml
    │   ├── schedule_template.yaml
    │   ├── slack_templates
    │   │   ├── aborted-reload.handlebars
    │   │   └── failed-reload.handlebars
    │   └── teams_templates
    │       ├── aborted-reload.handlebars
    │       └── failed-reload.handlebars
    ├── docker
    ├── docker-compose.yaml
    ├── docker-healthcheck.js
    ├── globals.js
    ├── k8s
    ├── lib
    │   ├── disk_utils.js
    │   ├── heartbeat.js
    │   ├── incident_mgmt
    │   │   └── signl4.js
    │   ├── logRESTCall.js
    │   ├── msteams_notification.js
    │   ├── post-to-influxdb.js
    │   ├── scheduler.js
    │   ├── scriptlog.js
    │   ├── service_uptime.js
    │   ├── slack_api.js
    │   ├── slack_notification.js
    │   ├── smtp.js
    │   ├── telemetry.js
    │   └── webhook_notification.js
    ├── log
    ├── mqtt
    │   ├── index.js
    │   └── mqtt_handlers.js
    ├── node_modules
    ├── package-lock.json
    ├── package.json
    ├── qrs_util
    │   ├── getAppOwner.js
    │   ├── index.js
    │   └── senseStartTask.js
    ├── rest
    │   ├── activeUserCount.js
    │   ├── activeUsers.js
    │   ├── api.js
    │   ├── baseConversion.js
    │   ├── butlerPing.js
    │   ├── disk_utils.js
    │   ├── getDiskSpace.js
    │   ├── index.js
    │   ├── keyValueStore.js
    │   ├── mqttPublishMessage.js
    │   ├── scheduler.js
    │   ├── senseApp.js
    │   ├── senseAppDump.js
    │   ├── senseListApps.js
    │   ├── senseQRSPing.js
    │   ├── senseStartTask.js
    │   └── slackPostMessage.js
    ├── udp
    │   ├── index.js
    │   └── udp_handlers.js
    └── udp_client
        ├── README.md
        ├── node_modules
        ├── package-lock.json
        ├── package.json
        └── udp_client.js 

If you want to keep the cert files within the Butler directory structure you could for example place them in src/config/cert, and then point the Butler.cert.* values in the production.yaml file to the cert files.

Finally, to get alert emails working you also need to add (and edit) the XML log appender files to the server where the Sense scheduler runs.
Make sure to read the docs about this, https://butler.ptarmiganlabs.com/docs/getting-started/setup/reload-alerts/

[jpjustus]

Hi - I am revisiting to configure Butler again. So I have done all the configuration steps.
And when I started Butler on windows, the command prompt window just stays here and don't move forward.
Can you please take a look at the attached and let me know? If you need any config files from me or any other information, please let me know. I really want this to work in our environment. Thank you so much!
Butler.docx

[mountaindude]

Well - that's actually looks pretty good.

You don't have any errors in the log, and with logging level set to info, you will not get a lot of log output.
If you want to ensure things are working, you can enable the uptime monitor in the config file, and set logging level to verbose.

And: Please consider upgrading to the most recent version. A lot of goodness happened in v7, an upgrade is strongly recommended.
Admittedly, v7 did bring some breaking changes that will require some (minor) changes in Butler's config file and Sense scripts using Butler APIs. But if you go for v7 now you don't have to migrate later on.

Your call of course, but my recommendation is on v7 for sure.

[mountaindude]

Another way of verifying that Butler is actually running is to enable the ping API endpoint.
This is done in the YAML config file (the example below is for Butler v6.1.1, but is the same for v7):

Butler:
...
...
  restServerEndpointsEnable:
    ...
    butlerping: true
...
...

In my case Butler is running on my laptop (which has IP 192.168.1.168) on port 8080, so.... using a web browser I go to http://192.168.1.168:8080/v4/butlerping and get

Yup. Butler is running and doing well.

[jpjustus]

Thank you so much for the detailed information. So from your comment, everything looks good. When I have an reload task failure I am not getting any emails. I have configured the LocalLogConfig.xml file as per the instructions.
Also, I tried to test the ping but I get HTTP Error 4040.0 - Not Found. I have enabled API point on YAML config file.
Can you please advice?

Thanks,
John

[mountaindude]

John,

If you don't get a response on the ping endpoint you are using the wrong IP or port, or have a firewall blocking.

The alert emails are a slightly different story.
The XML appender files deployed on the Sense server has to be correct. They must contain the IP and port of the UDP server where Butler is running. And that Butler IP/port must be reachable from the Sense server(s) - check firewalls etc. Or simply test by using tools like ping or telnet - if Sense cannot send those UDP messages with info about failed tasks, Butler of course won't send any emails.

I'm happy to help in what (reasonable) ways I can, to do more at this point I need to understand what your full config file looks like.
If you are happy posting it here that's prob easiest. You can always put in some dummy IP if you want to.
Or send it to info ptarmiganlabs com, and I'll take a look.
As you don't get any errors in the log my feeling is that you are close - let's fix this together!

[jpjustus]

Thank you so much for the encouragement, I hope we are so close and may be with few additional tweaks, we can fix it.
By the way, I sent the config files to 'info@ptarmiganlabs.com'. Can you please take a look and let me know if there is anything wrong?

Thank you,
John