Windows Version? Docker has problem

[gogito]

So I tried this on Docker using Windows 10 Home with WSL2 backend and it's having problem with node_exporter can't mount the / path as shared/slave.

I saw that there is a Windows Exporter for Prometheus available but I'm not well verse enough to work on it. If possible can you try and make a Windows version?

Thank you for the great work!

[retzkek]

Yes, you'd need to use the Windows exporter, which should be easy to install with their .msi installer. Then you should just have to point Prometheus at port 9182, although there may be some issues with host-to-container networking on Windows. If so, you'd probably want to download and run the Prometheus native Windows binary instead of running it in Docker.

If possible can you try and make a Windows version?

I rarely use Windows, so I probably won't be able to explore this further, sorry.

[digitalkisan]

With retzkek help in the instructions have installed some of this on windows as a total novice, however the data links are failing as follows:

[digitalkisan]

I have commented the node_exporter part out of the docker-compose.yml
and
Edited prometheus/prometheus.yml, change localhost:9100 to localhost:9182 for the "node" job

Windows Exporter appears to be running and I have pointed mtail to the mtail/chialog.mtail file within the windows app

[gogito]

I have commented the node_exporter part out of the docker-compose.yml
and
Edited prometheus/prometheus.yml, change localhost:9100 to localhost:9182 for the "node" job

Windows Exporter appears to be running and I have pointed mtail to the mtail/chialog.mtail file within the windows app

By the "node" job you mean Windows Exporter? And I'm a bit confused on how you are setting this up. Are you setting up everything natively or using docker (I see you included the bit that you commented the node_exporter part out of docker-compose.yml)

[digitalkisan]

I installed docker and took the package that was shared by the retzkek, then I installed windows exporter using Github link shared.
made the changes above in the files to try to get it to work :-) that's about where I am up to

[gogito]

I installed docker and took the package that was shared by the retzkek, then I installed windows exporter using Github link shared.
made the changes above in the files to try to get it to work :-) that's about where I am up to

I'm trying this myself but haven't gotten far. Hope we can make this work :).

[retzkek]

Alright, I fired up a Windows desktop to see if I could get this running. Buckle up, this will be a bumpy ride!

All changes and a new dashboard are pushed to the windows branch.

• Install Docker Desktop
• Install Visual Studio Code
• Install git
• Install Windows exporter
• Clone the chiamon repository with VSCode
• Modify docker-compose.yml:
  • Comment out the node_exporter service
  • Change ${HOME} to my home directory. There's probably a way to set the environment variable but meh.
  • Expose ports for prometheus and grafana services. This isn't needed on Linux since network_mode is set to host but seems to be required on Windows (using host mode was a lazy hack anyways).
  • Add -url argument for chia_exporter to connect to Chia node at host address, which Docker sets to host.docker.internal (see docker networking in Windows)
• Modify prometheus.yml:
  • Change localhost:9100 to host.docker.internal:9182 for the node job
  • Change localhost to service names for other services
• Run services. In VSCode with docker extension you can just right-click on docker-compose.yml and select "Compose Up"
• Check target status in Prometheus at http://localhost:9090/targets
• Access Grafana at http://localhost:3000 (admin/admin). Lots of changes here:
  • Change "prometheus" datasource to connect to address http://prometheus:9090
  • Replace all node_ metrics with equivalent windows_ metrics. I didn't explore too much what the Windows exporter has to offer, just did some quick replacements, some of them are probably even wrong, so please check.

I installed Chia, but it's not farming, so I can't check that those metrics are working. Otherwise it looks good, proof:

[digitalkisan]

Could you share your docker-compose for windows just so I can check I'm am adding the entries correctly, then I will start to work out how to make the changes to the grafana portion. slow and steady!

Thanks for your awesome work and the sharing

[retzkek]

All the files I changed for Windows, including the dashboard, are in the "windows" branch: https://github.com/retzkek/chiamon/tree/windows

[digitalkisan]

All the files I changed for Windows, including the dashboard, are in the "windows" branch: https://github.com/retzkek/chiamon/tree/windows

Following these instructions I seem to have it working, just got it running so assuming it will start pulling the plotting data through soon. Really nice dash!

[gromran]

is there a way to use this without docker- and vscode-shit?

[retzkek]

@gromran yes.

[digitalkisan]

Quick Update - appears to be working for all machine metrics, however does not seem to be pulling Chia specifics like proofs or plots

[gogito]

Quick Update - appears to be working for all machine metrics, however does not seem to be pulling Chia specifics like proofs or plots

That's great progress. Hope you can sort it out soon. Though why is the CPU reaching 2000%+? XD

[retzkek]

however does not seem to be pulling Chia specifics like proofs or plots

@digitalkisan those metrics come from the mtail program scanning the chia logs. Some things to check:

• is the path to the chia log in the compose file correct?
• is mtail running? hint: docker-compose ps and docker-compose logs mtail
• is the chia log level set to INFO?

Though why is the CPU reaching 2000%+? XD

@gogito like I said, I quickly replaced the metrics, so they may not be right. Someone who actually uses Windows can figure out the right metrics. I'm not going to spoonfeed everyone, sorry.

[digitalkisan]

@digitalkisan those metrics come from the mtail program scanning the chia logs. Some things to check:

• is the path to the chia log in the compose file correct? Yes the path is correct, but the log level had reverted to warning so I have changed this back to info
• is mtail running? hint: docker-compose ps and docker-compose logs mtail have restarted everything and run it via the commandline shared
• is the chia log level set to INFO? It was on warning changed to info again now

Though why is the CPU reaching 2000%+? XD - no idea tried to change the graph axis from 0.0-1.0 to 0-100% this then shows CPU usage at 25% which it isn't but like retzkek mentioned maybe someone who is using this on windows can jump into this thread and help us all so capturing as many things as i find along the way to hopefully help everyone.

@gogito like I said, I quickly replaced the metrics, so they may not be right. Someone who actually uses Windows can figure out the right metrics. I'm not going to spoonfeed everyone, sorry.

Thanks again guys for your help a screenshot of mtail below

[SiliPSX]

The Expressions for the network usage is wrong.
Should be like this:
Received bytes:
"expr": "sum(increase(windows_net_bytes_received_total{nic=~\"$interfaces\"}[24h]))",
Sent bytes:
"expr": "sum(increase(windows_net_bytes_sent_total{nic=~\"$interfaces\"}[24h]))",

[SiliPSX]

Now i also got the mtail part working.
In the chialog.mtail File delete the $ after plots
Before: Total (?P<total_plots>\d+) plots$/
After: Total (?P<total_plots>\d+) plots/

Now it looks good:

[digitalkisan]

Thank you, i will make those changes and report back, also after observing the dash i noticed the CPU % is actually the inverse, so it is showing % of CPU utilisation remaining this is best seen if you idle the cpu and run the dash.

[SiliPSX]

Ok, i see that $ is for the line ending....
this will work in Windows: (\r\n|\r|\n)/
so the line should be .....Total (?P<total_plots>\d+) plots(\r\n|\r|\n)/

[retzkek]

@SiliPSX thanks for investigating! If you want to open a PR against the windows branch I'd happily merge your changes.

[digitalkisan]

Thanks Retzkek and SiliPSX for your hardwork!

[yanadhorn]

The Expressions for the network usage is wrong.
Should be like this:
Received bytes:
"expr": "sum(increase(windows_net_bytes_received_total{nic=~\"$interfaces\"}[24h]))",
Sent bytes:
"expr": "sum(increase(windows_net_bytes_sent_total{nic=~\"$interfaces\"}[24h]))",

where to change this?

[yanadhorn]

Now i also got the mtail part working.
In the chialog.mtail File delete the $ after plots
Before: Total (?P<total_plots>\d+) plots$/
After: Total (?P<total_plots>\d+) plots/

Now it looks good:

and do you have trick how to get a lot of peer like that

[SiliPSX]

The Network Fix is in chiamon/grafana/dashboards/Chia on Windows.json

For the peers you have to open Port 8444 in your router and forward it to your node.

[yanadhorn]

The Network Fix is in chiamon/grafana/dashboards/Chia on Windows.json

For the peers you have to open Port 8444 in your router and forward it to your node.

is it line 978 right , i already copy ur code but still not working. do i need to do something in Dashboard when running. or restart some service?

[yanadhorn]

  "pluginVersion": "7.5.5",
  "targets": [
    {
      "exemplar": true,
      "expr": "sum(increase(windows_net_bytes_received_total{nic=~\"$interfaces\"}[24h]))",
      "interval": "",
      "intervalFactor": 1,
      "legendFormat": " rx",
      "metric": "node_network_receive_bytes",
      "refId": "A",
      "step": 10
    },
    {
      "exemplar": true,
      "expr": "sum(increase(windows_net_bytes_sent_total{nic=~\"$interfaces\"}[24h]))",
      "interval": "",
      "intervalFactor": 1,
      "legendFormat": "tx",
      "metric": "node_network_transmit_bytes",
      "refId": "B",
      "step": 10
    }

this is mine.

[yanadhorn]

work now i change the code and then come post down and then up again.

[gogito]

@yanadhorn @SiliPSX Is it possible for you guys to have a tutorial/git page that can help us with less technical knowledge on how to set this up for Windows?

[yanadhorn]

@yanadhorn @SiliPSX Is it possible for you guys to have a tutorial/git page that can help us with less technical knowledge on how to set this up for Windows?

hey bro. i can help as much as i can what is your problem right now tell me.

[felipecaldas]

Am I meant to see anything running after I install Windows Exporter? I clicked the .msi file, installed it but I can't see anything running on my windows tray, I don't even know where it got installed.

The chia_exporter container won't start up:

2021/05/16 03:51:18 chia_exporter version 0.2

2021/05/16 03:51:20 error calling get_network_info: Post "http://host.docker.internal:8555/get_network_info": dial tcp 192.168.65.2:8555: connect: connection refused

What should be running on port 8555?

Also, this error happens when I don't have the Chia GUI running.

If I run it, then I get this error when initializing chia_exporter docker container:

2021/05/16 04:11:22 error calling get_network_info: Post "http://host.docker.internal:8555/get_network_info": EOF

[felipecaldas]

Btw, going to this URL http://localhost:9182/metrics I can see information there, I am guessing that's Windows Exporter?

[retzkek]

What should be running on port 8555?

The full node RPC API, so yes you need the Chia node running.

2021/05/16 04:11:22 error calling get_network_info: Post "http://host.docker.internal:8555/get_network_info": EOF

That URL needs to use https not http. The example compose file looks right, did you change it?

Btw, going to this URL http://localhost:9182/metrics I can see information there, I am guessing that's Windows Exporter?

yes

[felipecaldas]

What should be running on port 8555?

The full node RPC API, so yes you need the Chia node running.

2021/05/16 04:11:22 error calling get_network_info: Post "http://host.docker.internal:8555/get_network_info": EOF

That URL needs to use https not http. The example compose file looks right, did you change it?

Btw, going to this URL http://localhost:9182/metrics I can see information there, I am guessing that's Windows Exporter?

yes

you legend, thanks. All containers running now.

If my full node runs on a different machine than chiamon, then all I have to do is to change:

"-url"
      - "https://host.docker.internal:8555"
      - "-wallet"
      - "https://host.docker.internal:9256"

more specifically, to change host.docker.internal to the full node's IP address, right?

[retzkek]

more specifically, to change host.docker.internal to the full node's IP address, right?

The RPC APIs only listen on localhost by default, so you can't connect from another host. It looks like you can change that with the self_hostname config, but I wouldn't recommend doing that (bad security practice, and may have some other implications). Instead I would run the chia_exporter on that node, and change the prometheus config to scrape it there.

[digitalkisan]

Now i also got the mtail part working.
In the chialog.mtail File delete the $ after plots
Before: Total (?P<total_plots>\d+) plots$/
After: Total (?P<total_plots>\d+) plots/

Now it looks good:

Still can't seem to get the plots to show even though i have implemented the change to remove the "$" any ideas?

[blazeruk1]

Now i also got the mtail part working.
In the chialog.mtail File delete the $ after plots
Before: Total (?P<total_plots>\d+) plots$/
After: Total (?P<total_plots>\d+) plots/
Now it looks good:

Still can't seem to get the plots to show even though i have implemented the change to remove the "$" any ideas?

Same here

[digitalkisan]

here is the output from mtail

PS C:\Users\Digit\OneDrive\Documents\GitHub\chiamon> docker-compose logs mtail
Attaching to chiamon_mtail_1
mtail_1 | I0509 17:03:11.604069 1 main.go:114] mtail version v3.0.0-rc45-57-g8f0b94fa git revision 8f0b94fad7f6c8359e31a01d7ee72e8312d915a3 go version go1.16.4 go arch amd64 go os linux
mtail_1 | I0509 17:03:11.604166 1 main.go:115] Commandline: ["/usr/bin/mtail" "-progs" "/etc/mtail" "-logs" "/var/log/chia/debug.log" "-logtostderr"]
mtail_1 | I0509 17:03:11.604579 1 store.go:178] Starting metric store expiry loop every 1h0m0s
mtail_1 | I0509 17:03:11.605466 1 runtime.go:175] Loaded program chialog.mtail
mtail_1 | I0509 17:03:11.609478 1 tail.go:259] Tailing /var/log/chia/debug.log
mtail_1 | I0509 17:03:11.609602 1 mtail.go:197] Listening on [::]:3903
mtail_1 | I0509 18:03:11.605073 1 store.go:153] Running Store.Expire()
mtail_1 | I0509 19:03:11.605509 1 store.go:153] Running Store.Expire()
mtail_1 | I0509 20:03:11.604985 1 store.go:153] Running Store.Expire()
mtail_1 | I0509 21:03:11.605432 1 store.go:153] Running Store.Expire()
mtail_1 | I0509 22:03:11.604983 1 store.go:153] Running Store.Expire()
mtail_1 | I0509 23:03:11.605052 1 store.go:153] Running Store.Expire()
mtail_1 | I0510 00:03:11.605328 1 store.go:153] Running Store.Expire()
mtail_1 | I0510 01:03:11.605121 1 store.go:153] Running Store.Expire()
mtail_1 | I0510 02:03:11.605405 1 store.go:153] Running Store.Expire()
mtail_1 | I0510 03:03:11.605239 1 store.go:153] Running Store.Expire()
mtail_1 | I0510 04:03:11.605507 1 store.go:153] Running Store.Expire()
mtail_1 | I0510 05:03:11.604848 1 store.go:153] Running Store.Expire()
mtail_1 | I0510 06:03:11.605003 1 store.go:153] Running Store.Expire()
mtail_1 | I0510 08:33:23.242371 1 main.go:114] mtail version v3.0.0-rc45-57-g8f0b94fa git revision 8f0b94fad7f6c8359e31a01d7ee72e8312d915a3 go version go1.16.4 go arch amd64 go os linux
mtail_1 | I0510 08:33:23.242411 1 main.go:115] Commandline: ["/usr/bin/mtail" "-progs" "/etc/mtail" "-logs" "/var/log/chia/debug.log" "-logtostderr"]
mtail_1 | I0510 08:33:23.243119 1 store.go:178] Starting metric store expiry loop every 1h0m0s
mtail_1 | I0510 08:33:23.245913 1 runtime.go:175] Loaded program chialog.mtail
mtail_1 | I0510 08:33:23.248140 1 tail.go:259] Tailing /var/log/chia/debug.log
mtail_1 | I0510 08:33:23.248201 1 mtail.go:197] Listening on [::]:3903
mtail_1 | I0510 09:33:23.243622 1 store.go:153] Running Store.Expire()
mtail_1 | I0510 10:33:23.243540 1 store.go:153] Running Store.Expire()
mtail_1 | I0510 11:33:23.243580 1 store.go:153] Running Store.Expire()
mtail_1 | I0510 12:33:23.243626 1 store.go:153] Running Store.Expire()
mtail_1 | I0510 13:33:23.243243 1 store.go:153] Running Store.Expire()
mtail_1 | I0510 14:33:23.243292 1 store.go:153] Running Store.Expire()
mtail_1 | I0510 15:33:23.243643 1 store.go:153] Running Store.Expire()
mtail_1 | I0510 16:33:23.243580 1 store.go:153] Running Store.Expire()
mtail_1 | I0510 17:33:23.243878 1 store.go:153] Running Store.Expire()
mtail_1 | I0510 18:33:23.243280 1 store.go:153] Running Store.Expire()
mtail_1 | I0510 19:33:23.243511 1 store.go:153] Running Store.Expire()
mtail_1 | I0510 20:33:23.243269 1 store.go:153] Running Store.Expire()
mtail_1 | I0510 21:33:23.243631 1 store.go:153] Running Store.Expire()
mtail_1 | I0510 22:33:23.243390 1 store.go:153] Running Store.Expire()
mtail_1 | I0510 23:33:23.243598 1 store.go:153] Running Store.Expire()
mtail_1 | I0511 00:33:23.243469 1 store.go:153] Running Store.Expire()
mtail_1 | I0511 01:33:23.243544 1 store.go:153] Running Store.Expire()
mtail_1 | I0511 02:33:23.243227 1 store.go:153] Running Store.Expire()
mtail_1 | I0511 03:33:23.243236 1 store.go:153] Running Store.Expire()
mtail_1 | I0511 04:33:23.243570 1 store.go:153] Running Store.Expire()
mtail_1 | I0511 05:33:23.243594 1 store.go:153] Running Store.Expire()
mtail_1 | I0511 06:33:23.243383 1 store.go:153] Running Store.Expire()
mtail_1 | I0511 07:33:23.243459 1 store.go:153] Running Store.Expire()
mtail_1 | I0511 08:33:23.243830 1 store.go:153] Running Store.Expire()
mtail_1 | I0511 09:33:23.243483 1 store.go:153] Running Store.Expire()
mtail_1 | I0511 10:33:23.243675 1 store.go:153] Running Store.Expire()
mtail_1 | I0511 11:33:23.243260 1 store.go:153] Running Store.Expire()
mtail_1 | I0511 12:33:23.243294 1 store.go:153] Running Store.Expire()
mtail_1 | I0511 13:33:23.243319 1 store.go:153] Running Store.Expire()
mtail_1 | I0511 14:33:23.243236 1 store.go:153] Running Store.Expire()
mtail_1 | I0511 15:33:23.243332 1 store.go:153] Running Store.Expire()
mtail_1 | I0511 16:33:23.243221 1 store.go:153] Running Store.Expire()
mtail_1 | I0511 17:33:23.243832 1 store.go:153] Running Store.Expire()
mtail_1 | I0511 18:33:23.243747 1 store.go:153] Running Store.Expire()
mtail_1 | I0511 19:33:23.243382 1 store.go:153] Running Store.Expire()
mtail_1 | I0511 20:33:23.243608 1 store.go:153] Running Store.Expire()
mtail_1 | I0511 21:32:32.993287 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0511 21:32:33.243546 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0511 21:32:33.492956 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0511 21:32:33.743427 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0511 21:32:33.992817 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0511 21:32:34.243314 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0511 21:32:34.492983 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0511 21:32:34.743524 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0511 21:32:34.992903 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0511 21:32:35.243411 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0511 21:32:35.492777 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0511 21:32:35.743245 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0511 21:32:35.993706 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0511 21:32:36.243169 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0511 21:32:36.493656 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0511 21:32:36.743103 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0511 21:32:36.993532 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0511 21:32:37.242967 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0511 21:32:37.493384 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0511 21:32:37.742744 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0511 21:32:37.993143 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0511 21:32:38.243610 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0511 21:32:38.493560 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0511 21:32:38.742946 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0511 21:32:38.993376 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0511 21:32:39.242761 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0511 21:32:39.493274 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0511 21:32:39.743703 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0511 21:32:39.993157 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0511 21:32:40.243631 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0511 21:32:40.493160 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0511 21:32:40.743586 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0511 21:32:40.992982 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0511 21:32:41.243437 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0511 21:32:41.492766 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0511 21:32:41.743226 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0511 21:32:41.993646 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0511 21:32:42.243100 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0511 21:32:42.493571 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0511 21:32:42.742949 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0511 21:32:42.993396 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0511 21:32:43.243051 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0511 21:32:43.493476 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0511 21:32:43.742782 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0511 21:32:43.993229 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0511 21:32:44.243659 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0511 21:32:44.493371 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0511 21:32:44.742697 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0511 21:32:44.993133 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0511 21:32:45.243539 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0511 21:32:45.492929 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0511 21:32:45.743319 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0511 21:32:45.992698 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0511 21:32:46.243110 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0511 21:32:46.493534 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0511 21:32:46.742877 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0511 21:32:46.993270 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0511 21:32:47.242622 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0511 21:32:47.493074 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0511 21:32:47.743643 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0511 21:32:47.993343 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0511 21:55:11.576494 1 main.go:114] mtail version v3.0.0-rc45-57-g8f0b94fa git revision 8f0b94fad7f6c8359e31a01d7ee72e8312d915a3 go version go1.16.4 go arch amd64 go os linux
mtail_1 | I0511 21:55:11.576537 1 main.go:115] Commandline: ["/usr/bin/mtail" "-progs" "/etc/mtail" "-logs" "/var/log/chia/debug.log" "-logtostderr"]
mtail_1 | I0511 21:55:11.578692 1 store.go:178] Starting metric store expiry loop every 1h0m0s
mtail_1 | I0511 21:55:11.580743 1 runtime.go:175] Loaded program chialog.mtail
mtail_1 | I0511 21:55:11.582943 1 tail.go:259] Tailing /var/log/chia/debug.log
mtail_1 | I0511 21:55:11.583004 1 mtail.go:197] Listening on [::]:3903
mtail_1 | I0511 22:55:11.579641 1 store.go:153] Running Store.Expire()
mtail_1 | I0511 23:55:11.578767 1 store.go:153] Running Store.Expire()
mtail_1 | I0512 00:55:11.578760 1 store.go:153] Running Store.Expire()
mtail_1 | I0512 01:55:11.578749 1 store.go:153] Running Store.Expire()
mtail_1 | I0512 02:55:11.578852 1 store.go:153] Running Store.Expire()
mtail_1 | I0512 03:55:11.578875 1 store.go:153] Running Store.Expire()
mtail_1 | I0512 04:55:11.578789 1 store.go:153] Running Store.Expire()
mtail_1 | I0512 05:55:11.578805 1 store.go:153] Running Store.Expire()
mtail_1 | I0512 06:55:11.578781 1 store.go:153] Running Store.Expire()
mtail_1 | I0512 07:55:11.578812 1 store.go:153] Running Store.Expire()
mtail_1 | I0512 08:55:11.578862 1 store.go:153] Running Store.Expire()
mtail_1 | I0512 09:55:11.578880 1 store.go:153] Running Store.Expire()
mtail_1 | I0512 10:55:11.578748 1 store.go:153] Running Store.Expire()
mtail_1 | I0512 11:55:11.578783 1 store.go:153] Running Store.Expire()
mtail_1 | I0512 12:55:11.578983 1 store.go:153] Running Store.Expire()
mtail_1 | I0512 13:47:04.957955 1 main.go:156] Received terminated, exiting...
mtail_1 | I0512 13:47:04.958101 1 mtail.go:210] Shutdown requested.
mtail_1 | I0512 13:47:04.959260 1 runtime.go:270] END OF LINE
mtail_1 | I0512 13:47:04.959302 1 vm.go:1026] VM "chialog.mtail" finished
mtail_1 | I0512 13:47:05.830331 1 main.go:114] mtail version v3.0.0-rc45-57-g8f0b94fa git revision 8f0b94fad7f6c8359e31a01d7ee72e8312d915a3 go version go1.16.4 go arch amd64 go os linux
mtail_1 | I0512 13:47:05.830388 1 main.go:115] Commandline: ["/usr/bin/mtail" "-progs" "/etc/mtail" "-logs" "/var/log/chia/debug.log" "-logtostderr"]
mtail_1 | I0512 13:47:05.830795 1 store.go:178] Starting metric store expiry loop every 1h0m0s
mtail_1 | I0512 13:47:05.831600 1 runtime.go:175] Loaded program chialog.mtail
mtail_1 | I0512 13:47:05.835185 1 tail.go:259] Tailing /var/log/chia/debug.log
mtail_1 | I0512 13:47:05.835273 1 mtail.go:197] Listening on [::]:3903
mtail_1 | I0512 14:47:05.830975 1 store.go:153] Running Store.Expire()
mtail_1 | I0512 15:47:05.830861 1 store.go:153] Running Store.Expire()
mtail_1 | I0512 16:47:05.830927 1 store.go:153] Running Store.Expire()
mtail_1 | I0512 17:47:05.830893 1 store.go:153] Running Store.Expire()
mtail_1 | I0512 18:47:05.830909 1 store.go:153] Running Store.Expire()
mtail_1 | I0512 19:47:05.830938 1 store.go:153] Running Store.Expire()
mtail_1 | I0512 20:47:05.830870 1 store.go:153] Running Store.Expire()
mtail_1 | I0512 20:49:04.786125 1 filestream.go:131] read /var/log/chia/debug.log: bad address
mtail_1 | I0512 20:49:04.831454 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0512 20:49:05.081260 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0512 20:49:05.331803 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0512 20:49:05.581262 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0512 20:49:05.831174 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0512 20:49:06.081563 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0512 20:49:06.331218 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0512 20:49:06.581596 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0512 20:49:06.830953 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0512 20:49:07.081355 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0512 20:49:07.330724 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0512 20:49:07.581272 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0512 20:49:07.831744 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0512 20:49:08.081164 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0512 20:49:08.331601 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0512 20:49:08.581151 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0512 20:49:08.831625 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0512 20:49:09.081016 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0512 20:49:09.331476 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0512 20:49:09.580858 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0512 20:49:09.831415 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0512 20:49:10.080835 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0512 20:49:10.331292 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0512 20:49:10.581824 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0512 20:49:10.831566 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0512 20:49:11.080962 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0512 20:49:11.331404 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0512 20:49:11.580733 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0512 20:49:11.831174 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0512 20:49:12.081650 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0512 20:49:12.331030 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0512 20:49:12.581454 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0512 20:49:12.830802 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0512 20:49:13.081236 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0512 20:49:13.331669 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0512 20:49:13.581120 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0512 20:49:13.831532 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0512 20:49:14.081325 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0512 20:49:14.330970 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0512 20:49:14.581389 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0512 20:49:14.830731 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0512 20:49:15.081293 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0512 20:49:15.331733 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0512 20:49:15.581167 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0512 20:49:15.831724 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0512 20:49:16.081247 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0512 20:49:16.331688 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0512 20:49:16.581091 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0512 20:49:16.831531 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0512 20:49:17.081100 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0512 20:49:17.331327 1 filestream.go:131] read /var/log/chia/debug.log: input/output error
mtail_1 | I0512 23:26:11.229320 1 main.go:114] mtail version v3.0.0-rc45-57-g8f0b94fa git revision 8f0b94fad7f6c8359e31a01d7ee72e8312d915a3 go version go1.16.4 go arch amd64 go os linux
mtail_1 | I0512 23:26:11.229366 1 main.go:115] Commandline: ["/usr/bin/mtail" "-progs" "/etc/mtail" "-logs" "/var/log/chia/debug.log" "-logtostderr"]
mtail_1 | I0512 23:26:11.230498 1 store.go:178] Starting metric store expiry loop every 1h0m0s
mtail_1 | I0512 23:26:11.235754 1 runtime.go:175] Loaded program chialog.mtail
mtail_1 | I0512 23:26:11.238898 1 tail.go:259] Tailing /var/log/chia/debug.log
mtail_1 | I0512 23:26:11.238952 1 mtail.go:197] Listening on [::]:3903
mtail_1 | I0513 00:26:11.230557 1 store.go:153] Running Store.Expire()
mtail_1 | I0513 01:26:11.230739 1 store.go:153] Running Store.Expire()
mtail_1 | I0513 02:26:11.230625 1 store.go:153] Running Store.Expire()
mtail_1 | I0513 03:26:11.230857 1 store.go:153] Running Store.Expire()
mtail_1 | I0513 04:26:11.230547 1 store.go:153] Running Store.Expire()
mtail_1 | I0513 05:26:11.230681 1 store.go:153] Running Store.Expire()
mtail_1 | I0513 06:26:11.230535 1 store.go:153] Running Store.Expire()
mtail_1 | I0513 07:26:11.230546 1 store.go:153] Running Store.Expire()
mtail_1 | I0513 08:26:11.230628 1 store.go:153] Running Store.Expire()
mtail_1 | I0513 09:26:11.230630 1 store.go:153] Running Store.Expire()
mtail_1 | I0513 10:26:11.230599 1 store.go:153] Running Store.Expire()
mtail_1 | I0513 11:26:11.230586 1 store.go:153] Running Store.Expire()
mtail_1 | I0513 12:26:11.230548 1 store.go:153] Running Store.Expire()
mtail_1 | I0513 13:26:11.230569 1 store.go:153] Running Store.Expire()
mtail_1 | I0513 14:26:11.230706 1 store.go:153] Running Store.Expire()
mtail_1 | I0513 15:26:11.230552 1 store.go:153] Running Store.Expire()
mtail_1 | I0513 16:26:11.230557 1 store.go:153] Running Store.Expire()
mtail_1 | I0513 17:23:18.196745 1 main.go:156] Received terminated, exiting...
mtail_1 | I0513 17:23:18.196821 1 mtail.go:210] Shutdown requested.
mtail_1 | I0513 17:23:18.197261 1 runtime.go:270] END OF LINE
mtail_1 | I0513 17:23:18.197285 1 vm.go:1026] VM "chialog.mtail" finished
mtail_1 | I0513 17:23:19.319989 1 main.go:114] mtail version v3.0.0-rc45-57-g8f0b94fa git revision 8f0b94fad7f6c8359e31a01d7ee72e8312d915a3 go version go1.16.4 go arch amd64 go os linux
mtail_1 | I0513 17:23:19.320027 1 main.go:115] Commandline: ["/usr/bin/mtail" "-progs" "/etc/mtail" "-logs" "/var/log/chia/debug.log" "-logtostderr"]
mtail_1 | I0513 17:23:19.320378 1 store.go:178] Starting metric store expiry loop every 1h0m0s
mtail_1 | I0513 17:23:19.321115 1 runtime.go:175] Loaded program chialog.mtail
mtail_1 | I0513 17:23:19.324123 1 tail.go:259] Tailing /var/log/chia/debug.log
mtail_1 | I0513 17:23:19.324188 1 mtail.go:197] Listening on [::]:3903
mtail_1 | I0513 17:24:17.002835 1 main.go:156] Received terminated, exiting...
mtail_1 | I0513 17:24:17.002988 1 mtail.go:210] Shutdown requested.
mtail_1 | I0513 17:24:17.003791 1 runtime.go:270] END OF LINE
mtail_1 | I0513 17:24:17.003837 1 vm.go:1026] VM "chialog.mtail" finished
mtail_1 | I0513 17:24:20.180950 1 main.go:114] mtail version v3.0.0-rc45-57-g8f0b94fa git revision 8f0b94fad7f6c8359e31a01d7ee72e8312d915a3 go version go1.16.4 go arch amd64 go os linux
mtail_1 | I0513 17:24:20.180991 1 main.go:115] Commandline: ["/usr/bin/mtail" "-progs" "/etc/mtail" "-logs" "/var/log/chia/debug.log" "-logtostderr"]
mtail_1 | I0513 17:24:20.181263 1 store.go:178] Starting metric store expiry loop every 1h0m0s
mtail_1 | I0513 17:24:20.181752 1 runtime.go:175] Loaded program chialog.mtail
mtail_1 | I0513 17:24:20.184630 1 tail.go:259] Tailing /var/log/chia/debug.log
mtail_1 | I0513 17:24:20.184750 1 mtail.go:197] Listening on [::]:3903
mtail_1 | I0513 18:24:20.182219 1 store.go:153] Running Store.Expire()
mtail_1 | I0513 19:24:20.182133 1 store.go:153] Running Store.Expire()
mtail_1 | I0513 20:24:20.182008 1 store.go:153] Running Store.Expire()
mtail_1 | I0513 21:24:20.182207 1 store.go:153] Running Store.Expire()
mtail_1 | I0513 22:24:20.181714 1 store.go:153] Running Store.Expire()
mtail_1 | I0513 23:24:20.181882 1 store.go:153] Running Store.Expire()
mtail_1 | I0514 00:24:20.181516 1 store.go:153] Running Store.Expire()
mtail_1 | I0514 01:24:20.181308 1 store.go:153] Running Store.Expire()
mtail_1 | I0514 02:24:20.182199 1 store.go:153] Running Store.Expire()
mtail_1 | I0514 03:24:20.181402 1 store.go:153] Running Store.Expire()
mtail_1 | I0514 04:24:20.181444 1 store.go:153] Running Store.Expire()
mtail_1 | I0514 05:24:20.181348 1 store.go:153] Running Store.Expire()
mtail_1 | I0514 06:24:20.181294 1 store.go:153] Running Store.Expire()
mtail_1 | I0514 07:24:20.181426 1 store.go:153] Running Store.Expire()
mtail_1 | I0514 08:24:20.181379 1 store.go:153] Running Store.Expire()
mtail_1 | I0514 09:24:20.181310 1 store.go:153] Running Store.Expire()
mtail_1 | I0514 10:24:20.181311 1 store.go:153] Running Store.Expire()
mtail_1 | I0514 11:24:20.181393 1 store.go:153] Running Store.Expire()
mtail_1 | I0514 12:24:20.181375 1 store.go:153] Running Store.Expire()
mtail_1 | I0514 13:24:20.181364 1 store.go:153] Running Store.Expire()
mtail_1 | I0514 14:24:20.181313 1 store.go:153] Running Store.Expire()
mtail_1 | I0514 15:24:20.181319 1 store.go:153] Running Store.Expire()
mtail_1 | I0514 16:24:20.181363 1 store.go:153] Running Store.Expire()
mtail_1 | I0514 17:24:20.181344 1 store.go:153] Running Store.Expire()
mtail_1 | I0514 18:24:20.181294 1 store.go:153] Running Store.Expire()
mtail_1 | I0514 19:24:20.181298 1 store.go:153] Running Store.Expire()
mtail_1 | I0514 20:24:20.181306 1 store.go:153] Running Store.Expire()
mtail_1 | I0514 21:24:20.181370 1 store.go:153] Running Store.Expire()
mtail_1 | I0514 22:24:20.181319 1 store.go:153] Running Store.Expire()
mtail_1 | I0514 23:24:20.181343 1 store.go:153] Running Store.Expire()
mtail_1 | I0515 00:24:20.181411 1 store.go:153] Running Store.Expire()
mtail_1 | I0515 01:24:20.181412 1 store.go:153] Running Store.Expire()
mtail_1 | I0515 02:24:20.181301 1 store.go:153] Running Store.Expire()
mtail_1 | I0515 03:24:20.181344 1 store.go:153] Running Store.Expire()
mtail_1 | I0515 04:24:20.181373 1 store.go:153] Running Store.Expire()
mtail_1 | I0515 05:24:20.181398 1 store.go:153] Running Store.Expire()
mtail_1 | I0515 06:24:20.181362 1 store.go:153] Running Store.Expire()
mtail_1 | I0515 07:24:20.181440 1 store.go:153] Running Store.Expire()
mtail_1 | I0515 08:24:20.181338 1 store.go:153] Running Store.Expire()
mtail_1 | I0515 09:24:20.181319 1 store.go:153] Running Store.Expire()
mtail_1 | I0515 10:24:20.181297 1 store.go:153] Running Store.Expire()
mtail_1 | I0515 11:24:20.181315 1 store.go:153] Running Store.Expire()
mtail_1 | I0515 12:24:20.181345 1 store.go:153] Running Store.Expire()
mtail_1 | I0515 13:24:20.181310 1 store.go:153] Running Store.Expire()
mtail_1 | I0515 14:24:20.181404 1 store.go:153] Running Store.Expire()
mtail_1 | I0515 15:24:20.181354 1 store.go:153] Running Store.Expire()
mtail_1 | I0515 16:24:20.181455 1 store.go:153] Running Store.Expire()
mtail_1 | I0515 17:24:20.181302 1 store.go:153] Running Store.Expire()
mtail_1 | I0515 18:24:20.181426 1 store.go:153] Running Store.Expire()
mtail_1 | I0515 19:24:20.181300 1 store.go:153] Running Store.Expire()
mtail_1 | I0515 20:24:20.181313 1 store.go:153] Running Store.Expire()
mtail_1 | I0515 21:24:20.181372 1 store.go:153] Running Store.Expire()
mtail_1 | I0515 22:24:20.181355 1 store.go:153] Running Store.Expire()
mtail_1 | I0515 23:24:20.181331 1 store.go:153] Running Store.Expire()
mtail_1 | I0516 00:24:20.181318 1 store.go:153] Running Store.Expire()
mtail_1 | I0516 01:24:20.181314 1 store.go:153] Running Store.Expire()
mtail_1 | I0516 02:24:20.181295 1 store.go:153] Running Store.Expire()
mtail_1 | I0516 03:24:20.181355 1 store.go:153] Running Store.Expire()
mtail_1 | I0516 04:24:20.181346 1 store.go:153] Running Store.Expire()
mtail_1 | I0516 05:24:20.181409 1 store.go:153] Running Store.Expire()
mtail_1 | I0516 06:24:20.181373 1 store.go:153] Running Store.Expire()
mtail_1 | I0516 07:24:20.181298 1 store.go:153] Running Store.Expire()
mtail_1 | I0516 08:24:20.181296 1 store.go:153] Running Store.Expire()
mtail_1 | I0516 09:24:20.181394 1 store.go:153] Running Store.Expire()
mtail_1 | I0516 10:24:20.181319 1 store.go:153] Running Store.Expire()
mtail_1 | I0516 11:24:20.181301 1 store.go:153] Running Store.Expire()
mtail_1 | I0516 12:24:20.181380 1 store.go:153] Running Store.Expire()
mtail_1 | I0516 13:24:20.181375 1 store.go:153] Running Store.Expire()
mtail_1 | I0516 14:24:20.181321 1 store.go:153] Running Store.Expire()
mtail_1 | I0516 15:24:20.181318 1 store.go:153] Running Store.Expire()
mtail_1 | I0516 16:24:20.181386 1 store.go:153] Running Store.Expire()
mtail_1 | I0516 17:24:20.181374 1 store.go:153] Running Store.Expire()
mtail_1 | I0516 18:24:20.181346 1 store.go:153] Running Store.Expire()
mtail_1 | I0516 19:24:20.181327 1 store.go:153] Running Store.Expire()
mtail_1 | I0516 20:24:20.181360 1 store.go:153] Running Store.Expire()
mtail_1 | I0516 21:24:20.181361 1 store.go:153] Running Store.Expire()
mtail_1 | I0516 22:24:20.181296 1 store.go:153] Running Store.Expire()
mtail_1 | I0516 22:42:11.034858 1 main.go:156] Received terminated, exiting...
mtail_1 | I0516 22:42:11.034964 1 mtail.go:210] Shutdown requested.
mtail_1 | I0516 22:42:11.039786 1 runtime.go:270] END OF LINE
mtail_1 | I0516 22:42:11.039822 1 vm.go:1026] VM "chialog.mtail" finished
mtail_1 | I0516 22:42:17.630898 1 main.go:114] mtail version v3.0.0-rc45-57-g8f0b94fa git revision 8f0b94fad7f6c8359e31a01d7ee72e8312d915a3 go version go1.16.4 go arch amd64 go os linux
mtail_1 | I0516 22:42:17.630939 1 main.go:115] Commandline: ["/usr/bin/mtail" "-progs" "/etc/mtail" "-logs" "/var/log/chia/debug.log" "-logtostderr"]
mtail_1 | I0516 22:42:17.631208 1 store.go:178] Starting metric store expiry loop every 1h0m0s
mtail_1 | I0516 22:42:17.631751 1 runtime.go:175] Loaded program chialog.mtail
mtail_1 | I0516 22:42:17.634476 1 tail.go:259] Tailing /var/log/chia/debug.log
mtail_1 | I0516 22:42:17.634544 1 mtail.go:197] Listening on [::]:3903
mtail_1 | I0516 22:47:31.991392 1 main.go:156] Received terminated, exiting...
mtail_1 | I0516 22:47:31.991511 1 mtail.go:210] Shutdown requested.
mtail_1 | I0516 22:47:31.992241 1 runtime.go:270] END OF LINE
mtail_1 | I0516 22:47:31.992281 1 vm.go:1026] VM "chialog.mtail" finished
mtail_1 | I0516 22:47:37.119784 1 main.go:114] mtail version v3.0.0-rc45-57-g8f0b94fa git revision 8f0b94fad7f6c8359e31a01d7ee72e8312d915a3 go version go1.16.4 go arch amd64 go os linux
mtail_1 | I0516 22:47:37.119831 1 main.go:115] Commandline: ["/usr/bin/mtail" "-progs" "/etc/mtail" "-logs" "/var/log/chia/debug.log" "-logtostderr"]
mtail_1 | I0516 22:47:37.120143 1 store.go:178] Starting metric store expiry loop every 1h0m0s
mtail_1 | I0516 22:47:37.120636 1 runtime.go:175] Loaded program chialog.mtail
mtail_1 | I0516 22:47:37.124798 1 tail.go:259] Tailing /var/log/chia/debug.log
mtail_1 | I0516 22:47:37.124833 1 mtail.go:197] Listening on [::]:3903
mtail_1 | I0516 22:47:51.279714 1 main.go:156] Received terminated, exiting...
mtail_1 | I0516 22:47:51.279775 1 mtail.go:210] Shutdown requested.
mtail_1 | I0516 22:47:51.280557 1 runtime.go:270] END OF LINE
mtail_1 | I0516 22:47:51.280608 1 vm.go:1026] VM "chialog.mtail" finished
mtail_1 | I0516 22:47:51.831027 1 main.go:114] mtail version v3.0.0-rc45-57-g8f0b94fa git revision 8f0b94fad7f6c8359e31a01d7ee72e8312d915a3 go version go1.16.4 go arch amd64 go os linux
mtail_1 | I0516 22:47:51.831063 1 main.go:115] Commandline: ["/usr/bin/mtail" "-progs" "/etc/mtail" "-logs" "/var/log/chia/debug.log" "-logtostderr"]
mtail_1 | I0516 22:47:51.831382 1 store.go:178] Starting metric store expiry loop every 1h0m0s
mtail_1 | I0516 22:47:51.831911 1 runtime.go:175] Loaded program chialog.mtail
mtail_1 | I0516 22:47:51.834045 1 tail.go:259] Tailing /var/log/chia/debug.log
mtail_1 | I0516 22:47:51.834104 1 mtail.go:197] Listening on [::]:3903
mtail_1 | I0516 23:17:23.629175 1 main.go:156] Received terminated, exiting...
mtail_1 | I0516 23:17:23.629444 1 mtail.go:210] Shutdown requested.
mtail_1 | I0516 23:17:23.630178 1 runtime.go:270] END OF LINE
mtail_1 | I0516 23:17:23.630225 1 vm.go:1026] VM "chialog.mtail" finished
mtail_1 | I0516 23:17:27.850782 1 main.go:114] mtail version v3.0.0-rc45-57-g8f0b94fa git revision 8f0b94fad7f6c8359e31a01d7ee72e8312d915a3 go version go1.16.4 go arch amd64 go os linux
mtail_1 | I0516 23:17:27.850816 1 main.go:115] Commandline: ["/usr/bin/mtail" "-progs" "/etc/mtail" "-logs" "/var/log/chia/debug.log" "-logtostderr"]
mtail_1 | I0516 23:17:27.850999 1 store.go:178] Starting metric store expiry loop every 1h0m0s
mtail_1 | I0516 23:17:27.851581 1 runtime.go:175] Loaded program chialog.mtail
mtail_1 | I0516 23:17:27.853888 1 tail.go:259] Tailing /var/log/chia/debug.log
mtail_1 | I0516 23:17:27.853951 1 mtail.go:197] Listening on [::]:3903
mtail_1 | I0516 23:21:10.717713 1 main.go:156] Received terminated, exiting...
mtail_1 | I0516 23:21:10.717839 1 mtail.go:210] Shutdown requested.
mtail_1 | I0516 23:21:10.718869 1 runtime.go:270] END OF LINE
mtail_1 | I0516 23:21:10.718903 1 vm.go:1026] VM "chialog.mtail" finished
mtail_1 | I0516 23:21:11.237652 1 main.go:114] mtail version v3.0.0-rc45-57-g8f0b94fa git revision 8f0b94fad7f6c8359e31a01d7ee72e8312d915a3 go version go1.16.4 go arch amd64 go os linux
mtail_1 | I0516 23:21:11.237683 1 main.go:115] Commandline: ["/usr/bin/mtail" "-progs" "/etc/mtail" "-logs" "/var/log/chia/debug.log" "-logtostderr"]
mtail_1 | I0516 23:21:11.237910 1 store.go:178] Starting metric store expiry loop every 1h0m0s
mtail_1 | I0516 23:21:11.238387 1 runtime.go:175] Loaded program chialog.mtail
mtail_1 | I0516 23:21:11.241163 1 tail.go:259] Tailing /var/log/chia/debug.log
mtail_1 | I0516 23:21:11.242676 1 mtail.go:197] Listening on [::]:3903
mtail_1 | I0516 23:21:14.532894 1 main.go:156] Received terminated, exiting...
mtail_1 | I0516 23:21:14.533034 1 mtail.go:210] Shutdown requested.
mtail_1 | I0516 23:21:14.533800 1 runtime.go:270] END OF LINE
mtail_1 | I0516 23:21:14.533834 1 vm.go:1026] VM "chialog.mtail" finished
mtail_1 | I0516 23:21:15.013855 1 main.go:114] mtail version v3.0.0-rc45-57-g8f0b94fa git revision 8f0b94fad7f6c8359e31a01d7ee72e8312d915a3 go version go1.16.4 go arch amd64 go os linux
mtail_1 | I0516 23:21:15.013885 1 main.go:115] Commandline: ["/usr/bin/mtail" "-progs" "/etc/mtail" "-logs" "/var/log/chia/debug.log" "-logtostderr"]
mtail_1 | I0516 23:21:15.014121 1 store.go:178] Starting metric store expiry loop every 1h0m0s
mtail_1 | I0516 23:21:15.014664 1 runtime.go:175] Loaded program chialog.mtail
mtail_1 | I0516 23:21:15.016797 1 tail.go:259] Tailing /var/log/chia/debug.log
mtail_1 | I0516 23:21:15.018167 1 mtail.go:197] Listening on [::]:3903
mtail_1 | I0517 00:21:15.014236 1 store.go:153] Running Store.Expire()
mtail_1 | I0517 01:21:15.014176 1 store.go:153] Running Store.Expire()
mtail_1 | I0517 02:21:15.014308 1 store.go:153] Running Store.Expire()
mtail_1 | I0517 03:21:15.014178 1 store.go:153] Running Store.Expire()
mtail_1 | I0517 04:21:15.014293 1 store.go:153] Running Store.Expire()
mtail_1 | I0517 05:21:15.014174 1 store.go:153] Running Store.Expire()
mtail_1 | I0517 06:21:15.014173 1 store.go:153] Running Store.Expire()
mtail_1 | I0517 07:21:15.014192 1 store.go:153] Running Store.Expire()
PS C:\Users\Digit\OneDrive\Documents\GitHub\chiamon>

When i look at docker all appear to be running, and windows explorer is running.

[retzkek]

@digitalkisan sorry, I have no clue. Probably an issue with the mtail program still. Did you re-build the mtail image after changing the program (docker-compose build --no-cache mtail)?

[digitalkisan]

@digitalkisan sorry, I have no clue. Probably an issue with the mtail program still. Did you re-build the mtail image after changing the program (docker-compose build --no-cache mtail)?

Thank you, I did but I need to restart the machine for an update anyway so I will redo it with that command.

[yanadhorn]

where can i learn about customize dashboard like the linux version?

[retzkek]

I'm going to close this issue, it's become a mess and the original question was solved long ago. I pushed the changes recommended by @SiliPSX. We can use #11 to continue to troubleshoot mtail.

[digitalkisan]

@retzkek
windows: fix mtail program and network queries …
9daf0c0
HT: @SiliPSX (#2)

This worked for me I had left an expression unchanged that was breaking the dash. Thanks guys