Update transport file logging

[ersonp]

Did you run make format && make check?

Fixes #1363

Changes:

• Update dmsgfileserver to use API
• Changed transport file logging to save to csv
• Added folder ./local/transport_logs
• Dmsghttp server explicitly serves /privacy.json and system.json
• Dmsghttp now servers everything inside ./local/transport_logs and ./local/custom

How to test this PR:

1. Start a vpn-server
2. Connect to it from your visor
3. Create privacy.json file
4. From dmsg build and run ./bin/dmsgget dmsg://<pk>:80/system.json and check output
5. From dmsg build and run ./bin/dmsgget dmsg://<pk>:80/privacy.json and check output
6. From dmsg build and run ./bin/dmsgget dmsg://<pk>:80/2022-10-10.csv(todays date) and check output
7. Add hello.txt to ./local/custom
8. From dmsg build and run ./bin/dmsgget dmsg://<pk>:80/hello.txt and check output

[0pcom]

https://github.com/ersonp/skywire/blob/feat/update-transport-logging/pkg/visor/init.go#L371

This should be info or debug, not error, right?

[ersonp]

https://github.com/ersonp/skywire/blob/feat/update-transport-logging/pkg/visor/init.go#L371

This should be info or debug, not error, right?

Yes, it's just for testing. This is a rough draft just to test the functionality.

[ersonp]

<pk>:80/system.json will get you the system json form ./local
you will have to manually add files to ./local/transport_logs
and you can access those files by <pk>:80/filename

Just like you wanted the logs will be in its own directory in ./local/transport_logs and the config files will be directly under ./local without exposing other files like skysocks_log.db

and also for privacy.json <pk>:80/privacy.json which should also be in ./local

[0pcom]

I don't understand how exactly you were able to accomplish this, but it does work as desired.

however, it would be nice to be able to include arbitrary files here which can be accessed over dmsg.

If running a visor with a dmsghttp server will effectively interfere or conflict with a separate dmsghttp server instance, it would be prudent to leave it open to creative actions by the user.

The way it is in your PR currently is ok. If you can think of a way to leave it more open to customization that does not interfere with the core functionality, that would be good, but this will eventually require its own a consideration.

The logserver itself isn't exactly core functionality, skywire works without it. Who is to say that people are more interested in rewards than having, for instance, a website over skywire (or, in this specific case, over dmsg)? Eventually you'll have more nodes than you can be rewarded for.

[ersonp]

prudent

For that, we can have a custom folder where the user can have their own files, just like transport_logs where any file inside it can be accessed from the dmsghttp server.

I think there might be a way for us to exclude specific files in ./local like apps-pid.txt and skysocks_log.db and the rest will be available on dmsg which is the opposite way we are currently doing where system.json and privacy.json are explicitly included.

But I think the first way would be better to give a specific folder for the users to tinker around instead of the base directory which includes essential files.

[0pcom]

I've tested this and it works.

For some reason I have the session cookie error when attempting to access the hypervisor UI. Not sure what that is about, and I don't think you modified any files that would do that.

using the cli, I was having issues getting the VPN to work to my local nodes. Even when I run the server on develop. There are no errors evident in the logging, the vpn says it is running, but I'm not able to load any pages in a browser. Not much more luck from remote nodes.

will retest develop branch after merge

Just fix the appveyor errors and then its good to go

[0pcom]

its not apparent why the last commit did not work for appveyor.

merging because there is no sensible error from appveyor to address