Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a statcounter that prints statistics about the connection #117

Merged
merged 6 commits into from
Aug 1, 2019

Conversation

Res260
Copy link
Collaborator

@Res260 Res260 commented Jun 12, 2019

Logs various data about the connection when the client disconnects. Looks like this:
image
each field is separated so it can be used as filters and to build visualizations in ES/Kibana.

What I implemented for now:

    CONNECTION_TIME = "connectionTime"
    # Duration (in secs) for the TCP connection

    CLIENT_SERVER_RATIO = "clientServerRatio"
    # Ratio of the # of messages coming from the client vs from the server. High value (>1) means high client interaction.

    TOTAL_INPUT = "totalInput"
    # # of messages coming from the client to the server.

    TOTAL_OUTPUT = "totalOutput"
    # # of messages coming from the server to the client.

    IO_INPUT = "input"
    # Packet coming from the client to the server for the io channel

    IO_INPUT_FASTPATH = "fastPathInput"
    # Packet coming from the client to the server for the io channel as a fastpath packet

    IO_INPUT_SLOWPATH = "slowPathInput"
    # Packet coming from the client to the server for the io channel as a slowpath packet

    IO_OUTPUT = "output"
    # Packet coming from the server to the client for the io channel

    IO_OUTPUT_FASTPATH = "fastPathOutput"
    # Packet coming from the server to the client for the io channel as a fastpath packet

    IO_OUTPUT_SLOWPATH = "slowPathOutput"
    # Packet coming from the server to the client for the io channel as a slowpath packet

    MCS = "mcs"
    # Packet Coming from either end for any channel

    MCS_OUTPUT = "mcsOutput"
    # Packet Coming from the server to the client for any channel

    MCS_OUTPUT_ = "mcsOutput_"
    # Packet Coming from the server to the client for a given channel (must append channel # after it)

    MCS_INPUT = "mcsInput"
    # Packet Coming from the client to the server for any channel

    MCS_INPUT_ = "mcsInput_"
    # Packet Coming from the client to the server for a given channel (must append channel # after it)

    VIRTUAL_CHANNEL = "virtualChannel"
    # Packet Coming from either end for any virtual channel that doesnt have a specific implementation (ex clipboard)

    VIRTUAL_CHANNEL_INPUT = "virtualChannelInput"
    # Packet Coming from the client to the server for any virtual channel that doesnt have a specific implementation (ex clipboard)

    VIRTUAL_CHANNEL_OUTPUT = "virtualChannelOutput"
    # Packet Coming from the server to the client for any virtual channel that doesnt have a specific implementation (ex clipboard)

    DEVICE_REDIRECTION = "deviceRedirection"
    # Packet coming from either end for the rdpdr channel

    DEVICE_REDIRECTION_CLIENT = "deviceRedirectionClient"
    # Packet coming from the client to the server for the rdpdr channel

    DEVICE_REDIRECTION_SERVER = "deviceRedirectionServer"
    # Packet coming from the server to the client for the rdpdr channel

    DEVICE_REDIRECTION_IOREQUEST = "deviceRedirectionIORequest"
    # IORequest packets for the rdpdr channel

    DEVICE_REDIRECTION_IORESPONSE = "deviceRedirectionIOResponse"
    # IOResponse packets for the rdpdr channel

    DEVICE_REDIRECTION_IOERROR = "deviceRedirectionIOError"
    # IO error packets for the rdpdr channel

    DEVICE_REDIRECTION_FILE_CLOSE = "deviceRedirectionFileClose"
    # File Close packets for the rdpdr channel

    DEVICE_REDIRECTION_FORGED_FILE_READ = "deviceRedirectionForgedFileRead"
    # File read packets forged by pyrdp for the rdpdr channel

    DEVICE_REDIRECTION_FORGED_DIRECTORY_LISTING = "deviceRedirectionForgedDirectoryListing"
    # Directory listing packets forged by pyrdp for the rdpdr channel

    CLIPBOARD = "clipboard"
    # Number of clipboard PDUs coming from either end

    CLIPBOARD_CLIENT = "clipboardClient"
    # Number of clipboard PDUs coming from the client

    CLIPBOARD_SERVER = "clipboardServer"
    # Number of clipboard PDUs coming from the server

    CLIPBOARD_COPY = "clipboardCopies"
    # Number of times data has been copied by either end

    CLIPBOARD_PASTE = "clipboardPastes"
    # Number of times data has been pasted by either end

…disconnection

- Rename mitm.py to RDPMITM.py to follow convention
- Add an info log statement for virtual channel name <---> channel ID association
@Res260 Res260 requested review from Pourliver and xshill June 12, 2019 03:00
@Pourliver
Copy link
Contributor

I like the idea, but I don't think it gives any useful information for a user monitoring the connection. Since it can be really good for data collection, should this be "silently" logged at a DEBUG level?

@Res260
Copy link
Collaborator Author

Res260 commented Jun 13, 2019

We've had this discussion before, but debug vs info should not be honeypot vs pentest. Right now we dont have a way to differenciate both afaik. Right now, only INFO logs are written to the json file to be injested by elasticsearch.

@Pourliver
Copy link
Contributor

My point was for both honeypot and pentest, but I didn't know that only INFO log were inside the JSON. I assumed that DEBUG were logged as well.

@Res260
Copy link
Collaborator Author

Res260 commented Jun 13, 2019

The most relevant information it gives for someone monitoring the logs would be the connection time and the input / output ratio, which helps identify connections worth looking at imo

@Res260
Copy link
Collaborator Author

Res260 commented Jun 13, 2019

And of course clipboard and/or rdpdr activity

@Pourliver
Copy link
Contributor

Pourliver commented Jun 19, 2019

After a few tests, I got this error message. Maybe validate the TOTAL_OUTPUT before dividing.

Unhandled Error
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/twisted/internet/asyncioreactor.py", line 267, in run
    self._asyncioEventloop.run_forever()
  File "/usr/lib/python3.6/asyncio/base_events.py", line 427, in run_forever
    self._run_once()
  File "/usr/lib/python3.6/asyncio/base_events.py", line 1440, in _run_once
    handle._run()
  File "/usr/lib/python3.6/asyncio/events.py", line 145, in _run
    self._callback(*self._args)
--- <exception caught here> ---
  File "/usr/local/lib/python3.6/dist-packages/twisted/python/log.py", line 103, in callWithLogger
    return callWithContext({"system": lp}, func, *args, **kw)
  File "/usr/local/lib/python3.6/dist-packages/twisted/python/log.py", line 86, in callWithContext
    return context.call({ILogContext: newCtx}, func, *args, **kw)
  File "/usr/local/lib/python3.6/dist-packages/twisted/python/context.py", line 122, in callWithContext
    return self.currentContext().callWithContext(ctx, func, *args, **kw)
  File "/usr/local/lib/python3.6/dist-packages/twisted/python/context.py", line 85, in callWithContext
    return func(*args,**kw)
  File "/usr/local/lib/python3.6/dist-packages/twisted/internet/asyncioreactor.py", line 141, in _readOrWrite
    self._disconnectSelectable(selectable, why, read)
  File "/usr/local/lib/python3.6/dist-packages/twisted/internet/posixbase.py", line 255, in _disconnectSelectable
    selectable.connectionLost(f)
  File "/usr/local/lib/python3.6/dist-packages/twisted/internet/tcp.py", line 327, in connectionLost
    protocol.connectionLost(reason)
  File "/usr/local/lib/python3.6/dist-packages/twisted/protocols/tls.py", line 403, in connectionLost
    ProtocolWrapper.connectionLost(self, reason)
  File "/usr/local/lib/python3.6/dist-packages/twisted/protocols/policies.py", line 125, in connectionLost
    self.wrappedProtocol.connectionLost(reason)
  File "/usr/local/lib/python3.6/dist-packages/pyrdp-0.2.0-py3.6-linux-x86_64.egg/pyrdp/layer/tcp.py", line 66, in connectionLost
    self.observer.onDisconnection(reason)
  File "/usr/local/lib/python3.6/dist-packages/pyrdp-0.2.0-py3.6-linux-x86_64.egg/pyrdp/core/observer.py", line 82, in __call__
    self.composite.doCall(self.item, args, kwargs)
  File "/usr/local/lib/python3.6/dist-packages/pyrdp-0.2.0-py3.6-linux-x86_64.egg/pyrdp/core/observer.py", line 56, in doCall
    getattr(observer, item)(*args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/pyrdp-0.2.0-py3.6-linux-x86_64.egg/pyrdp/mitm/TCPMITM.py", line 80, in onClientDisconnection
    self.statCounter.stop()
  File "/usr/local/lib/python3.6/dist-packages/pyrdp-0.2.0-py3.6-linux-x86_64.egg/pyrdp/logging/StatCounter.py", line 154, in stop
    self.stats[STAT.CLIENT_SERVER_RATIO] = self.stats[STAT.TOTAL_INPUT] / self.stats[STAT.TOTAL_OUTPUT]
builtins.ZeroDivisionError: division by zero

@Res260
Copy link
Collaborator Author

Res260 commented Jun 19, 2019

Right, didnt think about that but with a simple tcp scan that would trigger the error. Will update.

@Res260 Res260 requested a review from Pourliver June 20, 2019 22:34
@Res260
Copy link
Collaborator Author

Res260 commented Jun 25, 2019

The fix was simpler than I thought. Apparently something changed when @xshill refactored the logging classes and now we can put arbitrary key/values objects even if they don't appear in the "message" part of the log, which is nice.

@Pourliver
Copy link
Contributor

So, after testing, it "works" because the log goes in the mitm.json, but here is what the console output looks like:

[2019-06-25 12:45:22,225] - INFO - Philip232610 - pyrdp.mitm.connections.tcp - Connection report

This is counter-intuitive for the user, we should either print the connection report to the user, or log it silently. This mixes both.

@Res260
Copy link
Collaborator Author

Res260 commented Jun 25, 2019 via email

@Pourliver
Copy link
Contributor

I tested and merged master into the branch since there was a lot of conflicts.

@Res260
Copy link
Collaborator Author

Res260 commented Jul 21, 2019

Alright it should be good now @Pourliver

@Pourliver
Copy link
Contributor

Looks good to me! Merging.

@Pourliver Pourliver merged commit 06c171c into master Aug 1, 2019
@obilodeau obilodeau deleted the statcounter branch November 27, 2019 21:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants