[Feature request] Improve output customisation

[prryplatypus]

Hello!

Description
I am trying to access the output of this tool programmatically, and it is being slightly more complicated than it could be due to the all the logging and colouring in the output. I've dropped here a possible solution I've managed to come up with; let me know your thoughts!

Possible solution

1. Add a --colourless flag that makes the tool output everything without colours.
2. Add a --silent flag that makes the tool not output anything to stdout other than the results. This would imply the output being colourless too.
3. Consider --logfile - or -l - to mean "output to stdout". This is common behaviour defined in the POSIX utility syntax guidelines (for utilities that use operands to represent files to be opened for either reading or writing, the '-' operand should be used only to mean standard input (or standard output when it is clear from context that an output file is being specified)). If such a value was specified, then the output should be silent too.
4. Add a --format flag which allows selection of the output format desired (e.g.: --format=jsonlines, --format=csv, etc...). In the former, the tool could return a format such as {"platform": "aws", "status":"protected", "url":"xxxx.xxx.xxx"} for every result. Formatted results would then be output to the specified logfile. Note that I've used those formats as examples on purpose, as they would allow outputting results "on the go".

Thank you!

[initstring]

Hi @prryplatypus ,

These are great ideas!

If I was writing this today from scratch, I think it would have done things differently, like implement a class for findings with some methods that handle data in dictionaries and then allow for various output methods.

Hindsight is 20-20, as they say. Maybe a re-write is in order one of these days.

Anyway, let me think on this and see if I might be able to find some time for a simple solution - possibly either 2 or 3 as you propose above.

Thanks again for such a detailed and thoughtful feature request.

[initstring]

[REMOVED DUMB IDEA, SORRY]

[initstring]

Eh, strike that. Let me come up with a better solution, more in line with your option 4 above.

[prryplatypus]

Haha, no rush. Thanks for looking into it :)

[initstring]

Hi @prryplatypus

OK, I have something working in this branch.

Right now, it will default to JSON. Output on the console will be the same, but the log file is JSON. I will add an argument to choose between JSON or CSV, as the logic to do either is included here.

What do you think? Does this do the trick?

[prryplatypus]

Looks to me like that would work! May I just suggest adding an extra boolean field to inform about whether the resource is protected or open (something like public, maybe)?

[initstring]

How about some severity level? I could correspond that to the current color coding, actually.

[prryplatypus]

We've got a deal as long as the being-public-or-not info can be gathered from it! ;)

[initstring]

OK, let me think on it a bit and I'll continue to iterate on this probably over the next week.

Thanks again for your input.

[prryplatypus]

Anytime! Thanks for the help 😃

[initstring]

Hi @prryplatypus -

I've implemented the following tags:

• public
• protected
• disabled

There are some items where it doesn't necessarily fit cleanly into one item, but I did my best to make reasonable choices. An example would be Azure storage accounts. Public ones can be brute forced to find buckets, but they themselves aren't that valuable. But they wouldn't count as protected, as those can't be brute forced.

Anyway, let me know what you think.

[prryplatypus]

Sounds good!

On a different note: Looks like I got flagged by some antispam system on AWS S3 when using cloud_enum and now every bucket returns "unknown bucket" unless I switch my IP 😳.

[initstring]

Yeah, I've seen that happen while back. Were you using non-default option for threads? It's why I have it set so low by default, just to be safe.

I've actually not been using the tool much myself lately, so probably won't notice if/when things like that change.

[prryplatypus]

Nope, was using the default option! I did run it several times one after the other while I was trying to figure out some stuff though :S

[initstring]

Hmmm... yeah, I imagine these types of things will get even more strict with time.

During this work, I actually discovered that DNS brute-forcing seems to be totally blocked when working on a common VPN provider and not supplying your own DNS server.

Anyway, I think this feature request is complete. I've merged the changes in and tagged a new version 0.7.

Please let me know if you have any issues with it.

Thanks again for the suggestions!

[prryplatypus]

Thanks so much for the quick replies and implementation!