[FEATURE REQUEST] JSON lines output

[ocervell]

Consider adding a JSON lines output so that we can consume real-time output with other tools like jq or others. Maybe a --jsonl flag ?

[epi052]

hey there! give the following command a try. depending on the tool you're piping into, --silent may be useful. However, even with --silent you'd still get the found urls coming through

feroxbuster -u http://example.com --json --output /dev/stdout | jq -R "fromjson? | . "

[ocervell]

Looks pretty good ! Ideally the output would be only the JSON lines going to stdout and the rest to stderr (which is what for instance ProjectDiscovery tools like katana or httpx do) but I can deal with this. Maybe an extra option to hide the flat URLs would be nice ;)

[epi052]

Yea, the thought back then was that the best thing ok ng to pipe into further tools would be the url itself.

I can test out a version where --json + --silent would give json instead of the url

[ocervell]

That would be top notch ! Thanks for this amazing tool btw, very well coded !

edit: for some more context, I'm calling feroxbuster from a Python subprocess and adding hooks to send the JSON output to a MongoDB database, thus the need of "live output".

[epi052]

cargo run -- -u http://127.0.0.1:8000  --json --silent -w smol | jq

{
  "type": "response",
  "url": "http://127.0.0.1:8000/3",
  "original_url": "http://127.0.0.1:8000",
  "path": "/3",
  "wildcard": false,
  "status": 500,
  "method": "GET",
  "content_length": 0,
  "line_count": 0,
  "word_count": 0,
  "headers": {
    "content-length": "0",
    "date": "Wed, 08 Nov 2023 12:00:49 GMT"
  },
  "extension": ""
}
{
  "type": "response",
  "url": "http://127.0.0.1:8000/",
  "original_url": "http://127.0.0.1:8000",
  "path": "/",
  "wildcard": false,
  "status": 307,
  "method": "GET",
  "content_length": 120,
  "line_count": 1,
  "word_count": 5,
  "headers": {
    "content-type": "text/plain; charset=utf-8",
    "x-content-type-options": "nosniff",
    "permissions-policy": "interest-cohort=()",
    "content-length": "120",
    "date": "Wed, 08 Nov 2023 12:00:49 GMT",
    "x-frame-options": "SAMEORIGIN",
    "server": "Rocket"
  },
  "extension": ""
}

[epi052]

@all-contributors add @ocervell for ideas

[allcontributors]

@epi052

I've put up a pull request to add @ocervell! 🎉