Request file support

README.md

@@ -20,11 +20,13 @@ Bypass Url Parser, made with love by @TheLaluka
 A tool that tests MANY url bypasses to reach a 40X protected page.
 
 Usage:
-    ./bypass_url_parser.py -u <URL> [-m <mode>] [-o <outdir>] [-S <level>] [(-H <header>)...] [-r <num>]
-                           [-s <ip>] [--spoofip-replace] [-p <port>] [--spoofport-replace] [--dump-payloads]
-                           [-t <threads>] [-T <timeout>] [-x <proxy_url>] [-v | -d | -dd]
+    ./bypass_url_parser.py (-u <URL> | -R <file>) [-m <mode>] [-o <outdir>] [-S <level>] [ (-H <header>)...] [-r <num>]
+                           [-s <ip>] [--spoofip-replace] [-p <port>] [--spoofport-replace] [--request-tls]
+                           [--dump-payloads] [-t <threads>] [-T <timeout>] [-x <proxy_url>] [-v | -d | -dd]
 
 Program options:
+    -u, --url <URL>           URL (path is optional) to run bypasses against
+    -R, --request <file>      Load HTTP raw request from a file
     -H, --header <header>     Header(s) to use, format: "Cookie: can_i_haz=fire"
     -m, --mode <mode>         Bypass modes. See 'Bypasser.BYPASS_MODES' in code [Default: all]
     -o, --outdir <outdir>     Output directory for results
@@ -35,7 +37,6 @@ Program options:
     -r, --retry <num>         Retry attempts of failed requests. Set 0 to disable all retry tentatives [Default: 3]
     -t, --threads <threads>   Scan with N parallel threads [Default: 1]
     -T, --timeout <timeout>   Request times out after N seconds [Default: 5]
-    -u, --url <URL>           URL (path is optional) to run bypasses against
 
 General options:
     -h, --help                Show help, you are here :)
@@ -47,21 +48,23 @@ General options:
 Misc options:
     --spoofip-replace         Disable list of default internal IPs in 'http_headers_ip' bypass mode
     --spoofport-replace       Disable list of default internal ports in 'http_headers_port' bypass mode
+    --request-tls             Force usage of TLS/HTTPS for the request load with the '-R, --request' option
     --dump-payloads           Print all payloads (curls) generated by this tool.
 
 Examples:
     ./bypass_url_parser.py -u "http://127.0.0.1/juicy_403_endpoint/" -s 8.8.8.8 -d
     ./bypass_url_parser.py -u /path/urls -t 30 -T 5 -H "Cookie: me_iz=admin" -H "User-agent: test"
+    ./bypass_url_parser.py -R /path/request_file --request-tls -m "mid_paths, end_paths"
 ```
 
 ## Expected result
 
 ```bash
 ./bypass_url_parser.py -u http://127.0.0.1:8000/foo/bar
-2022-08-09 14:52:40 lalu-perso bup[361559] WARNING Trying to bypass 'http://127.0.0.1:8000/foo/bar' url (1625 payloads)...
-2022-08-09 14:52:40 lalu-perso bup[361559] INFO Doing: 50 / 1625
+2022-08-09 14:52:40 lalu-perso bup[361559] WARNING Trying to bypass 'http://127.0.0.1:8000/foo/bar' url (3213 payloads)...
+2022-08-09 14:52:40 lalu-perso bup[361559] INFO Doing: 50 / 3213
 [...]
-2022-08-09 14:52:54 lalu-perso bup[361559] INFO Doing: 1600 / 1625
+2022-08-09 14:52:54 lalu-perso bup[361559] INFO Doing: 3200 / 3213
 2022-08-09 14:52:54 lalu-perso bup[361559] INFO Retry (1/3) the '16' failed curl commands with 10 threads and 10s timeout
 2022-08-09 14:52:54 lalu-perso bup[361559] INFO Retry (2/3) the '16' failed curl commands with 5 threads and 20s timeout
 2022-08-09 14:52:54 lalu-perso bup[361559] INFO Retry (3/3) the '16' failed curl commands with 1 threads and 30s timeout
@@ -125,6 +128,12 @@ cat /path/urls | ./bypass_url_parser.py -u -
 echo 'http://thinkloveshare.com/test' | ./bypass_url_parser.py -u -
 ```
 
+### Target definition
+
+A target must be defined for the tool to work. 2 options:
+- `-u, --url`: URL(s), in GET
+- `-R, --request`: Request file. The protocol can't be guessed from file, so `http` by default or `https` if `--request-tls` option is present.
+
 ### Bypass mode
 
 If `-m, --mode` is specified, you can select the desired bypass mode to run a specific test (or tests) and reduce the number of requests sent by the tool.