Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

-r Recursive option #129

Merged
merged 29 commits into from
Jun 23, 2018
Merged
Show file tree
Hide file tree
Changes from 11 commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
2ebe595
new args
omergunal May 10, 2018
ef3a21d
added recursive args
omergunal May 11, 2018
38be6e2
Update __main__.py
omergunal May 11, 2018
759f632
Update __main__.py
omergunal May 11, 2018
ed38dbb
Created discover_files() function
omergunal May 12, 2018
fcf4638
Merge branch 'master' into patch-4
omergunal Jun 6, 2018
3ac883c
added recursive option
omergunal Jun 6, 2018
e246104
discover_files
omergunal Jun 6, 2018
2cbac72
added recursive, targets
omergunal Jun 7, 2018
7875c82
update discover_files()
omergunal Jun 7, 2018
ca0b2d7
removed file_list
omergunal Jun 7, 2018
d9db9dd
"targets" must be required
omergunal Jun 10, 2018
c35ae81
created loop for discover_files()
omergunal Jun 10, 2018
9c54d8c
new params
omergunal Jun 10, 2018
40c0f8f
Update __main__.py
omergunal Jun 16, 2018
42759f0
Update __main__.py
omergunal Jun 16, 2018
5931faf
Merge branch 'master' into patch-4
omergunal Jun 16, 2018
5546c3d
changed func. and added baseline
omergunal Jun 19, 2018
8d1d805
new parameters for discover_files
omergunal Jun 19, 2018
35b8001
test_valid_args_but_no_targets()
omergunal Jun 20, 2018
2e4d07a
edited expected values
omergunal Jun 20, 2018
0c6b082
changed vulnerabilities list location
omergunal Jun 20, 2018
ae84a44
Update usage_test.py
omergunal Jun 20, 2018
1944b4a
Update usage_test.py
omergunal Jun 20, 2018
ba3d438
changed location of "recursive control"
omergunal Jun 21, 2018
6a25e25
Update usage.py
omergunal Jun 22, 2018
f42d283
de-dent some lines
omergunal Jun 22, 2018
c7b2f73
test_no_args
omergunal Jun 22, 2018
2afc177
test_no_args passed
omergunal Jun 22, 2018
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 26 additions & 0 deletions pyt/__main__.py
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,24 @@
)


def discover_files(targets, excluded_files, recursive=False):
included_files = list()
excluded_list = excluded_files.split(",")

for target in targets:
if os.path.isdir(target):
if recursive:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So having if recursive: here it will make it so that if you don't have -r then you won't search directories.

You can change it to:

def discover_files(targets, excluded_files, recursive=False):
    included_files = list()
    excluded_list = excluded_files.split(",")

    for target in targets:
        if os.path.isdir(target):
            for root, dirs, files in os.walk(target):
                for f in files:
                    fullpath = os.path.join(root, f)
                    if os.path.splitext(fullpath)[1] == '.py' and fullpath.split("/")[-1] not in excluded_list:
                        included_files.append(fullpath)
                if not recursive:
                    break
        else:
            if target not in excluded_list:
                included_files.append(target)
    return included_files

for root, dirs, files in os.walk(target):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this line, for root, dirs, files in os.walk(target): is indented one more level than it has to be.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i will try to do better for returning "included_files"

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: You can de-dent from line 38 to line 44

for f in files:
fullpath = os.path.join(root, f)
if os.path.splitext(fullpath)[1] == '.py' and fullpath.split("/")[-1] not in excluded_list:
included_files.append(fullpath)
else:
if targets not in excluded_list:
included_files.append(targets[0])
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So if targets is a list of files, e.g. python -m pyt examples/vulnerable_code/command_injection.py examples/vulnerable_code/XSS.py, then discover_files will return the first file N times. (Where N is the len of targets.)

return(included_files)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you change this to return included_files, just to be consistent w/ the other code 👍



def main(command_line_args=sys.argv[1:]):
args = parse_args(command_line_args)

Expand All @@ -39,6 +57,14 @@ def main(command_line_args=sys.argv[1:]):
elif args.trim_reassigned_in:
ui_mode = UImode.TRIM



targets = args.targets
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it might be more DRY if you did

files = discover_files(
  args.targets,
  args.excluded_paths,
  args.recursive
)

excluded_files = args.excluded_paths
recursive = args.recursive
test = discover_files(targets, excluded_files, recursive) #just for see files in directory
print(test)

path = os.path.normpath(args.filepath)

if args.ignore_nosec:
Expand Down
24 changes: 19 additions & 5 deletions pyt/usage.py
Original file line number Diff line number Diff line change
Expand Up @@ -29,11 +29,11 @@ def valid_date(s):

def _add_required_group(parser):
required_group = parser.add_argument_group('required arguments')
required_group.add_argument(
'''required_group.add_argument(
'-f', '--filepath',
help='Path to the file that should be analysed.',
type=str
)
)'''


def _add_optional_group(parser):
Expand Down Expand Up @@ -91,7 +91,21 @@ def _add_optional_group(parser):
action='store_true',
help='do not skip lines with # nosec comments'
)

optional_group.add_argument(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe make it

    parser.add_argument(
        '-r', '--recursive', dest='recursive',
        action='store_true', help='find and process files in subdirectories'
    )

'-r', '--recursive', dest='recursive',
action='store_true', help='find and process files in subdirectories'
)
optional_group.add_argument(
'-x', '--exclude',
dest='excluded_paths',
action='store',
default='',
help='Separate files with commas'
)
optional_group.add_argument(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess targets will be part of _add_required_group b/c it's replacing -f files

'targets', metavar='targets', type=str, nargs='*',
help='source file(s) or directory(s) to be tested'
)

def _add_print_group(parser):
print_group = parser.add_argument_group('print arguments')
Expand All @@ -110,8 +124,8 @@ def _add_print_group(parser):


def _check_required_and_mutually_exclusive_args(parser, args):
if args.filepath is None:
parser.error('The -f/--filepath argument is required')
if args.targets is None:
parser.error('The target argument is required')


def parse_args(args):
Expand Down