Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Script parsing fails when list/dictionary literals are not valid JSON #97

Closed
jeremyadamsfisher opened this issue Aug 6, 2020 · 1 comment
Assignees
Labels

Comments

@jeremyadamsfisher
Copy link

jeremyadamsfisher commented Aug 6, 2020

Describe the bug
When importing a pipeline with {import,update}_pipe_project.py, some script constants that are valid python literals cannot be parsed. Although the overall script parsing succeeds, this leads to errors downstream:

  • When ENVS are invalid JSON, the script hangs on execution
  • When the ARGUMENTS are invalid JSON, the script can be run, but the arguments cannot be changed from the frontend

To Reproduce: script hangs on execution

  1. In any script, make the constant a valid python literal, but an invalid JSON. For example:
ENVS = ["custom-worker"]  # comment about custom worker
  1. This results in a json.decoder.JSONDecodeError
  2. Script hangs indefinitely
    image

To Reproduce: Arguments cannot be changed from the frontend

  1. In any script, make ARGUMENTS a valid python literal, but an invalid JSON. For example:
ARGUMENTS = {
    # comments are not allowed in JSON
    "file_name": {
        "value": "annos.csv",
        "help": "Name of the file with exported bbox annotations."
    }, # <- also, notice the trailing comma
}
  1. This results in a json.decoder.JSONDecodeError
  2. Front-end no longer displays "file_name" as an argument

image
image

Expected behavior

  • Front-end should display arguments
  • ARGUMENTS and ENVS hould be parsed like python literals, raise a more descriptive error or fail outright.

Additional context
It seems to me that this line could simply be changed to arg_dict = ast.literal_eval(args) to fix both issues. I've tested this minimally and it seems to work.

I would be happy to write some unit tests and make a PR, if there is interest.

@jeremyadamsfisher jeremyadamsfisher changed the title Script parsing fails when constants Script parsing fails when constants are not strictly valid JSON Aug 6, 2020
@jeremyadamsfisher jeremyadamsfisher changed the title Script parsing fails when constants are not strictly valid JSON Script parsing fails when dictionary literals are not valid JSON Aug 7, 2020
@jeremyadamsfisher jeremyadamsfisher changed the title Script parsing fails when dictionary literals are not valid JSON Script parsing fails when list/dictionary literals are not valid JSON Aug 7, 2020
@jaeger-j jaeger-j self-assigned this Nov 24, 2020
@jaeger-j
Copy link
Collaborator

Hi @jeremyadamsfisher,
thank you for your contribution. Your proposed solution will be released with version 1.3.0.
It would be great if you write some unit test.
Best,
Jonas

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants