Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for _load_spec_from_json_module to use json.load() method to parse the .JSON specification. #1629

Open
smit-mehta25 opened this issue Jan 30, 2023 · 5 comments

Comments

@smit-mehta25
Copy link

smit-mehta25 commented Jan 30, 2023

Description

I have been using the connexion library for many years with the flask framework, I would like to highlight a few of the difficulties I have faced while using this package.

  • Support for splitting large YAML files into small files. Basically, we have aggregated the APIs based on the base_path and created multiple YAML modules. and, a main YAML for the end-points that are unique. later on, we faced performance issues as parsing YAML files (14-16 YAML modules) using prance was taking a huge time. So, we have written a script that will again merge the compiled/resolved specifications (prance.ResolvingParser(spec).specification) into a single JSON module. (we used the deepmerge lib to merge all resolved specifications) since prance resolves the references and the output is serialized into the dictionary we have to use the JSON module. The performance issues are still there as the connexion uses the yaml.safe_load() to load the specification from JSON modules.

the process of merging all the compiled/resolved specifications into a single entry point will be done before the container builds to save processing time. and once the container builds the JSON module will be consumed by the connexion_app.add_api method at the start up time.

@hjacobs @RobbeSneyders

Expected behavior

In short, the connexion library should support _load_spec_from_json_module using json.load() or provide a way to use custom parse since the performance of each parser is different. Also, the argument param does not work with the specification passed as a dictionary while using the connexion_app.add_api method

Actual behavior

As of now, the connexion library does not support a way to accept specifications as a path to the JSON module and use the json.load parser instead of the yaml.safe_load method, while using the add_api method. Also, the argument parameter should work for JSON specification as well for dict specification.

Steps to reproduce

Additional info:

Output of the commands:

  • python --version - python3
  • pip show connexion | grep "^Version\:" connexion version 2.14.2
@smit-mehta25
Copy link
Author

@hjacobs @RobbeSneyders FYI,

@smit-mehta25
Copy link
Author

@RobbeSneyders
Copy link
Member

Hi @smit-mehta25,

In short, the connexion library should support _load_spec_from_json_module using json.load() or provide a way to use custom parse since the performance of each parser is different

Would this still be relevant if Connexion would support splitting the specification into multiple files? Or would you still need it to merge the specifications before building your container to save on startup time?

Also, the argument param does not work with the specification passed as a dictionary while using the connexion_app.add_api method

This isn't possible, as Jinja templating needs to be resolved before parsing to a dict.

@smit-mehta25
Copy link
Author

@RobbeSneyders Hello,

I am looking for performance improvement, splitting and merging the specifications are features if the connexion supports its plus one. I would suggest supporting a JSON parse in connexion as ultimately the add_api method accepts a dictionary and currently, connexion uses yaml.safe_load() to parse the JSON module. having a JSON parser could improve performance as it may possible that the specification consumed by the connexion could have 1000s of end-points. Which results in performance issues.

Let me know if i could be more useful in implementing these features. Happy to contribute in each possible way.

@RobbeSneyders
Copy link
Member

Ok thanks for the feedback. We'd welcome a contribution on this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants