Enhanced Generator

[ondrej-fabry]

Warning
The post is not complete and is being updated over time with more relevant info.

Note
This discussion contains info about development around making the generator enhancements.

Table of Contents

• Enhanced Generator

---

💠 Enhanced Generator

The Enhanced Generator is a development around a new features aiming to make the generator more extendable and improving generated code.

Here is overview of areas relevant to this development:

• Make the generator extendable - support loading external plugins
• Generate meta info about the VPP API files - each generated API file will include structured metadata (version, hash, name..)
• Add compatibility checking for the generated API - run check automatically for the used API (prevent incompatible clients)
• Improve generator front-end - make it more user-friendly and dead simple to use (accepting various formats of VPP API input source: local build, installed VPP, specific release/commit..)
• Test generator against multiple VPP versions - run CI tests for the generator against supported VPP version (3 most recent releases) and also against master

Generator Extensions

Generator Extensions refers to custom generator plugins loaded from file. This feature will allow users to generate their own code along with the existing generated code bindings and enable new methods for developing and generating custom specialized VPP API clients.

Planned Tasks

This is quick overview of the planned tasks and their current status:

• universal input target parameter to binapi generator (support for API DIR, local VPP repo DIR..)
• update Nathan's PRs in GoVPP related to generator improvements
• update vpplink code to leverage all features of GoVPP and optimize the code
  • remove unneeded mutex locks
  • use generated RPC client for calling the VPP API
  • remove redundant code (retval value checks)
  • make the error messages consistent
• refactor Calico's vpp-dataplane usage of vpplink (update of Nathan's PR)
• move some common functionality to upstream (from vpp-dataplane to vpplink)
• optimize and cleanup the remaining of vpp-dataplane/vpplink code
• TBD

References

Documents

• Google Doc from Nathan (Mar 2021) - GoVPP improvements

Pull Requests

• FDio/govpp repository
  • Enhancements for binapigen #93
  • Move plugins to own package & improve apigen CLI #29
  • binapigen cli improvements #67
  • Move plugins to their own module #68
• calico-vpp/vpplink repository - extension of GoVPP generator
  • Refactor GoVPP usage and generation calico-vpp/vpplink#22
• projectcalico/vpp-dataplane repository - vpplink consumer usage example
  • Leverage govpp templates from vpplink projectcalico/vpp-dataplane#430
  • Refactor vpplink projectcalico/vpp-dataplane#519

Code

• Repository calico-vpp/vpplink - links to relevant code
  • Common types - api/v0
  • Wrapper generator - pkg/wrappergen
  • Templates for wrappergen - pkg/templates
  • GoVPP generator plugin implementation - pkg

Details / Spec

🔹 Generator Plugin Extension (more...)

Generator Plugin Extension

Generator Plugin Extension refers to the plugins loaded from a binary file, which must be a compiled Go package containing GenerateFile function. This file is loaded using plugin package and executed after internal generator plugins (API, RPC..).

🚧 TBD

🔹 Input Source (more...)

Input Source

Input Source refers to the source of the VPP API as input for the generator.

For the CLI front-end of the generator it's an option (flag) --input=<INPUT>. Using a single parameter to define the input this opens up possibility to to accept various types of sources which makes it very convenient and simple for the user.

Depending on the value of <INPUT> we can determine the source used to resolve VPP API files:

1. input="" - is not set (default value)
   • check if VPP is installed
   • load API files from installed VPP or return error if not installed
   • generate
2. input=<API_DIR> - is a path to directory with core and plugins directories
   • then assume it is the API directory
   • load API files from input
   • generate
3. input=<VPP_DIR> - is a path to directory with VPP source code (cloned git repository)
   • then assume it is the VPP directory we do development on
   • run make json-api-files
   • load API files from build-root/install-vpp-native/vpp/share/vpp/api/ (relative to input)
   • generate
4. input=<VPP_REPO_REF> - is a URL to a remote VPP repository / commit
   • then clone the repository into temp dir
   • continue as in step 3
5. input=<VPP_VERSION> - is a VPP version / branch name
   • then clone the VPP repository into temp dir
   • checkout tag or branch specified by input
   • continue as in step 3

🚧 TBD