Warn about targets not matching any OpenAPI operation

[Kallys]

Hi!

It would be great if portman can show a warning when a target did not match any OpenAPI operation.
My use case is a big openapi definition split in multiple files, if someone in my team renames an operation id and forget to update portman configuration, associated tests will silently disapears.

Let's say for example someone rename operation getProducts to getAllProducts in OpenApi but config remains:

--
version: 1
tests:
  contentTests:
    - openApiOperationId: getProducts
      responseBodyTests:
        - key: 'products'
          minLength: 1
[...]

This content test will silently disapear from generated postman collection (which is, I think, dangerous).

[nicklloyd]

👍🏻 I like this idea - we could look at adding a config validation layer that could validate for this, and potentially other config issues.

[thim81]

linked to #632

[thim81]

hi @Kallys @nicklloyd
Would this be something you had in mind for the warning?

Question: what would be the default? That this warning is displayed or only when the --warn parameter is included?

[Kallys]

As far as I remember, this sounds great!
I would suggest producing warnings by default to stderr.
And either, providing a verbosity level adjustment option like --verbose=warn (by default) and a --verbose=error to disable warnings or a file redirection option like --warn=/dev/null (to disable warnings) or --warn=/path/to/warn.log.
This will help shell scripting for CI jobs.

[thim81]

hi @Kallys,

We just released version 1.30.5 of Portman, which includes the warning when a target does not exist in the OpenAPI spec.
Currently it only outputs the warning using console.log, so I'm not sure what would be the correct way to be used in CI jobs?

If you have suggestions to improvements, let us know in the discussions or open a new issue for it.

Closing this item, since a 1st version is released.