Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix connector validation based on usage in pipelines #8004

Merged
merged 4 commits into from
Jul 10, 2023

Conversation

djaglowski
Copy link
Member

Alternate to #8003

Fixes ##7892

Validation of connectors was too aggressive such that a connector that was used in any combination of unsupported roles would fail. Instead, validation should pass as long as each use of the connector has a supported corresponding use.

For example, the forward connector may forward traces and metrics at the same time. Previously, validation would fail because it detected that traces->metrics and metrics->traces were possible connections. Now it will pass as long as there is a supported connection type for each pipeline in which the connector is used.

Validation of connectors was too aggressive in that it failed if a connector
was used in any combination of unsupported roles. Instead, it should pass
validation as long as each use of the connector has a supported corresponding
use.

For example, the forward connector may forward traces and metrics at the same
time. Previously, validation would fail because it detected that traces->metrics
and metrics->traces were possible connections. Now it will pass as long as there
is a supported connection type for each pipeline in which the connector is used.
@djaglowski
Copy link
Member Author

Validation of connectors is currently split between otelcol and service/internal/graph packages, roughly as follows:

otelcol: Obvious problems that can be reasoned about just be looking at the config.

  • Invalid connector type
  • Connector not used as both exporter and receiver

service/internal/graph: Less obvious problems that basically require building the graph / components.

  • Connector introduces cycle into graph
  • Connector config invalid

The validation added in this PR could arguably be added to either package, as it seems to fall somewhere in between, but having tried adding it to otelcol in #8003, I believe this is a better solution. I also pulled the "Connector not used as both exporter and receiver" logic from otelcol into this solution because it is very closely related.

@codecov
Copy link

codecov bot commented Jun 29, 2023

Codecov Report

Patch coverage: 98.68% and project coverage change: -0.10 ⚠️

Comparison is base (ec033ca) 90.90% compared to head (9fc673b) 90.80%.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #8004      +/-   ##
==========================================
- Coverage   90.90%   90.80%   -0.10%     
==========================================
  Files         300      300              
  Lines       15074    15146      +72     
==========================================
+ Hits        13703    13754      +51     
- Misses       1097     1110      +13     
- Partials      274      282       +8     
Impacted Files Coverage Δ
otelcol/config.go 100.00% <ø> (ø)
...rvice/internal/testcomponents/example_connector.go 100.00% <ø> (ø)
service/internal/graph/graph.go 98.66% <98.68%> (-0.03%) ⬇️

... and 3 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

@djaglowski djaglowski marked this pull request as ready for review June 29, 2023 20:37
@djaglowski djaglowski requested a review from a team as a code owner June 29, 2023 20:37
connector/connectortest/connector.go Outdated Show resolved Hide resolved
connector/connectortest/connector.go Outdated Show resolved Hide resolved
service/internal/graph/graph.go Outdated Show resolved Hide resolved
Copy link
Member

@mx-psi mx-psi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, one last thought that we don't need to address here:

Validation of connectors is currently split between otelcol and service/internal/graph packages

I am wondering if everything should be on service/internal/graph; if we are to eventually support #4970/#6888 it seems like the best solution is to expose the graph package, in which case all validation should happen there.

We don't need to solve this on this PR necessarily, since moving the validation around is not a breaking change.

@djaglowski
Copy link
Member Author

@mx-psi, thanks for the reviews.

w/r/t where validation is located, I think this PR effectively consolidates it into the graph package. The only exception now is the very high level checks like "is this a valid type of connector", which we still do in otelcol along with the other components.

@mx-psi
Copy link
Member

mx-psi commented Jul 6, 2023

which we still do in otelcol along with the other components.

alright, sounds like it's something we should solve independently then :)

@codeboten codeboten merged commit e1d0f26 into open-telemetry:main Jul 10, 2023
29 checks passed
@github-actions github-actions bot added this to the next release milestone Jul 10, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants