feat(trace-otlp-grpc): configure security with env vars

[svetlanabrennan]

Which problem is this PR solving?

Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.

Partially Fixes #2706 (issue)

Currently the transport security for the gRPC exporter can be set either programmatically or the default insecure transport is used. This PR adds insecure environment variables and certificate file environment variables that related to security configuration.

Short description of the changes

• Added "insecure" and "certificate file" configuration env vars.
• Added configureSecurity function that sets up the transport security configuration.
• Updated default url from localhost:4317 to http://localhost:4317 as discussed during the SIG.
• Removed grpc(s) scheme and tests related to checking `grpc(s) scheme.

Type of change

Please delete options that are not relevant.

• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

• Unit Tests

Checklist:

• Followed the style guidelines of this project
• Unit tests have been added
• Documentation has been updated

TODO:

• Clarify how security should be configured in certain cases (scheme in endpoint, default security)
• Clarify using OTEL_EXPORTER_OTLP_SPAN_INSECURE as part of configureSecurity function.

[codecov]

Codecov Report

Merging #2827 (11df1a1) into main (22085fc) will decrease coverage by 0.00%.
The diff coverage is 92.72%.

@@            Coverage Diff             @@
##             main    #2827      +/-   ##
==========================================
- Coverage   92.52%   92.51%   -0.01%     
==========================================
  Files         183      183              
  Lines        5980     6029      +49     
  Branches     1268     1283      +15     
==========================================
+ Hits         5533     5578      +45     
- Misses        447      451       +4     

Impacted Files	Coverage Δ
...ckages/opentelemetry-core/src/utils/environment.ts	96.00% <ø> (ø)
...ental/packages/otlp-grpc-exporter-base/src/util.ts	78.26% <92.45%> (+11.07%)	⬆️
.../exporter-trace-otlp-grpc/src/OTLPTraceExporter.ts	100.00% <100.00%> (ø)
...porter-metrics-otlp-grpc/src/OTLPMetricExporter.ts	100.00% <100.00%> (ø)

[svetlanabrennan]

This PR doesn't break anything when it comes to setting up transport security.

I would like some feedback on the following items:

1. Spec says that the default "insecure" should be "false" which means a secure transport channel. The current default implementation (before this pr) is "insecure". Should we leave this as it or change it to match the spec that it should be secure as the default (which means it'll be a breaking change)?

2. Spec says that a scheme of https indicates a secure connection and takes precedence over the "insecure" config settings. The current implementation (before this pr) doesn't take the scheme in the endpoint into consideration at all. Should we add logic that takes scheme in endpoint into consideration in order to determine security. Here are some current use cases with the scheme endpoint of the current implementation (before this pr and continued with this pr):

A. using all defaults - no endpoint specified, no channel credentials specified => returns insecure channel
Note: spec says we should create secure channel but current implementation creates insecure

B. endpoint = localhost:4317, no channel credentials specified => returns insecure channel
Note: follows spec (no scheme means insecure channel)

C. endpoint = http://localhost:4317, no channel credentials specified => returns insecure channel
Note: follows spec (http scheme means insecure channel)

D. endpoint = https://localhost:4317, no channel credentials specified => returns insecure channel
Note: spec says this should be a secure channel but current implementation creates insecure

E. endpoint = grpc://localhost:4317, no channel credentials specified => returns insecure channel
Note: follows spec (http scheme means insecure channel)

F. endpoint = grpcs://localhost:4317, no channel credentials specified => returns insecure channel
Note: spec says this should be a secure channel but current implementation creates insecure

G. endpoint = https://localhost:4317, channel credentials (either secure or insecure) specified => return 
same `channelCredentials`
Note: there's no details from the spec on what to do in this case but I think that if a user specifically 
created channel credentials programmatically (either insecure or secure) that we should use those 
exact credentials no matter what scheme the endpoint contains. Also this is what is happening with 
the current implementation (before this pr and continued with this pr) - we use the security credentials 
set by the user. 

3. Spec mentions that OTEL_EXPORTER_OTLP_SPAN_INSECURE is obsolete but they are still supported because of a stable release of the spec. Should we read this OTEL_EXPORTER_OTLP_SPAN_INSECURE env var in my configureSecurity function?

[dyladan]

This PR doesn't break anything when it comes to setting up transport security.

I would like some feedback on the following items:

1. Spec says that the default "insecure" should be "false" which means a secure transport channel. The current default implementation (before this pr) is "insecure". Should we leave this as it or change it to match the spec that it should be secure as the default (which means it'll be a breaking change)?
2. Spec says that a scheme of https indicates a secure connection and takes precedence over the "insecure" config settings. The current implementation (before this pr) doesn't take the scheme in the endpoint into consideration at all. Should we add logic that takes scheme in endpoint into consideration in order to determine security. Here are some current use cases with the scheme endpoint of the current implementation (before this pr and continued with this pr):

A. using all defaults - no endpoint specified, no channel credentials specified => returns insecure channel
Note: spec says we should create secure channel but current implementation creates insecure

B. endpoint = localhost:4317, no channel credentials specified => returns insecure channel
Note: follows spec (no scheme means insecure channel)

C. endpoint = http://localhost:4317, no channel credentials specified => returns insecure channel
Note: follows spec (http scheme means insecure channel)

D. endpoint = https://localhost:4317, no channel credentials specified => returns insecure channel
Note: spec says this should be a secure channel but current implementation creates insecure

E. endpoint = grpc://localhost:4317, no channel credentials specified => returns insecure channel
Note: follows spec (http scheme means insecure channel)

F. endpoint = grpcs://localhost:4317, no channel credentials specified => returns insecure channel
Note: spec says this should be a secure channel but current implementation creates insecure

G. endpoint = https://localhost:4317, channel credentials (either secure or insecure) specified => return 
same `channelCredentials`
Note: there's no details from the spec on what to do in this case but I think that if a user specifically 
created channel credentials programmatically (either insecure or secure) that we should use those 
exact credentials no matter what scheme the endpoint contains. Also this is what is happening with 
the current implementation (before this pr and continued with this pr) - we use the security credentials 
set by the user. 

3. Spec mentions that OTEL_EXPORTER_OTLP_SPAN_INSECURE is obsolete but they are still supported because of a stable release of the spec. Should we read this OTEL_EXPORTER_OTLP_SPAN_INSECURE env var in my configureSecurity function?

Discussed in SIG yesterday and spent a little more time reading the spec today. The spec states "The endpoint SHOULD accept any form allowed by the underlying gRPC client implementation" and also only specifies https scheme as overriding the secure config. Given that, I think we should respect the insecure config in all cases (including secure by default) except when the scheme is https where the spec specifically says we should be secure. The only thing I'm not sure about is default behavior if nothing is configured because a secure connection to localhost is likely to fail certificate validation. I have opened open-telemetry/opentelemetry-specification#2420 for this question.

[dyladan]

@svetlanabrennan what's the current status here?

[svetlanabrennan]

@svetlanabrennan what's the current status here?

I just need confirmations on these use cases before I push up my changes to this pr:

1. if user wants to use all defaults, return secure channel
2. anytime user provides their own credentials, use those same credentials (no matter the scheme or env insecure settings)
3. if user sets https scheme return secure channel (ignoring insecure env settings)
4. if user sets http scheme and no other items (like providing credentials or env insecure settings) return insecure channel
5. if user sets no scheme and no other items (like providing credentials or env insecure settings) return secure channel
6. if user sets http scheme and env insecure = false, return secure channel
7. if user sets http scheme and env insecure = true, return insecure channel
8. if user sets no scheme and env insecure = false, return secure channel
9. if user sets no scheme and env insecure = true, return insecure channel

Do you (and anyone else) agree with this configuration?

[dyladan]

• if user wants to use all defaults, return secure channel

According to @tigrannajaryan open-telemetry/opentelemetry-specification#2420 it should be insecure to localhost. Tigran agrees spec should be clarified.

• anytime user provides their own credentials, use those same credentials (no matter the scheme or env insecure settings)

This seems reasonable

• if user sets https scheme return secure channel (ignoring insecure env settings)

Yes

• if user sets http scheme and no other items (like providing credentials or env insecure settings) return insecure channel

This is not as clear. I'll ask Tigran. The spec says insecure is false by default, but http implies insecure.

• if user sets no scheme and no other items (like providing credentials or env insecure settings) return secure channel

Yes

• if user sets http scheme and env insecure = false, return secure channel

Yes. I think this will also be the answer for "user sets http scheme and no other items" but we'll wait for Tigran.

• if user sets http scheme and env insecure = true, return insecure channel

Yes

• if user sets no scheme and env insecure = false, return secure channel

Yes

• if user sets no scheme and env insecure = true, return insecure channel

Yes

[svetlanabrennan]

Thank you for that feedback @dyladan. I'll wait for clarification on use case 4 and 6.

Note: this pr is also held up by the two lerna configurations issue.

[dyladan]

• if user wants to use all defaults, return secure channel

According to @tigrannajaryan open-telemetry/opentelemetry-specification#2420 it should be insecure to localhost. Tigran agrees spec should be clarified.

Tigran has clarified that the intention of the spec was for the default to be an insecure localhost connection.

• if user sets http scheme and no other items (like providing credentials or env insecure settings) return insecure channel

This is not as clear. I'll ask Tigran. The spec says insecure is false by default, but http implies insecure.

Tigran's comment here clarifies that endpoint with http scheme overrides insecure setting. Insecure setting is only used if no scheme is provided.

• if user sets http scheme and env insecure = false, return secure channel

Yes. I think this will also be the answer for "user sets http scheme and no other items" but we'll wait for Tigran.

The same clarification above also applies to this. The insecure config is only used if a scheme is not provided.

[svetlanabrennan]

The same clarification above also applies to this. The insecure config is only used if a scheme is not provided.

Perfect. Thanks for the clarification.

[dyladan]

Sorry for the delay on review here. i've got this on my list for the morning

[dyladan]

Thanks for doing the significant research and spec legwork on this. Sorry it took so long to get reviewed.

[legendecas]

This is great work, Thank you! LGTM % nits.

[legendecas]

Do we need to validate a hardcoded default URL?

[svetlanabrennan]

Yes because the DEFAULT_COLLECTOR_URL didn't have a scheme so we didn't need to pass it to validateAndNormalizeUrl like we do when user adds a url to config.url or when using env vars.

Now that I updated DEFAULT_COLLECTOR_URL to have an http scheme (like we discussed in the sig so it matches the spec) so we need to pass it to validateAndNormalizeUrl to remove the scheme.

[legendecas]

Ditto

[svetlanabrennan]

Same answer as this one