Install all transitive dependencies for Python instrumentation

[rapphil]

Fixes: #1482

This change will install all Python transitive dependencies of the instrumentation supported by the lambda layer. This has to be done to reduce the friction for users, so that they are not required to install extra dependencies for their instrumentation to work.

There are some exceptions that will be handled separately because they are depending on a pinned version of opentelemetry-propagator-aws-xray. We wont need to keep those exceptions in the next release because it is already fixed open-telemetry/opentelemetry-python-contrib#2773

[ocelotl]

Since this is already merged we can make a new Python release and this won't be necessary.

[rapphil]

yes, this will work for v.0.48b0 (which is not released yet.). being merged does not mean that it is released.

[tylerbenson]

How much does moving all this to the lambda layer increase the layer size? Is that enough to be concerned about for adding to the cold start time?

[rapphil]

I simulated locally to check what is the delta:

without installing all dependencies

python3 -m pip install -r otel_sdk/requirements.txt -t ./deps && \
  python3 -m pip install -r otel_sdk/nodeps-requirements.txt -t ./deps --no-deps
du -h --max-depth 0 deps
25M     deps

installing all transitive dependencies

python3 -m pip install -r otel_sdk/requirements.txt -t ./deps && \
  python3 -m pip install -r otel_sdk/nodeps-requirements.txt -t ./deps --no-deps
du -h --max-depth 0 deps
27M     deps

The impact on the cold startup will be negligible.

[tylerbenson]

Looks like the last version was 7.4 MB. We can see where things end up with the next release.

[rapphil]

the data presented before regarding size was just for the dependencies and it was not zipped. Here is the resulting layer for reference:

8.3M Aug 22 15:51 build/opentelemetry-python-layer.zip

[tylerbenson]

So an added 1mb.... that's not insignificant. 🤷

[tylerbenson]

I assume the no-deps idea was to allow users to only add dependency for the specific instrumentation they want, but bundling it all makes it easier to keep versions in sync.