feat(langchain): improve callbacks

[tibor-reiss]

Builds on top of #1317 which should be merged first:

• Added (a)stream and (a)transform to wrapped function list.

• Removed "instance" logic.

• I have added tests that cover my changes.

• If adding a new instrumentation or changing an existing one, I've added screenshots from some observability platform showing the change.

• PR name follows conventional commits format: feat(instrumentation): ... or fix(instrumentation): ....

• (If applicable) I have updated the documentation accordingly.

[nirga]

@tibor-reiss it's not that the OpenAI span is not generated - it's generated on a separate trace, which is a problem. It should be on the same trace as the rest of the LangChain spans:

[tibor-reiss]

@tibor-reiss it's not that the OpenAI span is not generated - it's generated on a separate trace, which is a problem. It should be on the same trace as the rest of the LangChain spans:

Hi @nirga, could you please share your code which resulted in this?

I checked several places and noticed that this seems to be already broken in main. E.g. in test_lcel / test_streaming (I renamed this test to test_invoke in this PR).

The issue seems to come from that BaseChatModel.generate is called. This is still in the non-callback world. It has a different signature (no config in 2nd place), so adding it to SYNC_FUNCTIONS is not enough - probably needs a separate logic to use it's 3rd argument, callbacks - I'll look into it asap...

[nirga]

Hmm @tibor-reiss might be new since we introduced the callbacks, cause this used to work with the previous mechanism

[tibor-reiss]

Hmm @tibor-reiss might be new since we introduced the callbacks, cause this used to work with the previous mechanism

@nirga Thought the same, so was just checking the stuff before the callbacks (https://github.com/tibor-reiss/openllmetry/blob/aaa303b9ffe6926d1830c51f911918d570ce2b1c/packages/opentelemetry-instrumentation-langchain/opentelemetry/instrumentation/langchain/__init__.py): the test test_simple_lcel already has this problem, i.e. openai_chat is not part of the workflow.

[nirga]

@tibor-reiss I think I'm going to merge this anyway since it's an improvement over what we currently have. Long term, I think we'll need to figure out how to combine the "previous" way with this one, since it seems with the callback way of instrumenting things we're losing the context across the trace (as I've shown in the example above).