Possibility of Multi-Label Evaluation

[burakgzt]

Hi,
I'm working on a multi-label classification problem using LLMs and evaluating it with LangSmith using below snippet. Currently, I'm creating repetitions of the same record for each label, which is costly due to multiple inferences. Is there a way to optimize the evaluator for multi-label evaluation in a single pass while maintaining observability in the LangSmith UI?

def correct_label(root_run: Run, example: Example) -> dict:
    score = root_run.outputs.get("output") == example.outputs.get("label")
    return {"score": int(score), "key": "correct_label"}

@traceable
def label_text(inputs):
# do llm classification for given input
results = evaluate(
        lambda inputs: label_text(inputs),
        data=examples,
        evaluators=[correct_label],
        experiment_prefix="Safebear LLM Classification Evaluation",
        description="Validating the Groq classifications.",
    )

Thank you!