To have a controll on raw answers from LLMs before being parsed in doc.ents in NER tasks

[innocent-charles]

Hello! I was suggesting if it could be possible to also allow people to have the access control or be able to use the raw answers in NER returned by the LLMs just before being passed in the doc.ents.

Why ?
Because i have faced situations like LLMs return answer that are semantically correct but they are not as they have been written in the document. So this has caused error when those responses being taken to doc.ents .

Example :
i was extracting date of birth in the document : in the document it has been written as 20/11/1993 , when i check logs the answer was
month: November
day: 20th
year: 1993

which semantically, LLM has got it correct .......

But with this spacy LLM after final pipeline of doc.ents -- it seems that it can't give out those entities,

so was wandering on that.

[rmitsch]

Hi @innocent-charles, sometimes the reply provided by the LLM doesn't fit the data schema requested in the prompt. In that case parsing fails and we can't extract/map the data automatically. If you want raw access to the data, you can set save_io to True:

[components.llm]
factory = "llm"
save_io = True

[innocent-charles]

Then , where and how do i read response after save_io=True ?

…

On Mon, Nov 27, 2023 at 11:09 AM Raphael Mitsch ***@***.***> wrote: Hi @innocent-charles <https://github.com/innocent-charles>, sometimes the reply provided by the LLM doesn't fit the data schema requested in the prompt. In that case parsing fails and we can't extract/map the data automatically. If you want raw access to the data, you can set save_io to True: [components.llm]factory = "llm"save_io = True — Reply to this email directly, view it on GitHub <#385 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AQMYESRFTDZE2PXTAJ2A5K3YGRDD3AVCNFSM6AAAAAA725WLRGVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM3TMNZXHA3TS> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[rmitsch]

In Doc.user_data["llm_io"]. E. g.

doc = nlp("This is a test")
print(doc.user_data["llm_io"]["prompt"])  # Output raw prompt
print(doc.user_data["llm_io"]["response"])  # Output raw response

[innocent-charles]

Thanks a lot, Raphel, I have got it.

…

On Tue, Nov 28, 2023 at 11:13 AM Raphael Mitsch ***@***.***> wrote: In Doc.user_data["llm_io"]. — Reply to this email directly, view it on GitHub <#385 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AQMYESXJ3XQW5EEVBRRFI6TYGWMJ5AVCNFSM6AAAAAA725WLRGVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM3TMOBZGM4TK> . You are receiving this because you were mentioned.Message ID: ***@***.***>