Feature: prompt tuning

[bdecarne]

Hello !

Great project, so much easy to understand/hack than the microsoft one

Is there any plan to support the prompt tuning feature ? https://microsoft.github.io/graphrag/posts/prompt_tuning/overview/

It would be really nice !

[gusye1234]

Yeah, @NumberChiffre is actually on this right now. We already merged some PRs about prompt tuning and keep working on it.
#44
#27

Stay tuned！

[NumberChiffre]

@bdecarne I'll take a look sometimes next week, in the meantime if you are interested in optimizing prompt instructions for entity and relationship extraction, take a look at:

• Generating datasets: https://github.com/gusye1234/nano-graphrag/blob/main/examples/generate_entity_relationship_dspy.ipynb
• Optimization: https://github.com/gusye1234/nano-graphrag/blob/main/examples/finetune_entity_relationship_dspy.ipynb

I used dspy-ai=2.4.16 for the above notebooks^

[bdecarne]

Thank you so much, it's look promising !!

[NumberChiffre]

@bdecarne some updates were made with self-refine, you can re-run the above Jupyter notebooks with your datasets to optimize your prompt instructions or use other DSPy optimizers:
#73