Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Demo: Fine-tuning and post train chat eval POC #401

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

nerdalert
Copy link
Member

@nerdalert nerdalert commented Dec 9, 2024

  • Demonstrates an end to end knowledge submission, generate, train and a post-training side-by-side model chat comparison, enabling the user to validate their knowledge submission is included in the newly trained checkpoint.
  • All aspects of the pipeline are done via the UI. No command-line operations are required at any stage.
  • For this to function for the frontend to make REST API calls to Instructlab via an api-server that frontends ilab until REST endpoints are provided via InstructLab natively. The code is here https://github.com/nerdalert/ilab-api-server
  • The demo was run on a 24GB GPU leveraging the simple pipeline. Will get an example accelerated/full pipeline demo with some hardware soon (which also produces a better tuned model).
  • Integrates the new Patternfly Chatbot component. Note: the node that ilab is running inference needs to have ports open for connectivity since as I haven't gotten the chat streaming working with Next.js app router server side rendering with the chatbot component yet.
  • Training and generation for the demo took around ~30-45m or so.
  • All functionality is decoupled from the system via REST making it serviceable out of the gate and enabling the UI functionality.
  • The knowledge submission was just a random new wiki that docling converted for the submission. It could be any topic of knowledge with accompanying documentation in the submission.
e2e-fine-tune-demo.mp4

@nerdalert nerdalert added the demo PR that contains Demo related changes label Dec 9, 2024
@nerdalert nerdalert marked this pull request as draft December 9, 2024 06:51
- Demonstrates an end to end knowledge submission, generate,
train and post-train side-by-side model comparison for the user
to validate their knowledge submission is included in the newly
trained checkpoint.
- For this to function for the frontend to make REST API calls to
Instructlab this uses an api-server that frontends ilab. The code
is here https://github.com/nerdalert/ilab-api-server
- The demo was run on a 24GB GPU leveraging the simple pipeline.
Will get an example acceslerated pipeline demo with some hardware
soon.
- Training and generation for the demo took around ~30-45m or so.
- All functionality is decoupled from the system via REST
making it serviceable out of the gate and enabling the UI
functionality.

Signed-off-by: Brent Salisbury <bsalisbu@redhat.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
demo PR that contains Demo related changes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant