Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Benchmark gpt-tfjs integration #657

Closed
3 of 5 tasks
JulienVig opened this issue Apr 3, 2024 · 0 comments · Fixed by #659
Closed
3 of 5 tasks

Benchmark gpt-tfjs integration #657

JulienVig opened this issue Apr 3, 2024 · 0 comments · Fixed by #659
Assignees
@JulienVig
Labels
discojs Related to Disco.js documentation Improvements or additions to documentation
Milestone
v3.0.0

Comments

@JulienVig
Copy link
Collaborator

JulienVig commented Apr 3, 2024 •
edited
Loading

Benchmark and report some information on what to expect and the row for improvement:

  • number of training iterations needed on an example dataset, expected loss/perplexity, example of text generation after training
  • number of tokens/sec
  • model sizes
  • memory requirements
  • compare numbers with ONNX

#656 should be addressed first
@JulienVig JulienVig added documentation Improvements or additions to documentation discojs Related to Disco.js labels Apr 3, 2024
This was referenced Apr 3, 2024
Open
Merged
@JulienVig JulienVig self-assigned this Apr 16, 2024
@JulienVig JulienVig mentioned this issue Apr 16, 2024
Merged
@martinjaggi martinjaggi added this to the v3.0.0 milestone Jul 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@JulienVig JulienVig

Labels
discojs Related to Disco.js documentation Improvements or additions to documentation
Projects
None yet
Milestone
v3.0.0
Development

Successfully merging a pull request may close this issue.

Benchmark GPT-tfjs epfml/disco
2 participants
@martinjaggi @JulienVig