You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I think a bigger advantage would be to do some refactoring in the existing examples and "hide" some of the state for sampling and KV cache management that we expose behind the common API
Hide some state for sampling (unsure what this mean)
Key Value cache management should be abstracted away behind the common API
If there is any other aspect of the example which is currently a pain point for developer grokking, feel free to also suggest some so it can be added here.
The text was updated successfully, but these errors were encountered:
GG in this PR was suggesting a refactoring of examples and provided an example of what he would like to refactor. This is a ticket to inform others about opportunities to improve examples
If there is any other aspect of the example which is currently a pain point for developer grokking, feel free to also suggest some so it can be added here.
The text was updated successfully, but these errors were encountered: