Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate to new model + proxy from server #5

Merged
merged 27 commits into from
Apr 19, 2023
Merged

Migrate to new model + proxy from server #5

Show file tree
Hide file tree
Changes from 25 commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
e0063d6
use fork of hf.js to support fully custom endpoints for testing purpose
Grsmto Apr 14, 2023
300d3ac
proxy textGenerationStream call to the backend to hide token from client
Grsmto Apr 14, 2023
33fcc32
migrate to patch-package instead of pnpm
Grsmto Apr 17, 2023
157bd49
fix issue after merge conflict
Grsmto Apr 17, 2023
495778b
Merge branch 'main' into adrien-proxy-server
Grsmto Apr 17, 2023
edc565e
use env var instead of hardcoded value for endpoint
Grsmto Apr 17, 2023
2e455f6
fix messages not being split between assistant/user
Grsmto Apr 17, 2023
dd24928
fix stream response sometimes not split by token
Grsmto Apr 17, 2023
334788c
remove PUBLIC_ from private env variables + rename ENDPOINT to MODEL_…
Grsmto Apr 18, 2023
7cb33c0
only set hf token as private, model can stay public
Grsmto Apr 18, 2023
915e026
move HF_TOKEN to a dynamic env
Grsmto Apr 18, 2023
cd6ec88
fix env var import typo
Grsmto Apr 18, 2023
717bc29
remove @microsoft/fetch-event-source
Grsmto Apr 18, 2023
7fb0ce2
update parameters to be identical to Python demo
Grsmto Apr 18, 2023
ab5dec4
small refactor to avoid typing issue
Grsmto Apr 18, 2023
3e36610
cleanup while loop
Grsmto Apr 18, 2023
ff743d2
make comment clearer on what is happening on stream chunks split
Grsmto Apr 18, 2023
7902b8c
fix chunk spliting not being handled properly
Grsmto Apr 18, 2023
4601324
cleanup model tokens sometimes containing "<|endoftext|>" text
Grsmto Apr 18, 2023
285f326
refactor how we proxy from the server to simplify logic
Grsmto Apr 18, 2023
3c5c332
use latest version of hf.js
Grsmto Apr 18, 2023
c436c29
use .env + .env.local instead of .env.example
Grsmto Apr 18, 2023
a74996d
rewrite logic to trim "<|endoftext|>" artifact properly
Grsmto Apr 18, 2023
cd5734b
update to latest hf.js
Grsmto Apr 18, 2023
4abaed9
Merge branch 'main' into adrien-proxy-server
Grsmto Apr 18, 2023
19b9f82
expose env var to Docker during build time for deployment
Grsmto Apr 19, 2023
db89a18
remove patch-package
Grsmto Apr 19, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .env
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
PUBLIC_MODEL_ENDPOINT=https://api-inference.huggingface.co/models/OpenAssistant/oasst-sft-1-pythia-12b
PUBLIC_MODEL_NAME=OpenAssistant/oasst-sft-1-pythia-12b
PUBLIC_MODEL_TAGLINE=This is the first iteration English supervised-fine-tuning (SFT) model of the <a class="underline" href="https://github.com/LAION-AI/Open-Assistant">Open-Assistant</a> project. It is based on a Pythia 12B that was fine-tuned on ~22k human demonstrations of assistant conversations collected through the <a class="underline" href="https://open-assistant.io/">https://open-assistant.io/</a> human feedback web app before March 7, 2023.
PUBLIC_DISABLE_INTRO_TILES=true
PUBLIC_USER_MESSAGE_TOKEN=<|prompter|>
PUBLIC_ASSISTANT_MESSAGE_TOKEN=<|assistant|>
PUBLIC_SEP_TOKEN=<|endoftext|>
7 changes: 0 additions & 7 deletions .env.example
View file
Open in desktop

This file was deleted.

Loading