Migrate to new model + proxy from server #5

Grsmto · 2023-04-17T08:50:45Z

Migrate to new model
Proxy API calls to server to avoid exposing token client-side
Cleanup model responses

# Conflicts: # src/routes/+page.svelte

src/routes/+page.svelte

package.json

src/routes/api/conversation/+server.ts

…ENDPOINT

src/routes/api/conversation/+server.ts

src/routes/+page.svelte

Co-authored-by: Julien Chaumond <julien@huggingface.co>

coyotte508 · 2023-04-18T11:58:14Z

I published @huggingface/inference@2.0.0-rc1: https://www.npmjs.com/package/@huggingface/inference?activeTab=versions. Do not hesitate to ask for things like this!

src/routes/api/conversation/+server.ts

.env.example

# Conflicts: # .env

package.json

coyotte508

LGTM (after removing patch-package), let's iterate 🚀

* Order desc you.com results by text length * You.com already provides texts from webpages

* add you.com integration * [Feat] Add types (#1) * feat: add types * feat: specify search provider * [Feat] add return values (#3) * feat: add types * feat: specify search provider * add values for ui * add idea to gitignore * lint and downgrade * Revert "lint and downgrade" This reverts commit fbfe012. * Updates to you.com integration (#4) * `npm run format` * enable search when `YDC_API_KEY` is available * typing nit * enum WebSearchProvider * eslint-disable-next-line no-shadow * downgrade * You.com already provides text snippets (#5) * Order desc you.com results by text length * You.com already provides texts from webpages * add to env * Update .env Co-authored-by: Mishig <mishig.davaadorj@coloradocollege.edu> * Update src/lib/server/websearch/searchWeb.ts --------- Co-authored-by: David Li <david.li3100@gmail.com> Co-authored-by: Mishig <mishig.davaadorj@coloradocollege.edu>

* use fork of hf.js to support fully custom endpoints for testing purpose * proxy textGenerationStream call to the backend to hide token from client * migrate to patch-package instead of pnpm * fix issue after merge conflict * use env var instead of hardcoded value for endpoint * fix messages not being split between assistant/user * fix stream response sometimes not split by token * remove PUBLIC_ from private env variables + rename ENDPOINT to MODEL_ENDPOINT * only set hf token as private, model can stay public * move HF_TOKEN to a dynamic env * fix env var import typo * remove @microsoft/fetch-event-source * update parameters to be identical to Python demo * small refactor to avoid typing issue * cleanup while loop Co-authored-by: Julien Chaumond <julien@huggingface.co> * make comment clearer on what is happening on stream chunks split * fix chunk spliting not being handled properly * cleanup model tokens sometimes containing "<|endoftext|>" text * refactor how we proxy from the server to simplify logic * use latest version of hf.js * use .env + .env.local instead of .env.example * rewrite logic to trim "<|endoftext|>" artifact properly * update to latest hf.js * expose env var to Docker during build time for deployment * remove patch-package --------- Co-authored-by: Julien Chaumond <julien@huggingface.co>

* add you.com integration * [Feat] Add types (huggingface#1) * feat: add types * feat: specify search provider * [Feat] add return values (huggingface#3) * feat: add types * feat: specify search provider * add values for ui * add idea to gitignore * lint and downgrade * Revert "lint and downgrade" This reverts commit fbfe012. * Updates to you.com integration (huggingface#4) * `npm run format` * enable search when `YDC_API_KEY` is available * typing nit * enum WebSearchProvider * eslint-disable-next-line no-shadow * downgrade * You.com already provides text snippets (huggingface#5) * Order desc you.com results by text length * You.com already provides texts from webpages * add to env * Update .env Co-authored-by: Mishig <mishig.davaadorj@coloradocollege.edu> * Update src/lib/server/websearch/searchWeb.ts --------- Co-authored-by: David Li <david.li3100@gmail.com> Co-authored-by: Mishig <mishig.davaadorj@coloradocollege.edu>

Grsmto added 5 commits April 14, 2023 18:16

use fork of hf.js to support fully custom endpoints for testing purpose

e0063d6

proxy textGenerationStream call to the backend to hide token from client

300d3ac

migrate to patch-package instead of pnpm

33fcc32

fix issue after merge conflict

157bd49

Merge branch 'main' into adrien-proxy-server

495778b

# Conflicts: # src/routes/+page.svelte

Grsmto requested a review from gary149 April 17, 2023 08:51

gary149 requested a review from julien-c April 17, 2023 09:15

Grsmto added 3 commits April 17, 2023 10:42

use env var instead of hardcoded value for endpoint

edc565e

fix messages not being split between assistant/user

2e455f6

fix stream response sometimes not split by token

dd24928

Grsmto force-pushed the adrien-proxy-server branch from b5260af to dd24928 Compare April 18, 2023 08:42