Code

Function server#1641

Code

Merged

pseudotensor merged 31 commits intomainh2oai/h2ogpt:mainfrom function_serverh2oai/h2ogpt:function_serverCopy head branch name to clipboardMay 25, 2024

+1,196-277

Commits

Commits on May 23, 2024

• Function server

  pseudotensor

  committed
  96fb03f

• make split faster when chunking by faking tokenizer, too slow otherwise in some extreme limits

  pseudotensor

  committed
  9e20ca4

• Update FAQ

  pseudotensor

  committed
  6b101a1

• Handle path_to_docs via FastAPI

  pseudotensor

  committed
  42dddbc

• Ensure gen_kwargs filled for function server, fix various things.

  pseudotensor

  committed
  d1a2854

• Ensure return values

  pseudotensor

  committed
  8f69d4c

• Avoid duplicating some things in function server mode

  pseudotensor

  committed
  96664d4

• Fix condition if model_lock=[]

  pseudotensor

  committed
  2c5adc9

• Verbose

  pseudotensor

  committed
  94e5d6d

• Gradio_pdf

  pseudotensor

  committed
  0cb9caf

• Fix whitespace

  pseudotensor

  committed
  2da730c

• Adjust import

  pseudotensor

  committed
  05b627b

• Check auth

  pseudotensor

  committed
  65eec19

• Deal with auth

  pseudotensor

  committed
  ab2e927

• FAQ for parallel openai or function servers

  pseudotensor

  committed
  15dfc22

• Fix idefics2 context length

  pseudotensor

  committed
  3999f4a

• Minor

  pseudotensor

  committed
  db2fbf1

• Not explained, but seems idefics2 works ok in longer context out to about 8192

  pseudotensor

  committed
  5246de7

• Not explained, but seems idefics2 works ok in longer context out to about 4096

  pseudotensor

  committed
  1aab8a5

• idefics2 starts to mess up after 4k tokens input

  pseudotensor

  committed
  1cdaa56

• hide debug

  pseudotensor

  committed
  730c0a4

• Default of 4096

  pseudotensor

  committed
  83c647a

Commits on May 24, 2024

• Support sglang via text and sglang via its own langchain llm class

  pseudotensor

  committed
  ac7086a

• Deal with sglang/llava stop tokens

  pseudotensor

  committed
  e9ca8f0

• Support multiple images for sglang

  pseudotensor

  committed
  8c137c8

• More docs and add /v1 if not present for sglang OpenAI chat

  pseudotensor

  committed
  2604138

Commits on May 25, 2024

• Handle our own system prompt

  pseudotensor

  committed
  a744cd1

• Update test

  pseudotensor

  committed
  b7ead1e

• Update test

  pseudotensor

  committed
  0117122

• guided_json test code split gradio and openai

  pseudotensor

  committed
  674193e

• Update test

  pseudotensor

  committed
  373ccdd