0.10.7 - fix gpu memory leak

What's Changed

Fix newly introduced regression where runners leak memory because we kill the parent process before cleaning up the children.

• fix runner gpu memory leak by @lukemarsden in #418

Full Changelog: 0.10.6...0.10.7

0.10.6 - improve logging

What's Changed

• Feature/llm logging middleware by @rusenask in #412
• Feature/llm logging middleware frontend by @lukemarsden in #414
• switch default tools model to one that exists by @lukemarsden in #415
• fix tests by @lukemarsden in #416

Full Changelog: 0.10.5...0.10.6

0.10.5 - fixes

What's Changed

• fix(api): fix unmarshalling of API key by @philwinder in #404
• fix: check for empty choices array by @rusenask in #407
• chore(ui): add waiting spinner when submitting a new request from an app by @philwinder in #406
• Feature/inspect app cmd by @rusenask in #408
• fix: text inference sessions restart using the openai controller by @philwinder in #409
• Add LLM logs in dashboard and fix tools bugs by @lukemarsden in #410

Full Changelog: 0.10.4...0.10.5

0.10.4 - declarative knowledge, custom rag server example, improve tools reliability

What's Changed

• Feature: declarative knowledge configuration by @rusenask in #399
• Feature/custom rag server by @rusenask in #402
• handle LLM sending backticks with no "json" by @lukemarsden in #403

Full Changelog: 0.10.3...0.10.4

0.10.3

What's Changed

Lots of fixes since big refactor in 0.10. Most notably, TOOLS_PROVIDER is no longer an option, it will use the same OpenAI client as INFERENCE_PROVIDER, which defaults to helix. What's more, when INFERENCE_PROVIDER=helix (now the default), API tools queries now work again. Sorry for the regression!

---

• Fix/api key auth for apps by @rusenask in #390
• Fix/rag source by @rusenask in #391
• Fix/system prompts when empty by @rusenask in #392
• Generic docs update by @chocobar in #394
• Update .env.example-prod by @lukemarsden in #395
• Update .env.example-prod by @lukemarsden in #396
• pointer to local-development.md by @lukemarsden in #397
• fix: app id from previous session by @rusenask in #398
• fix: internal tools client by @rusenask in #401

Full Changelog: 0.10.2...0.10.3

0.10.2 - frontend support for external LLM providers and stability fixes

What's Changed

The frontend and API now supports external LLM providers, configured by:

INFERENCE_PROVIDER=openai
OPENAI_API_KEY=sk-...
OPENAI_BASE_URL=<any OpenAI compatible API>

This works with RAG, API calling, and apps. You still need to attach a GPU runner for image inference, and text and image fine-tuning.

This release also improves stability using our own runners. A bug that caused models to be prematurely unloaded has been fixed. Another bug that caused more memory to be allocated than was available under load has also been fixed. Latency and availability of LLMs when using our runners should be improved.

RAG and system prompts now properly work with the new OpenAI compatible API path (for both external and local LLMs).

This release gives you full optionality over whether to use external LLMs or local HA ollama with our apps stack (RAG, API calling, system prompts - all configured via version controlled yaml), all running in docker or Kubernetes.

Next, we'll add function calling support so that gptscript can run fully locally.

---

• Fix/openai handler ctx by @rusenask in #385
• add /v1/models API and support external LLM providers in the frontend by @lukemarsden in #382
• Fix/rag by @rusenask in #388
• Fix/assistant sys prompt by @rusenask in #389
• Fix/conc error by @rusenask in #387

Full Changelog: 0.10.1...0.10.2

0.10.1

What's Changed

• text-finetune-test by @ObianuoObi in #379
• Feature/discord bot integration by @rusenask in #380
• fix/image type session fallback in the chat api by @rusenask in #383
• fix(api): version info comes from git, so must be included in the build by @philwinder in #384

Full Changelog: 0.10.0...0.10.1

0.10.0

What's Changed

• Feature/frontend tests by @ObianuoObi in #365
• psch yaml is the worst by @lukemarsden in #366
• handle case where hardlink target already exists by @lukemarsden in #367
• .env update by @ObianuoObi in #368
• image generation test by @ObianuoObi in #369
• feat(citations): add citations to the default prompt and fix UI bugs by @philwinder in #371
• chore(rag): add simple RAG dataprep and prompt tests by @philwinder in #372
• fully embrace non-obsolete version of docker compose by @lukemarsden in #373
• Feature/controller upgrade by @rusenask in #370
• never show [object Object] in snackbar by @lukemarsden in #377
• Feature/cli by @rusenask in #378

Full Changelog: 0.9.26...0.10.0

0.9.26

What's Changed

• chore/regen openapi spec by @rusenask in #363
• fix/lora dir filtering by @rusenask in #364

Full Changelog: 0.9.25...0.9.26

0.9.25

What's Changed

• feat: expose RUNTIME_AXOLOTL_ENABLED in runner chart by @JamesCallaghan in #360
• Fix/javascript types on session create by @rusenask in #362

New Contributors

• @JamesCallaghan made their first contribution in #360

Full Changelog: 0.9.24...0.9.25