Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(container): update image quay.io/go-skynet/local-ai to v2.19.1 #6775

Merged

Conversation

rosey-the-renovator-bot[bot]
Copy link
Contributor

@rosey-the-renovator-bot rosey-the-renovator-bot bot commented Jul 19, 2024

This PR contains the following updates:

Package Update Change
quay.io/go-skynet/local-ai minor v2.18.1-aio-gpu-nvidia-cuda-12 -> v2.19.1-aio-gpu-nvidia-cuda-12

Release Notes

mudler/LocalAI (quay.io/go-skynet/local-ai)

v2.19.1

Compare Source

local-ai-release-219-shadow

LocalAI 2.19.0 is out! 📣

TLDR; Summary spotlight

  • 🖧 Federated Instances via P2P: LocalAI now supports federated instances with P2P, offering both load-balanced and non-load-balanced options.
  • 🎛️ P2P Dashboard: A new dashboard to guide and assist in setting up P2P instances with auto-discovery using shared tokens.
  • 🔊 TTS Integration: Text-to-Speech (TTS) is now included in the binary releases.
  • 🛠️ Enhanced Installer: The installer script now supports setting up federated instances.
  • 📥 Model Pulling: Models can now be pulled directly via URL.
  • 🖼️ WebUI Enhancements: Visual improvements and cleanups to the WebUI and model lists.
  • 🧠 llama-cpp Backend: The llama-cpp (grpc) backend now supports embedding ( https://localai.io/features/embeddings/#llamacpp-embeddings )
  • ⚙️ Tool Support: Small enhancements to tools with disabled grammars.

🖧 LocalAI Federation and AI swarms

LocalAI is revolutionizing the future of distributed AI workloads by making it simpler and more accessible. No more complex setups, Docker or Kubernetes configurations – LocalAI allows you to create your own AI cluster with minimal friction. By auto-discovering and sharing work or weights of the LLM model across your existing devices, LocalAI aims to scale both horizontally and vertically with ease.

How it works?

Starting LocalAI with --p2p generates a shared token for connecting multiple instances: and that's all you need to create AI clusters, eliminating the need for intricate network setups. Simply navigate to the "Swarm" section in the WebUI and follow the on-screen instructions.

For fully shared instances, initiate LocalAI with --p2p --federated and adhere to the Swarm section's guidance. This feature, while still experimental, offers a tech preview quality experience.

Federated LocalAI

Launch multiple LocalAI instances and cluster them together to share requests across the cluster. The "Swarm" tab in the WebUI provides one-liner instructions on connecting various LocalAI instances using a shared token. Instances will auto-discover each other, even across different networks.

346663124-1d2324fd-8b55-4fa2-9856-721a467969c2

Check out a demonstration video: Watch now

LocalAI P2P Workers

Distribute weights across nodes by starting multiple LocalAI workers, currently available only on the llama.cpp backend, with plans to expand to other backends soon.

346663124-1d2324fd-8b55-4fa2-9856-721a467969c2

Check out a demonstration video: Watch now

What's Changed

Bug fixes 🐛
🖧 P2P area
Exciting New Features 🎉
🧠 Models
📖 Documentation and examples
👒 Dependencies
Other Changes

New Contributors

Full Changelog: mudler/LocalAI@v2.18.1...v2.19.0

v2.19.0

Compare Source

local-ai-release-219-shadow

LocalAI 2.19.0 is out! 📣

TLDR; Summary spotlight

  • 🖧 Federated Instances via P2P: LocalAI now supports federated instances with P2P, offering both load-balanced and non-load-balanced options.
  • 🎛️ P2P Dashboard: A new dashboard to guide and assist in setting up P2P instances with auto-discovery using shared tokens.
  • 🔊 TTS Integration: Text-to-Speech (TTS) is now included in the binary releases.
  • 🛠️ Enhanced Installer: The installer script now supports setting up federated instances.
  • 📥 Model Pulling: Models can now be pulled directly via URL.
  • 🖼️ WebUI Enhancements: Visual improvements and cleanups to the WebUI and model lists.
  • 🧠 llama-cpp Backend: The llama-cpp (grpc) backend now supports embedding ( https://localai.io/features/embeddings/#llamacpp-embeddings )
  • ⚙️ Tool Support: Small enhancements to tools with disabled grammars.

🖧 LocalAI Federation and AI swarms

LocalAI is revolutionizing the future of distributed AI workloads by making it simpler and more accessible. No more complex setups, Docker or Kubernetes configurations – LocalAI allows you to create your own AI cluster with minimal friction. By auto-discovering and sharing work or weights of the LLM model across your existing devices, LocalAI aims to scale both horizontally and vertically with ease.

How it works?

Starting LocalAI with --p2p generates a shared token for connecting multiple instances: and that's all you need to create AI clusters, eliminating the need for intricate network setups. Simply navigate to the "Swarm" section in the WebUI and follow the on-screen instructions.

For fully shared instances, initiate LocalAI with --p2p --federated and adhere to the Swarm section's guidance. This feature, while still experimental, offers a tech preview quality experience.

Federated LocalAI

Launch multiple LocalAI instances and cluster them together to share requests across the cluster. The "Swarm" tab in the WebUI provides one-liner instructions on connecting various LocalAI instances using a shared token. Instances will auto-discover each other, even across different networks.

346663124-1d2324fd-8b55-4fa2-9856-721a467969c2

Check out a demonstration video: Watch now

LocalAI P2P Workers

Distribute weights across nodes by starting multiple LocalAI workers, currently available only on the llama.cpp backend, with plans to expand to other backends soon.

346663124-1d2324fd-8b55-4fa2-9856-721a467969c2

Check out a demonstration video: Watch now

What's Changed

Bug fixes 🐛
🖧 P2P area
Exciting New Features 🎉
🧠 Models

Configuration

📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about this update again.


  • If you want to rebase/retry this PR, check this box

This PR has been generated by Renovate Bot.

@rosey-the-renovator-bot
Copy link
Contributor Author

rosey-the-renovator-bot bot commented Jul 19, 2024

--- HelmRelease: ai/localai Deployment: ai/localai-local-ai

+++ HelmRelease: ai/localai Deployment: ai/localai-local-ai

@@ -104,13 +104,13 @@

         - name: models
           mountPath: /models
         - name: output
           mountPath: /tmp/generated
       containers:
       - name: localai-local-ai
-        image: quay.io/go-skynet/local-ai:v2.18.1-aio-gpu-nvidia-cuda-12
+        image: quay.io/go-skynet/local-ai:v2.19.1-aio-gpu-nvidia-cuda-12
         imagePullPolicy: IfNotPresent
         resources:
           limits:
             memory: 10G
             nvidia.com/gpu: 1
           requests:

@rosey-the-renovator-bot
Copy link
Contributor Author

rosey-the-renovator-bot bot commented Jul 19, 2024

--- kubernetes/main/apps/ai/localai/app Kustomization: flux-system/ai-localai HelmRelease: ai/localai

+++ kubernetes/main/apps/ai/localai/app Kustomization: flux-system/ai-localai HelmRelease: ai/localai

@@ -43,13 +43,13 @@

         NVIDIA_DRIVER_CAPABILITIES: all
         NVIDIA_VISIBLE_DEVICES: all
         OPENAI_API_KEY: sk-XXXXXXXXXXXXXXXXXXXX
         debug: true
       image:
         repository: quay.io/go-skynet/local-ai
-        tag: v2.18.1-aio-gpu-nvidia-cuda-12
+        tag: v2.19.1-aio-gpu-nvidia-cuda-12
       runtimeClassName: nvidia
     ingress:
       annotations:
         hajimari.io/appName: LocalAI
         hajimari.io/enable: 'true'
         hajimari.io/group: AI

| datasource | package                    | from    | to      |
| ---------- | -------------------------- | ------- | ------- |
| docker     | quay.io/go-skynet/local-ai | v2.18.1 | v2.19.1 |
@rosey-the-renovator-bot rosey-the-renovator-bot bot force-pushed the renovate/quay.io-go-skynet-local-ai-2.x branch from 8555fa6 to f07a2ec Compare July 20, 2024 10:12
@rosey-the-renovator-bot rosey-the-renovator-bot bot changed the title feat(container): update image quay.io/go-skynet/local-ai to v2.19.0 feat(container): update image quay.io/go-skynet/local-ai to v2.19.1 Jul 20, 2024
@rosey-the-renovator-bot rosey-the-renovator-bot bot merged commit c50afe1 into main Jul 20, 2024
10 of 12 checks passed
@rosey-the-renovator-bot rosey-the-renovator-bot bot deleted the renovate/quay.io-go-skynet-local-ai-2.x branch July 20, 2024 20:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

0 participants