Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docs: Sync the BYOC doc with blog and update BentoCloud guides index page #4891

Merged
merged 2 commits into from
Jul 31, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 12 additions & 23 deletions docs/source/bentocloud/how-tos/byoc.rst
Original file line number Diff line number Diff line change
@@ -1,20 +1,19 @@
====================
Bring your own cloud
Bring Your Own Cloud
====================

BentoCloud provides Bring Your Own Cloud (BYOC) as a part of the Enterprise plan, which allows you to run BentoCloud services within your
private cloud environment. This means the BentoCloud Control Plane and the Data Plane are separated, enabling you to stay closer to your data
and meet any data compliance requirements. In addition, you are also able to leverage any available cloud credits and discounts.
BentoCloud's Bring Your Own Cloud (BYOC) option, part of the Enterprise plan, combines the advantages of both self-hosting and fully managed services. This means it not only ensures data privacy and security for you and but also helps you offload operations and maintenance burdens.

BentoCloud's BYOC offers a full featured AI inference platform which can be deployed into any organization's private cloud environment for maximum control and customization.

Architecture
------------

With BYOC, the data infrastructure and network are owned by you and managed by us. You can choose any of your cloud accounts.
We support platforms like AWS, Google Cloud, Microsoft Azure, and Oracle Cloud Infrastructure. The following diagram presents the BYOC architecture:
BYOC allows you to fully own your data infrastructure and network while we manage them for you. You can choose to host your services in any cloud account from providers such as AWS, Google Cloud, Microsoft Azure, or Oracle Cloud Infrastructure. The architecture diagram below illustrates the BYOC setup:

.. image:: ../../_static/img/bentocloud/get-started/byoc/bentocloud-byoc-architecture.png

When you choose BYOC, Bento operators and Deployments are deployed in your Virtual Private Cloud (VPC), ensuring that all data is securely stored in your own environment.
In the BYOC model, Bento operators and Deployments are deployed in your Virtual Private Cloud (VPC), ensuring that all data is securely stored in your own environment.
To create the necessary cloud resources required by operators within your account, BentoCloud requires relevant permissions to execute infrastructure-as-code automation.
This procedure automates the creation of resources like:

Expand All @@ -33,41 +32,31 @@ BYOC provides you with greater autonomy over your data infrastructure, catering
Data privacy
^^^^^^^^^^^^

Data privacy has become increasingly important in the age of artificial intelligence (AI). Generative AI, such as large language models (LLM),
often processes vast amounts of data that might include sensitive information. BYOC guarantees your data remains within your VPC,
thus ensuring your sensitive data never leaves the network.
Data privacy has become increasingly important in the age of AI. Generative AI, such as large language models (LLMs), often processes vast amounts of data that might include sensitive information. BYOC ensures your data and models remain within your VPC and that you can still use them with the latest foundational models and AI inference technologies.

Cost efficiency
^^^^^^^^^^^^^^^

The BYOC deployment option positions computational resources close to your data, effectively minimizing the hefty egress charges that might incur
due to data transfer out of your cloud account. For businesses that process vast volumes of data on a regular basis, executing model inference within your
cloud environment can lead to substantial cost savings.
BYOC instantiates computational resources in the same environment as your data, effectively minimizing the hefty egress charges that might incur due to data transfer out of your cloud account. For businesses that process vast volumes of data on a regular basis, executing model inference within your cloud environment can lead to substantial cost savings.

Credits and discounts
^^^^^^^^^^^^^^^^^^^^^

If you're involved with a startup or an incubator program, you likely qualify for cloud credits and discounts from leading cloud service providers.
To maximize the benefit of these credits and discounts, you must build an infrastructure that integrates with all these providers.
BentoCloud BYOC automatically generates cloud resources with your permissions and install operators for the different providers.
This offers you a consistent interface to manage Deployments across all cloud platforms and you are able to leverage any available credits or discounts on them.
If you're involved with a startup or an incubator program, you likely qualify for cloud credits and discounts from leading cloud service providers. To maximize the benefit of these credits and discounts, you must build an infrastructure that integrates with all these providers. BentoCloud's BYOC automatically generates cloud resources with your permissions and install operators for the different providers. This offers you a consistent interface to manage Deployments across all cloud platforms and you are able to leverage any available credits or discounts on them.

GPU availability
^^^^^^^^^^^^^^^^

For AI applications, GPUs are both a necessity and a rarity. With BYOC, you can select the cloud provider offering the most abundant and optimal GPU resources,
while also considering pricing or discount factors for better cost efficiency.
For AI applications, GPUs are both a necessity and a rarity. With BentoCloud's BYOC, you can select the cloud provider offering the most abundant and optimal GPU resources, while also considering pricing or discount factors for better cost efficiency.

No vendor lock-in
^^^^^^^^^^^^^^^^^

BentoCloud's architecture is built upon standard non-proprietary cloud utilities, ensuring compatibility with all general-purpose cloud providers.
This design ensures that your infrastructure stays independent from any specific cloud vendor, granting you the freedom to adapt, switch, or expand your cloud strategy as your needs evolve.
Such adaptability not only secures your operations for the future but also ensures you aren't tethered to a single provider's ecosystem.
BentoCloud's architecture is built with standard cloud components, ensuring it's compatible with all general-purpose cloud providers. This design ensures your infrastructure stays independent from any specific cloud vendor, granting you the freedom to adapt, switch, or expand your cloud strategy as your needs evolve.

Get started
-----------

As a part of the Enterprise plan, the BentoCloud BYOC deployment option is available to both new and existing customers.
You can choose to deploy it on AWS, Google Cloud, Microsoft Azure, Oracle Cloud Infrastructure, and other cloud platforms.
If you are interested in the plan, visit the `BentoCloud website <https://www.bentoml.com/cloud>`_ or `contact us <mailto:contact@bentoml.com>`_.
If you are interested in the plan, visit the `BentoML website <https://www.bentoml.com>`_ or `contact us <https://www.bentoml.com/contact>`_.
102 changes: 78 additions & 24 deletions docs/source/bentocloud/how-tos/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2,31 +2,85 @@
Guides
======

How-to guides take the reader through the steps required to solve a problem.
Read how-to guides to explore the key features of BentoCloud.

They are recipes, directions to achieve a specific end result, and are wholly **goal-oriented**.
.. grid:: 1 2 2 2
:gutter: 3
:margin: 0
:padding: 3 4 0 0

* :doc:`create-deployments`
* :doc:`configure-deployments`
* :doc:`manage-deployments`
* :doc:`call-deployment-endpoints`
* :doc:`autoscaling`
* :doc:`manage-access-token`
* :doc:`manage-secrets`
* :doc:`manage-users`
* :doc:`batch-inference-jobs`
* :doc:`byoc`
.. grid-item-card:: Create Deployments
:link: create-deployments
:link-type: doc

Create a Bento Deployment on BentoCloud.

.. grid-item-card:: Configure Deployments
:link: configure-deployments
:link-type: doc

Customize the configurations of your Deployment, such as scaling replicas, environment variables, and instance types.

.. grid-item-card:: Manage Deployments
:link: manage-deployments
:link-type: doc

Manage the Deployment lifecycle using the BentoML CLI or API.

.. grid-item-card:: Call Deployment endpoints
:link: call-deployment-endpoints
:link-type: doc

Run inference with Deployments.

.. grid-item-card:: Autoscaling
:link: autoscaling
:link-type: doc

Configure concurrency and autoscaling to achieve optimal resource utilization and cost-efficiency for your AI workloads.

.. grid-item-card:: Manage access tokens
:link: manage-access-token
:link-type: doc

Create and use API tokens to log in to BentoCloud or access protected Deployments.

.. grid-item-card:: Manage secrets
:link: manage-secrets
:link-type: doc

Store sensitive data like credentials in pre-defined secret templates or create custom secrets.

.. grid-item-card:: Manage users
:link: manage-users
:link-type: doc

Implement custom access control for BentoCloud users.

.. grid-item-card:: Batch inference jobs
:link: batch-inference-jobs
:link-type: doc

Run batch inference jobs with BentoML and BentoCloud.

.. grid-item-card:: Bring Your Own Cloud
:link: byoc
:link-type: doc

The BentoCloud BYOC deployment helps you run AI applications in your own environment in a secure and cost-effective way.

.. toctree::
:hidden:

create-deployments
configure-deployments
manage-deployments
call-deployment-endpoints
autoscaling
manage-access-token
manage-secrets
manage-users
batch-inference-jobs
byoc
:maxdepth: 1
:titlesonly:
:hidden:

create-deployments
configure-deployments
manage-deployments
call-deployment-endpoints
autoscaling
manage-access-token
manage-secrets
manage-users
batch-inference-jobs
byoc