v1.3.1

What's Changed

• docs: Add 2 examples by @Sherlock113 in #4872
• docs: update theme to match branding by @ssheng in #4874
• docs: Add task doc by @Sherlock113 in #4873
• fix: set prometheus env before importing service by @frostming in #4875
• docs: Add API docs by @Sherlock113 in #4877
• fix: inject prometheus env from arbiter by @frostming in #4878
• fix: Don't run bentoml.build_bentofile in subprocess by @frostming in #4876
• docs: update azure byoc instructions by @ssheng in #4881
• docs(azure): update azure byoc instructions by @ssheng in #4882
• fix(framework): transformers vs 1.3.0 "TypeError: init() got an unexpected keyword argument 'task'" by @frostming in #4880
• fix: silence git output by @frostming in #4883
• fix: surface config error on bento build by @frostming in #4888
• docs: Refactor tasks doc by @Sherlock113 in #4890
• Docs: Sync the BYOC doc with blog and update BentoCloud guides index page by @Sherlock113 in #4891
• fix: move user setup to base template to make custom base image work by @frostming in #4893
• fix: Cannot define custom duration histogram buckets via @bentoml.service(metrics=...) by @frostming in #4895

Full Changelog: v1.3.0...v1.3.1

v1.3.0

We are excited to announce the release of BentoML 1.3! Following the feedback received since the launch of 1.2 earlier this year, we are introducing a host of new features and enhancements in 1.3. Below are the key highlights of 1.3 and stay tuned for an upcoming blog post, where we'll provide a detailed exploration of the new features and the driving forces behind the development.

Here are some of the important points to note about 1.3:

• 1.3 ensures full backward compatibility, meaning that Bentos built with 1.2 will continue to work seamlessly with this release.
• We remain committed to supporting 1.2. Critical bug fixes and security updates will be backported to the 1.2 branch.
• The BentoML documentation has been updated with examples and guides for 1.3. More guides will be added in the coming weeks.
• BentoCloud supports Bento Deployments from both 1.2 and 1.3 releases of BentoML.

Now, let’s take a look at the major features and enhancements:

🕙 Implemented BentoML task execution

• Introduced the @bentoml.task decorator to set a task endpoint for executing a long-running workload (such as batch processing or video generation).
• Added the .submit() method to both the sync and async clients, which can submit task inputs via the task endpoint and dedicated worker processes constantly monitor task queues for new work to perform.
• Full compatibility with BentoCloud to run Bentos defined with task endpoints.
• See the Services and Clients doc with examples of a Service API by initializing a long running task in the Service constructor, creating clients to call the endpoint, and retrieving task status.

🚀 Optimized the build cache to accelerate the build process

• Enhanced build speed for bentoml build & containerize through pre-installed large packages like torch
• Switch to uv as the installer and resolver, replacing pip

🔨 Supported concurrency-based autoscaling on BentoCloud

• Added the concurrency configuration to the @bentoml.service decorator to set the ideal number of simultaneous requests a Service is designed to handle.
• Added the external_queue configuration to the @bentoml.service decorator to queue excess requests until they can be processed within the defined concurrency limits.
• See the documentation to configure concurrency and external queue.

🔒 Secure data handling with secrets in BentoCloud:

• You can now create and manage credentials, such as HuggingFace tokens and AWS secrets, securely on BentoCloud and easily apply them across multiple Deployments.
• Added secret subcommands to the BentoML CLI for secret management. Run bentoml secret -h to learn more.

🗒️ Added streamed logs for Bento image deployment.

• Easier to troubleshoot build issues and enable faster development iterations

🙏 Thank you for your continued support!

What's Changed

• fix: change forbid extra keys to false for bentocloud by @FogDong in #4866
• feat(dev): 1.3 by @frostming in #4849
• fix: delete cluster and ns if it is first cluster by @FogDong in #4869
• fix: auto login confirm ask logic by @xianml in #4864
• fix: secret default value by @xianml in #4870
• fix: fix typo in error msg by @FogDong in #4871

Full Changelog: v1.2.20...v1.3.0

v1.3.0a3

What's Changed

• fix: don't fail when image pod is not found by @frostming in #4862
• fix: async task response deserialization crash by @frostming in #4865

Full Changelog: v1.3.0a2...v1.3.0a3

v1.3.0a2

What's Changed

• fix: options are not required by @xianml in #4861

Full Changelog: v1.3.0a1...v1.3.0a2

v1.3.0a1

What's Changed

• feat: get_service api for selected frameworks by @frostming in #4782
• feat: cli auto login by @xianml in #4817
• feat: use rich.print to replace click.echo by @frostming in #4827
• chore: update print and fix cors error by @xianml in #4829
• feat: login to current context instead of default context if --context is not passed by @frostming in #4828
• feat: switch to uv as the resolver and installer by @frostming in #4822
• feat: Officially support Python 3.12 by @frostming in #4830
• Docs: Add auto cli login docs by @Sherlock113 in #4844
• feat: retry and cancel task by @frostming in #4852

Full Changelog: v1.2.20...v1.3.0a1

v1.2.20

What's Changed

• docs: Add logging and tracing docs by @Sherlock113 in #4725
• feat: --label option for bentoml build by @frostming in #4832
• docs: Update the concurrency doc by @Sherlock113 in #4834
• docs: Update services.rst by @vikramjit-sidhu in #4837
• docs: Update concurrency links by @Sherlock113 in #4840
• docs: Fix broken link by @Sherlock113 in #4841
• ci: pre-commit autoupdate [skip ci] by @pre-commit-ci in #4838
• fix: stop spinner before starting to stream logs by @frostming in #4846
• docs: Add azure byoc setup doc by @Sherlock113 in #4847
• docs: Update the BYOC architecture image by @Sherlock113 in #4848
• fix: fix error message with trace id by @FogDong in #4853
• doc: locking by @bojiang in #4850
• fix(containerize): command to create group and user in alpine base image by @frostming in #4855
• fix: encode special charaters in path by @frostming in #4854
• doc: locking for platform by @frostming in #4858
• fix: correct the params passed to keras.save_model by @frostming in #4857

New Contributors

• @vikramjit-sidhu made their first contribution in #4837

Full Changelog: v1.2.19...v1.2.20

v1.2.19

What's Changed

• docs: Update diffusion model repos to the new one by @Sherlock113 in #4797
• feat(monitoring): support GRPC LogExporter by @judahrand in #4808
• docs: Add Docker deployment for Bentos by @Sherlock113 in #4812
• docs: update examples service decorators with new features by @ssheng in #4798
• doc: Custom Error Handling by @jianshen92 in #4811
• doc: Wording change in error handling doc by @Sherlock113 in #4813
• fix: ASCII animation was not working properly in the image builder logs by @yetone in #4810
• fix: avoid editing list in during iteration by @judahrand in #4809
• docs: Add secrets doc by @Sherlock113 in #4814
• fix: improve the code of spinner and log tailer by @frostming in #4815
• doc: Update the docs about multipart output by @frostming in #4818
• docs: Add batch inference jobs doc by @Sherlock113 in #4816
• doc: Fix typo and format by @Sherlock113 in #4821
• fix(cli): hanging issue when deployment ready by @frostming in #4825
• fix(deployment-client): add build process and extend url by @Haivilo in #4790
• doc: Update vllm.rst by @Sherlock113 in #4824
• docs: Add custom models use case doc by @Sherlock113 in #4819
• docs: Reorg blip doc to multimodality section by @Sherlock113 in #4826
• fix: inject prometheus multiproc_dir into env var when worker starts by @frostming in #4820

Full Changelog: v1.2.18...v1.2.19

v1.2.18

What's Changed

• docs: scale from zero by @ssheng in #4769
• ci: pre-commit autoupdate [skip ci] by @pre-commit-ci in #4772
• docs: Update installation.rst by @parano in #4763
• docs: Add chatTTS to example project list by @Sherlock113 in #4777
• fix: bad performance when walking the file tree and check includes by @frostming in #4778
• update readme to include chattts by @ssheng in #4786
• docs: Update README.md by @parano in #4785
• Fixed FileExistsError: [Errno 17] File exists: '/root/bentoml' by @tripathiarpan20 in #4781
• fix: set env to null if it is empty for merge override by @FogDong in #4783
• fix: add none check for env for deploy by @FogDong in #4792
• fix(pandas): use correct mime type for Parquet by @judahrand in #4780
• fix: add memoization to pkg_version_info to reduce cpu overhead by @adrianlyjak in #4794
• fix: bug: memory leak when using bentoml>=1.2 by @frostming in #4775
• fix: honor host and port config in service decorator by @frostming in #4776
• fix(pandas): Annotated[pd.DataFrame, DataframeSchema(columns=["a"])] output spec by @judahrand in #4799
• fix: ensure relative path works within bento project by @aarnphm in #4803
• fix: map error responses from HTTP client to bentoml exceptions by @frostming in #4804
• fix(numpy): numpy annotation processing by @judahrand in #4795
• fix: populate bento name and version of server context by @frostming in #4787
• fix(cloud-client): consider httpcore timeout when getting status by @Haivilo in #4740
• fix: cd into src instead of bento parent by @frostming in #4805
• feat: show image builder logs by @yetone in #4796

New Contributors

• @tripathiarpan20 made their first contribution in #4781
• @adrianlyjak made their first contribution in #4794

Full Changelog: v1.2.17...v1.2.18

v1.2.17

What's Changed

• docs: update model store by @ssheng in #4741
• docs: Add env var for mistral 7b docs by @Sherlock113 in #4747
• refactor(tests): testing.util.async_request to client.AsyncHTTPClient by @sanspareilsmyn in #4746
• docs: Update GitHub readme by @Sherlock113 in #4749
• chore(build): update build_config.py by @eltociear in #4751
• docs: Add git package packing doc by @Sherlock113 in #4750
• fix: configure server logging before importing service by @frostming in #4744
• docs: Add user roles docs by @Sherlock113 in #4724
• feat(batching): break batch into smaller partitions if it's larger than max batch size by @frostming in #4752
• docs: Update the examples folder readme by @Sherlock113 in #4748
• docs: Add byoc setup docs by @Sherlock113 in #4757
• fix: new sdk e2e test by @xianml in #4758
• fix: metrics did not include 4xx by @xianml in #4756
• fix: should enable existing loggers in server logging config by @rickzx in #4759
• fix: client response deserialization when the output is string by @frostming in #4762
• fix: middleware reorder by @xianml in #4761
• fix: log errors inside a streaming response and detect service name conflicts by @frostming in #4767
• fix: don't try to validate non-bytes as Path by @frostming in #4765

New Contributors

• @sanspareilsmyn made their first contribution in #4746
• @eltociear made their first contribution in #4751
• @rickzx made their first contribution in #4759

Full Changelog: v1.2.16...v1.2.17

v1.2.16

What's Changed

• fix(build): make sure to parse pip_args in correct order by @aarnphm in #4722
• docs: Add compound data types doc by @Sherlock113 in #4721
• docs: Add docs for installing GitHub packages for Bento by @Sherlock113 in #4720
• docs: Update deployment details explanations by @Sherlock113 in #4726
• fix: load model aliases before loading new SDK service by @frostming in #4727
• docs: Fix cpu as strings by @Sherlock113 in #4735
• docs: Update the services doc by @Sherlock113 in #4734
• feat: add trace id to monitoring module by @jianshen92 in #4732
• feat: add a switch to opt-out packing git packages by @frostming in #4729
• fix: missing components from output schema if it is a root model by @frostming in #4733
• feat: expose the service url as a property by @frostming in #4730
• fix(container): preheat packages support markers by @bojiang in #4737

Full Changelog: v1.2.15...v1.2.16