Releases · kubeflow/pipelines

14 Nov 22:08

chensun

sdk-2.10.1

8faa24d

KFP SDK v2.10.1 Latest

Latest

Release of the KFP SDK only.

To install the KFP SDK:

pip install kfp==2.10.1

For changelog, see release notes.

Assets 2

08 Nov 17:39

chensun

sdk-2.10.0

7b26ca8

KFP SDK v2.10.0

Release of the KFP SDK only.

To install the KFP SDK:

pip install kfp==2.10.0

For changelog, see release notes.

Assets 2

09 Sep 23:24

chensun

sdk-2.9.0

d287403

KFP SDK v2.9.0

Release of the KFP SDK only.

To install the KFP SDK:

pip install kfp==2.9.0

For changelog, see release notes.

Assets 2

06 Sep 23:16

chensun

2.3.0

4467df5

Version 2.3.0

To deploy Kubeflow Pipelines in an existing cluster, follow the instruction in here or via UI here

Install python SDK (python 3.7 above) by running:

python3 -m pip install kfp kfp-server-api --upgrade

See the Change Log

What's Changed

chore: Update RELEASE.md by @chensun in #10767
fix(components): Add input param autorater_prompt_parameters to online_evaluation_pairwise component by @copybara-service in #10759
feat(components): create infer preprocessor component by @copybara-service in #10765
feat(components): create utility class for preprocessors and use it in rlhf preprocessor and infer preprocessor by @copybara-service in #10768
chore(sdk): drop Python 3.7 for kfp sdk by @rickyxie0929 in #10750
chore: Promote @rimolive as a backend approver by @chensun in #10780
feat(components): use preprocessor utility methods for the upload model graph by @copybara-service in #10774
chore: upgrade node to v18 by @HumairAK in #10794
feat(GH workflow): migrate periodic functional tests to GH actions by @shruti2522 in #10751
chore: Add Container Engine and Image Tag parameters to backend Makefiles by @gmfrasca in #10725
chore: add meaningful logs to client initialization. by @HumairAK in #10801
feat(kubernetes_platform): Update kubernetes_platform go package to include EnabledSharedMemory by @hsteude in #10703
feat(components): Retry on batch prediction internal errors in AutoSxS by @copybara-service in #10799
fix(components): Add staging and temp locations to prophet trainer component by @copybara-service in #10803
fix(components): Remove unused import function_based from infer pipeline by @copybara-service in #10802
docs(components): Bump the python version to 3.8 by @copybara-service in #10764
chore(components): GCPC 2.14.1 Release by @copybara-service in #10813
chore(components): Rollback GCPC 2.14.1 Release due to b/339911077 by @copybara-service in #10818
chore(components): Add TaskError proto by @copybara-service in #10823
chore(components): GCPC 2.14.1 Release by @copybara-service in #10821
feat(components): Create the write_user_defined_error function by @copybara-service in #10835
test: fix CI failure for periodic functional tests by @tmvfb in #10817
chore(docs): Fix link by @haoxins in #7533
chore: Update Periodic Functional Tests to run once a day by @DharmitD in #10842
test: fix version conflict for functional tests by @tmvfb in #10837
chore: server should skip sample load if path DNE by @DharmitD in #10833
fix(frontend): reduce list run latency by @droctothorpe in #10797
feat(components): Support parsing Gemini BP outputs in AutoSxS pipeline by @copybara-service in #10853
feat(components): Use GetModel integration test to manually test write_user_defined_error function by @copybara-service in #10843
docs: modified the ConfigMap as Env variable example in README.md with right key_name within use_config_map_as_env() block. by @vamsi-01 in #10855
fix(sdk): Add required auth scopes to RegistryClient for GCP service accounts credentials by @PChambino in #10819
chore(deps): bump requests from 2.31.0 to 2.32.2 in /backend/src/apiserver/visualization by @dependabot in #10858
chore: Change command to apply manifests in the docs by @rimolive in #10864
fix(manifests): Move metacontroller to the top in kustmization.yaml by @codablock in #10669
test: Migrate Backend tests to a GHA workflow by @DharmitD in #10871
feat(backend): mount EmptyDir volumes for launcher write locations by @HumairAK in #10857
docs(components): internal by @copybara-service in #10874
feat(components): Add role_field_name and model_name as input parameters to llm_evaluation_preprocessor component to support gemini model's input and output schema by @copybara-service in #10875
fix(components): add check and add log to call out the fallback to the default model checkpoint and remove the model checkpoint check condition in RLHF GCPC by @copybara-service in #10828
test: migrate kubeflow-pipeline-e2e-test to GitHub Actions by @hbelmiro in #10887
feat(components): Support dynamic machine type paramters in CustomTrainingJobOp by @KevinGrantLee in #10883
fix(kubernetes_platform): fix api-generator docker mount for SELinux by @gregsheremeta in #10890
feat(components): Add Starry Net forecasting pipeline to public preview by @copybara-service in #10888
chore(components): Update AutoSxS and RLHF image tags by @copybara-service in #10906
feat(components): Update Starry Net image tags by @copybara-service in #10920
chore: Updated Python image to 3.8 in backend/Dockerfile by @hbelmiro in #10941
Expose starry_net yaml to GitHub by @chensun in #10943
chore(sdk): release KFP SDK 2.8.0 by @chensun in #10940
chore: Upgrade Python version to 3.8 in readthedocs config by @chensun in #10945
feat(frontend&backend): Add UI support for object store customization and prefixes by @HumairAK in #10787
chore(components): Update AutoSxS and RLHF image tags by @copybara-service in #10958
chore(components): GCPC 2.15.0 Release by @copybara-service in #10951
docs(backend): Remove deprecated v2 compatibility mode docs by @droctothorpe in #10956
Upgrade go version to 1.21 by @DharmitD in #10911
chore: Add ability to mount self-signed certs to kfp by @DharmitD in #10849
feat(components): add persistent_resource_id to v1 GCPC custom job components/utils by @copybara-service in #10954
chore: Added @rimolive, @hbelmiro and @DharmitD as approvers and reviewers for GitHub Actions by @hbelmiro in #10963
test: Migrate KFP SDK YAPF test to a GHA Workflow by @DharmitD in #10961
test: Created a new GitHub action to reuse the creation of a KFP cluster by @hbelmiro in #10946
test: kubeflow-pipeline-upgrade-test migrated to GitHub Actions by @hbelmiro in #10932
test: Migrate Frontend tests to a GHA Workflow by @DharmitD in #10972
test: install kfp on github periodic functional tests workflow by @tmvfb in #10859
chore: Add an option to re-run GHA workflows by @rimolive in #10930
chore(components): Update Starry Net description and add nan- and zero-threshold args by @copybara-service in #10986
test: Moved KFP backend visualization tests to GitHub Actions by @hbelmiro in #10984
test: Migrate KFP SDK Upgrade Test to GHA by @DharmitD in #10989
test: Migrate KFP SDK Runtime Code Test to GHA by @DharmitD in #10991
chore(components): Bump image version for Structured Data pipelines by @copybara-service in #11000
chore(components): Fix bug due to protobuf library being upgraded by pinning protobuf version by @copybara-service in #11007
chore(components): Add target_field_name as input parameters to llm_evaluation_preprocessor component to support gemini model's input and output schema by @copybara-service in https://github.co...

Contributors

diegolovison, rimolive, and 26 other contributors

Assets 2

22 Jun 09:31

chensun

sdk-2.8.0

991a610

KFP SDK v2.8.0

Release of the KFP SDK only.

To install the KFP SDK:

pip install kfp==2.8.0

For changelog, see release notes.

Assets 2

30 Apr 21:35

chensun

2.2.0

dd59f48

Version 2.2.0

To deploy Kubeflow Pipelines in an existing cluster, follow the instruction in here or via UI here

Install python SDK (python 3.7 above) by running:

python3 -m pip install kfp kfp-server-api --upgrade

See the Change Log

What's Changed

feat(components): Report TensorBoard metrics for preview.llm.rlhf_pipeline in real time by @copybara-service in #10595
feat(kubernetes_platform): Update kubernetes_platform go package to include generic ephemerl volume by @abaland in #10602
fix(metadata envoy): upgrade envoy and config from 1.12 to 1.27 by @freefood89 in #10589
fix(components): Remove the unused resolve_data_paths from function_based by @copybara-service in #10606
chore: Add Question issue template by @rimolive in #10557
feat(kubernetes_platform): Update kubernetes_platform go package to include node affinities and pod (anti)affinities by @cjidboon94 in #10583
chore(components): Add test machine spec support to preview.llm pipelines by @copybara-service in #10616
fix(components): Ensure preview.llm.rlhf_pipeline runs if no tensorboard_id is provided by @copybara-service in #10626
fix(components): Remove the unused functions from function_based by @copybara-service in #10613
feat(components): Add model name preprocess component; Use publisher model if user uploaded model is non-tuned by @copybara-service in #10620
feat(components): add task_type as a parameter to rlaif by @copybara-service in #10607
feat(components): AutoSxS GA pending release by @copybara-service in #10628
fix(components): Update service account comment by @copybara-service in #10636
chore(components): GCPC 2.12.0 Release by @copybara-service in #10635
fix(samples): Fix the loop_output.py example to handle the new parallel loop type requirement by @Tomcli in #10637
feat(components): Added support for text-bison@002 to preview.llm.rlhf_pipeline by @copybara-service in #10641
chore(components): GCPC 2.13.0 Release by @copybara-service in #10644
fix(components): Make AutoSxS autorater_prompt_parameters required by @copybara-service in #10621
docs(components): internal by @copybara-service in #10649
chore(components): Update kserve component to v0.12.0 by @Tomcli in #10652
fix(backend): Update backend common code and integration tests with updated API Service Params by @gmfrasca in #10640
fix(Backend + SDK): Add missing optional field to SecretAsVolume and … by @revit13 in #10550
chore(manifests): refactor manifests for kustomize5 compatibility. Part of #10053 by @rawc0der in #10087
fix(components): Fix model eval import error in text generation/classification eval pipeline by @copybara-service in #10655
chore(components): GCPC 2.13.1 Release by @copybara-service in #10666
chore(deps): bump follow-redirects from 1.6.1 to 1.15.6 in /frontend by @dependabot in #10575
chore(deps): bump pillow from 10.0.1 to 10.3.0 in /backend/src/apiserver/visualization by @dependabot in #10658
fix(components): Remove the unused generate_default_instruction and resolve_upload_location from function_based by @copybara-service in #10638
chore(deps): bump express from 4.18.2 to 4.19.2 in /frontend by @dependabot in #10639
chore: Update argo images to 3.4.16 by @chensun in #10618
feat(components): Use larger base reward model when tuning text and chat variants of bison@001 with the preview.llm.rlhf_pipeline by @copybara-service in #10663
chore(deps): bump follow-redirects from 1.5.10 to 1.15.6 in /frontend/server by @dependabot in #10574
chore(sample): add note about secret needing to be pre-created by @gregsheremeta in #10659
Chore(components): Clean up old ibm components by @Tomcli in #10680
chore(components): Update AutoSxS and RLHF image tags by @copybara-service in #10683
feat(sdk+backend): Add support for generic ephemeral volume by @abaland in #10605
feat(components): Use larger base reward model when tuning t5-xxl with the preview.llm.rlhf_pipeline by @copybara-service in #10665
chore(backend): Promote @rimolive as the backend reviewer by @Tomcli in #10689
chore(sdk): release kfp-kubernetes 1.2.0 by @connor-mccarthy in #10692
chore(sdk): make kfp-kubernetes release instructions public by @connor-mccarthy in #10693
chore(kfp-kubernetes): change type of affinity weight to int32 by @cjidboon94 in #10671
fix(components): Fix image version parameter in rl pipelines by @copybara-service in #10698
docs(components): internal by @copybara-service in #10694
feat(backend): Merge kfp-tekton backend code by @rimolive in #10678
feat(backend): Upgrade argo to v3.4.16 by @gmfrasca in #10568
feat(backend): add namespace & prefix scoped credentials to kfp-launcher config for object store paths by @HumairAK in #10625
chore(deps): bump sqlparse from 0.4.4 to 0.5.0 in /backend/src/apiserver/visualization by @dependabot in #10700
feat(components): add resolve_reference_model_metadata to rlhf_preprocessor component by @copybara-service in #10612
feat(components): Move AutoSxS pipeline to v1 directory by @copybara-service in #10701
feat(components): add resolve_machine_spec and resolve_refined_image_uri to rlhf_preprocessor component by @copybara-service in #10608
fix(components): remove default prediction column names in evaluation classification component to fix incorrect column names for bigquery data source by @copybara-service in #10708
feat(components): Introduce placeholders: SERVICE_ACCOUNT_PLACEHOLDER, NETWORK_PLACEHOLDER, PERSISTENT_RESOURCE_ID_PLACEHOLDER and ENCYRPTION_SPEC_KMS_KEY_NAME_PLACEHOLDER. In addition, use PERSISTENT_RESOURCE_ID_PLACEHOLDER as the default value of persistent_resource_id for CustomTrainingJobOp and create_custom_training_job_op_from_component. With this change, custom job created without explicitly setting persistent_resource_id will inherit job level persistent_resource_id, if Persistent Resource is set as job level runtime by @copybara-service in #10650
chore: Add kfp-tekton integration tests and manifests by @rimolive in #10702
No public description by @copybara-service in #10726
chore(components): Drop GCPC Python 3.7 by @copybara-service in #10730
feat(components): use rlhf_preprocessor to replace the current value_exists call in rlhf by @copybara-service in #10584
chore(components): Drop support for Python 3.7 in GCPC by @copybara-service in #10735
feat(components): internal by @copybara-service in #10707
feat(components): Expand regions supported by preview.llm.rlhf_pipeline by @copybara-service in #10710
chore(components): Update AutoSxS and RLHF image tags by @copybara-service in #10749
chore(components): Change the warning for Python 3.8 by @copybara-service in #10755
feat(components): migrate function_based resolve_num_microbatches to rlhf_preprocessor component by @copybara-service in #10604
chore(components): GCPC 2.14.0 Release by @copybara-service in #10754
feat(components): migrate function_based convert_to_delimited_string to rlhf_preprocessor component by @copybara-service in https://github.com/kube...

Contributors

rimolive, freefood89, and 11 other contributors

Assets 2

25 Mar 20:43

chensun

2.1.0

f328f0b

Version 2.1.0

To deploy Kubeflow Pipelines in an existing cluster, follow the instruction in here or via UI here

Install python SDK (python 3.7 above) by running:

python3 -m pip install kfp kfp-server-api --upgrade

See the Change Log

What's Changed

feat(components): Bump image tag used by preview.llm pipelines by @copybara-service in #10295
feat(components): Add num_microbatches to _implementation.llm training components by @copybara-service in #10248
chore(sdk): remove cleanup param in local init #localexecution by @connor-mccarthy in #10293
feat(sdk): add local execution skeleton #localexecution by @connor-mccarthy in #10292
fix(components): Use llama-2-7b for the base reward model when tuning llama-2-13 with the preview.llm.rlhf_pipeline by @copybara-service in #10249
feat(sdk): add local task dispatcher validation and ExecutorInput construction #localexecution by @connor-mccarthy in #10298
test(sdk): install kfp-pipeline-spec from source in kfp sdk tests by @connor-mccarthy in #10300
fix(sdk): use kfp.dsl.types to replace kfp.components.types Fixes #10282 by @hsinhoyeh in #10283
feat(components): Enable text generation pipeline to generate row based metrics by @copybara-service in #10296
test(sdk): add placeholder_utils_test.py by @connor-mccarthy in #10301
feat(component): Migrate AutoSxS pipeline to preview and move related files to _implementation/llm directory to help Model Eval team use side by side metrics as part of their pipeline by @copybara-service in #10294
feat(components): Release new LLM Eval image version 0.5 by @copybara-service in #10313
chore(components): release GCPC 2.7.0 by @copybara-service in #10303
fix(components): Add autosxs_pipeline to the all variable for the preview/model_evaluation directory by @copybara-service in #10317
chore(components): release GCPC 2.8.0 by @copybara-service in #10316
feat(components): Add better docstrings for AutoSxS by @copybara-service in #10320
test(components): INTERNAL by @copybara-service in #10314
feat(sdk): add subprocess task handler #localexecution by @connor-mccarthy in #10302
fix(components): Use large_model_reference as model_reference_name when uploading models from preview.llm.rlhf_pipeline instead of hardcoding value as text-bison@001 by @copybara-service in #10321
test(sdk): add kfp-kubernetes execution tests by @connor-mccarthy in #10304
fix(components): Resolve unique model display name on each preview.llm.rlhf_pipeline run instead of reusing cached result by @copybara-service in #10322
fix(components): Upload the tuned adapter to Model Registry instead of model checkpoint from preview.llm.rlhf_pipeline by @copybara-service in #10323
feat(sdk): add local execution output collection #localexecution by @connor-mccarthy in #10325
chore(sdk): fix use of invalid escape sequence in tests by @connor-mccarthy in #10310
feat(sdk): add local execution logging #localexecution by @connor-mccarthy in #10326
chore(components): Fix argument description by @copybara-service in #10327
feat(backend): preserve querystring in pipeline root (fixes #10318) by @TobiasGoerke in #10319
feat(components): change output format to allow possible post eval by @copybara-service in #10281
fix(frontend): Add disableParsingRawHTML option for markdown-to-jsx component by @zijianjoy in #10315
feat(sdk): add DockerRunner #localexecution by @connor-mccarthy in #10328
feat(sdk): add special dsl.OutputPath read logic #localexecution by @connor-mccarthy in #10334
chore(sdk): write local execution logs to stdout #localexecution by @connor-mccarthy in #10330
feat(sdk): support local Container Component execution #localexecution by @connor-mccarthy in #10333
feat(components): Update RLHF env vars to handle empty string by @copybara-service in #10331
feat(components): Output errors as a separate table from Arbiter by @copybara-service in #10329
docs(components): Fix AutoSxS docstring formatting by @copybara-service in #10341
chore(components): add json_escape placeholder util by @copybara-service in #10351
fix(sdk): fix presentation of strings in local execution #localexecution by @connor-mccarthy in #10353
fix(sdk): remove redundant newline character in local DockerRunner logs #localexecution by @connor-mccarthy in #10354
test(sdk): improve KFP SDK local runner test safety #localexecution by @connor-mccarthy in #10336
feat(backend): Enable logging for KFP components by @DharmitD in #10288
feat(sdk): remove local execution feature flag #localexecution by @connor-mccarthy in #10355
fix(sdk): permit empty local execution outputs #localexecution by @connor-mccarthy in #10338
feat(sdk): support Concat and IfPresent placeholder in local container component execution #localexecution by @connor-mccarthy in #10348
test(sdk): add test for local execution of None default parameter #localexecution by @connor-mccarthy in #10339
chore(sdk): release KFP SDK 2.5.0 by @connor-mccarthy in #10364
chore(sdk): depend on protobuf 4 in kfp-pipeline-spec by @connor-mccarthy in #10305
docs(components): Document AutoML Tables util functions by @copybara-service in #10359
feat(components): Implement the feature store grounding pipeline by @copybara-service in #10332
feat(components): update eval pipeline documentation to clarify the required pipeline parameters by @copybara-service in #10366
docs(components): update BigqueryQueryJobOp docs on container args limit by @copybara-service in #10380
chore(components): GCPC 2.8.1 release by @copybara-service in #10381
feat(kubernetes_platform): Update kubernetes_platform go package to include pod labels and annotations by @Tomcli in #10357
chore(sdk): depend on protobuf 4 in kfp and kfp-kubernetes by @connor-mccarthy in #10307
feat(components): Implement new output format of inference component by @copybara-service in #10375
chore(components): rollback GCPC 2.8.1 release by @copybara-service in #10385
chore(sdk): release KFP SDK 2.6.0 by @connor-mccarthy in #10386
chore(components): Add v1.model.ModelGetOp components by @copybara-service in #10226
chore(sdk): release kfp-kubernetes 1.1.0 by @connor-mccarthy in #10387
chore(sdk): update kfp-kubernetes docs versions and release scripts by @connor-mccarthy in #10388
docs(sdk): fix kfp-kubernetes docs build error by @connor-mccarthy in #10389
chore(components): Sync AutoML components by @copybara-service in #10372
fix(components): Update base image for KFP lightweight component for VPC SC compliance by @copybara-service in #10374
chore: Fix metrics visualization v2 sample by @rimolive in #10399
docs(sdk): add kfp.local to reference docs #localexecution by @connor-mccarthy in #10395
chore(components): migrate GCPC to protobuf 4; require KFP>=2.6.0 by @copybara-service in #10401
feat(components): Support scheduling and labels in utils.bui...