Releases: fluid-cloudnative/fluid
v1.0.1
New Features
- Support data flow affinity scheduling.
- Support dynamic updates for dataset mount points.
- Support data acceleration for Fluid Persistent Volume Claim in sidecar mode
What's Changed
- Prepare 1.0.1 by @cheyang in #3886
- fix jindo rbac remove secret get by @frankleaf in #3974
- support parallel tasks for cron datamigrate by @xliuqq in #3732
- Support to configure the networkMode and podMetadata of vineyard components. by @dashanji in #3981
- Bump golangci/golangci-lint-action from 4 to 5 by @dependabot in #4007
- Upgrade controller runtime to 0.14.7 and Kubernetes deps to 1.26.10 by @cheyang in #4068
- Use generic Set for volumeLock by @cheyang in #4070
- fix code scaning alerts —— Incomplete regular expression for hostnames by @xliuqq in #4052
- Upgrade k8s to 1.26.15 by @cheyang in #4077
- Update docker file golang 1.12.10 by @cheyang in #4083
- Fix security issue introduced in vineyard samples by @dashanji in #4096
- feat: allow to disable metadata sync for thinruntime by @TrafalgarZZZ in #4091
- Bump golangci/golangci-lint-action from 5 to 6 by @dependabot in #4093
- refactor: deprecate fuse global mode by @TrafalgarZZZ in #4097
- support dataflow affinity by @xliuqq in #3829
- Fix crd generation issue by @cheyang in #4099
- Bump github.com/golang/glog from 1.0.0 to 1.2.1 by @dependabot in #3798
- Bump google.golang.org/grpc from 1.56.3 to 1.64.0 by @dependabot in #4100
- Support to configure the cache size of vineyard fuse by @dashanji in #3982
- upgrade jindocache to 6.4.0 version by @frankleaf in #4126
- Bump golang.org/x/time from 0.3.0 to 0.5.0 by @dependabot in #4111
- Fix the inconsistent rpc socket port with assigned port in vineyard worker. by @dashanji in #4141
- refactor: remove deprecated data operation interface by @xliuqq in #4130
- Fix NPE of runtime controller by @cheyang in #4139
- Bump github.com/onsi/gomega from 1.27.10 to 1.33.1 by @dependabot in #4142
- Bump github.com/prometheus/client_golang from 1.16.0 to 1.19.1 by @dependabot in #4147
- fix default value by @frankleaf in #4144
- Bump up the vineyard to v0.22.2 by @dashanji in #4153
- Bump github.com/agiledragon/gomonkey/v2 from 2.10.1 to 2.11.0 by @dependabot in #4151
- addons: add dynamic mount examples by @TrafalgarZZZ in #4148
- bugfix: check corrupted mount point in utils.RemoveSymlink by @TrafalgarZZZ in #4163
- Bump github.com/spf13/cobra from 1.7.0 to 1.8.0 by @dependabot in #4158
- Bump go.uber.org/zap from 1.24.0 to 1.27.0 by @dependabot in #3727
- refactor: rename
CreateUpdatePodForSchedulingHandler
toFluidMutatingHandler
by @TrafalgarZZZ in #4168 - minorfix: pin libfuse version in dynamic mount example by @TrafalgarZZZ in #4167
- Refactor admission webhook handler registration for better error handling by @cheyang in #4170
- refactor: avoid passing namespace to mutation functions by @TrafalgarZZZ in #4171
- minorfix: carefully handle pod.namespace to avoid side effect by @TrafalgarZZZ in #4172
- Bump github.com/stretchr/testify from 1.8.4 to 1.9.0 by @dependabot in #4164
- feat: support multiround sidecar injection by @TrafalgarZZZ in #4175
- docs: Correcting the usage steps of the CONTABUTING.md document by @Wercurial in #4176
- move dataflow affinity logic to fluidapp controller by @xliuqq in #4138
- Bump github.com/spf13/cobra from 1.8.0 to 1.8.1 by @dependabot in #4166
- minorfix: do not skip mutation for fluid pods by @TrafalgarZZZ in #4184
- Bump github.com/felixge/fgprof from 0.9.3 to 0.9.4 by @dependabot in #4169
- bugfix: fix application controller manage pod lifecycle by @TrafalgarZZZ in #4185
- Bump github.com/go-logr/logr from 1.4.1 to 1.4.2 by @dependabot in #4174
- dataset support pvc storageSize by @wangshulei098 in #4178
- docs: dev guide for en missing GOPATH setup by @backsapce in #4188
- Bump k8s.io/klog/v2 from 2.120.1 to 2.130.1 by @dependabot in #4183
- bugfix: allow users to override juicefs attr-cache and entry-cache options by @TrafalgarZZZ in #4194
- Bump google.golang.org/grpc from 1.64.0 to 1.65.0 by @dependabot in #4191
- Update go.mod to comply with Go 1.21 toolchain syntax by @cheyang in #4196
- bugfix: fix early break when cleaning mount point in NodeUnpublishVolume by @TrafalgarZZZ in #4198
- Bump github.com/agiledragon/gomonkey/v2 from 2.11.0 to 2.12.0 by @dependabot in #4199
- bugfix: fix jindoruntime name cannot contain jindofs by @TrafalgarZZZ in #4202
- revert: "upgrade jindocache to 6.4.0 version (#4126)" by @TrafalgarZZZ in #4205
- Build docker images for reverting jindoruntime from 6.4 to 6.2 by @cheyang in #4206
Full Changelog: v1.0.0...v1.0.1
v1.0.0
Highlights
- Introduced a flexible, configurable, tiered data locality scheduling capability to optimize the affinity between application pods and data cache.
- Provided DataProcess for custom data operations, and add three trigger strategies for data operations for all the data operations – once, onEvent, Cron.
- Provided a Python SDK with dataflow capabilities for data scientists.
- Implemented A new runtime for sharing in-memory immutable data: VineyardRuntime is supported in Fluid.
- Major improvements have been made in:
- Improved security measures by defining stricter, minimum necessary cluster role permissions for Fluid components.
- Significantly optimized FUSE Recovery features to enhance their performance and reliability.
- Made significant updates to the project documentation and website.
We acknowledge all the contributors who put their effort in the version release. Credits to @zwwhdls, @xliuqq, @wangshli, @Hexilee, @weixiao-huang, @abowloflrf, @uniqueni, @TrafalgarZZZ, @dashanji, @cheyang, @fengshunli, @frankleaf, @zhang-x-z, @BillyChen1, @wushiming540, @RongGu and many others for their great contributions to Fluid v1.0.0 !
New Features
Data Operation Features
- Add cron for data migrate.
- Support cron dataload.
- Make toleration configurable in helm charts.
- Data process crd definition.
- Support dataflow.
- Data operations support resources.
- Support ssh password free pod for parallel data migrate.
Vineyard Runtime
- Add the Vineyard runtime CRD definitions.
- Add the helm chart for Vineyard runtime.
- Add the controller and RBAC yaml for Vineyard runtime.
- Implement Vineyard runtime engine and controller.
- Add a replicas field to the Vineyard Runtime and delete the svc suffix in the Vineyard helm chart.
Enhancements & Optimizations
CSI Plugin & FUSE Recovery
- Check path existence and lock in NodeUnpublishVolume.
- umountDuplicate larger than the threshold.
- Fix csi plugin concurrency issue on FuseRecovery and NodeUnpublishVolume.
- Enhancement: recover fuse according to multiple peer group options.
- Support symlink for NodePublish.
Bug Fix
- Bugfix: Support batch/v1beta1 cronjobs for compatibility before Kubernetes v1.21.
- Bugfix: fix csi plugin loop mount bug
- Bugfix: reconcile thinruntime failed when dataset is deleted
- Bugfix:fix thinruntime stuck bug when deleting it
- Bugfix:fix worker&fuse options & worker tiredstore
- Bugfix: del metadata sync && del duplicate metrics
- Bugfix: clean up orphaned thinruntime resources
- Bugfix: fix Alluxio master in HA mode start error
- Bugfix: pass AccessModes to thinruntime fuse container
- Bugfix: fix fatal error: concurrent map writes for runtime controllers
- Bugfix: fix incorrect conversion between integer types
Website & Documentation Update
Full Changelog: v0.9.0...v1.0.0
v0.9.3
What's Changed
- Prepare 0.9.3 by @cheyang in #3638
- enhance: remove jindoruntime's fsGroup (#3632) by @frankleaf in #3641
- Remove fsgroup from alluxio runtime (#3635) by @TrafalgarZZZ in #3654
- cherry-pick to 0.9: del metadata sync && del duplicate metrics (#3380) by @TrafalgarZZZ @Hexilee in #3738
- fix rbac role issue by @BillyChen1 in #3753
- Build docker images for 0.9.3 by @cheyang in #3762
Full Changelog: v0.9.2...v0.9.3
v0.9.2
What's Changed
- Prepare 0.9.2 by @cheyang in #3374
- enhancement for 0.9: remove juicefs fuse readiness probe (#3435) by @zwwhdls in #3436
- Cherry-pick to 0.9: check path existence and lock in NodeUnpublishVolume (#3284) by @TrafalgarZZZ in #3450
- fail fast with wrong kubelet rootdir (#3331) by @wangshli in #3451
- umountDuplicate larger than the threshold (#3429) by @TrafalgarZZZ in #3452
- Cherry-pick to 0.9: fix csi plugin concurrency issue on FuseRecovery and NodeUnpublishVolume (#3448) by @TrafalgarZZZ in #3453
- Fix the problem of using option to append Dataset public option (#3454) by @lgy1027 in #3459
- Build docker images for release 0.9.2 by @cheyang in #3466
Full Changelog: v0.9.1...v0.9.2
v0.8.7
What's Changed
- Update to 0.8.7 by @cheyang in #3223
- Change reinvocationPolicy to IfNeeded by @lizzzcai in #3226
- Cherry-pick bugfix for setting fuse args when using Alluxio 2.8+ by @TrafalgarZZZ in #3231
- fix default master journal type setting (#3281) by @VincentLeeMax in #3291
- Mount kubelet.conf with HostToContainer mount propagation by @cheyang in #3318
- [BUGFIX] Fix unexpected dataset phase after helm install failure (#2751) by @TrafalgarZZZ in #3350
- Use mountinfo to check mountinfo by @zwwhdls in #3351
- Docker build 0.8.7 by @cheyang in #3352
Full Changelog: v0.8.6...v0.8.7
v0.9.1
v0.9.0
v0.9.0
Breaking Changes
- Change matching pod requests mode of webhook from namespaceSelector to objectSelector
Features
- Add thinRuntime to simplify integration with third-party storage systems
- Addon component for Fluid's open source CubeFS, NFS
- Support for accessing data across namespaces
- Support for subDataset
- Native acceleration system EFCRuntime for distributed file systems NFS, GPFS
- Support dataMigrate for data migration operations (currently only supported by JuiceFSRuntime)
- Add customizable configuration for cache cleanup timeout and maximum retry times, Webhook timeout limits
- Add Dataload configuration for ImagePullSecrets, node affinity
- RBAC permission reduction
- Upgrade to golang 1.18
- Support for installing Fluid via Helm Repo
Refactoring
- Use data operation framework to construct data migrate, load, backup behaviors
Bug Fix
- JindoRuntime should support configurable env variables
- Could not set nodeAffinity in dataset
- Runtime helm release stuck in "pending-install" status
- CSI failed to recover FUSE mount point for AlluxioRuntime
- [JuiceFS] FUSE pod scheduled failed because of conflict port
- Fluid csi on rke2 k8s(1.22) use mount output empty,it caused app pod not work
Runtime Upgrade
- AlluxioRuntime is upgrade from v2.8.2 to v2.9.1
- JindoRuntime is upgraded from from 4.5.1 to 4.6.7
- JuicefsRuntime is upgraded from v1.0.0 to v1.0.4
Credits to @yangyuliufeng ,@zwwhdls, @hahchenchen,@frankleaf, @xliuqq, @abowloflrf, @odidev , @allenhaozi, @uniqueni, @yangjun289519474, @wang-mask , @fengshunli , @zhang-x-z , @wangshli @ssz1997 , @myccccccc , @baowj-678 , @BillyChen1 , @Yanghaihai1020, @lizzzcai , @weixiao-huang ,@TrafalgarZZZ and many others for their great contributions to Fluid v0.9.0!
v0.8.6
What's Changed
- Add approvers for release 0.8 by @cheyang in #2864
- Update fluid version to 0.8.6 by @cheyang in #2843
- add reinvocationPolicy by @cheyang in #3047
- Cherry-pick fix worker volumes in alluxio chart (#2330) by @wangshli in #3191
- Fix rbacs and limit CSI Plugin's node related access by @TrafalgarZZZ
Full Changelog: v0.8.5...v0.8.6
v0.8.5
What's Changed
- Update fluid version to 0.8.5 by @cheyang in #2623
- [juicefs] fix worker cache when set option (#2563) @zwwhdls
- fix multi cache dir (#2639) @zwwhdls
- [Enhancement]CSI plugin checks mount point liveness before binding mount points (#2703) @TrafalgarZZZ
- Prettify error messages for exec.Commands in Fluid (#2718) @TrafalgarZZZ
- Cherry pick for 0.8.5 by @cheyang in #2785
Full Changelog: v0.8.4...v0.8.5
v0.8.4
What's Changed
- Origin/init release 0.8.4 by @cheyang in #2616
- Cherry-pick update fuse container as root user in sidecar mode (#2501) by @zwwhdls in #2618
- fix NPE issue of cleaning juicefsruntime (#2505) by @zwwhdls in #2619
- Cherry-pick Juicefs/fix secret key (#2509) by @zwwhdls in #2620
- Cherry-pick Random port allocation by @TrafalgarZZZ in #2621
- Update fluid version to 0.8.4 by @cheyang in #2622