Gcsfuse v2.5.2

Dependency Upgrades / CVE fixes:
This release is built on top of v2.5.1 with the additional security/CVE fixes for indirect dependency x/crypto library.

Gcsfuse v2.6.0

gRPC:

• A new gRPC client version provides significant read performance improvements at lower CPU utilization.
• Allowlist access is no longer needed to use gRPC. To use gRPC, pass the cli flag --client-protocol=grpc OR add gcs-connection:client-protocol: grpc in the mount-config file.
• Known issues:
  • gRPC: File system write operation hangs while repeatedly writing over a single > 16MiB file - Issue#2784
  • gRPC: File system returns Input/output error while repeatedly writing over a single < 16MiB file - Issue#2783

List Payload Reduction:

• The Cloud Storage ListObjects API now retrieves only essential GCS object attributes required for serving list requests, resulting in significantly less response payloads and reduced latency. PR#2627

Bug Fixes & Improvements:

• All flags are now supported in the persistent mounting via /etc/fstab. PR#2478
• SIGTERM to unmount: Sending SIGTERM signal to GCSFuse process will now unmount the filesystem.
• Enhanced error handling for background processes now captures runtime crash logs. Users can specify a log file path to store these logs (the file will have a .stderr extension).

Gcsfuse v2.5.1

Bug Fixes:
Improved error handling: GCSFuse will now retry requests to hierarchical namespace bucket APIs that encounter deadline exceeded errors. This improves stability and prevents unnecessary failures, especially in high latency environments.

Gcsfuse v2.5.0

• Hierarchical Namespace Enabled Buckets: With this release users can also mount buckets with hierarchical namespace enabled. HNS-enabled buckets offer several advantages over standard buckets when used with cloud storage fuse:
  • Renaming directories is now supported natively with Hierarchical Namespace Buckets, where Rename operation is fast and atomic to benefit workloads such as AI/ML checkpointing.
  • Users don't need to specify the --implicit-dirs command-line option when using hierarchical namespace enabled buckets. HNS buckets inherently understand directories, so gcsfuse does not need to simulate directories using placeholder objects ( 0-byte objects ending with '/' ).
  • Kernel’s Lookup operations are also improved in HNS buckets. The Objects.list api is replaced with a more efficient Folder:get api.
• Config CLI Parity:
  • All params in config-file will be available as CLI flags and vice versa going forward. Documentation links:
    • CLI Flags
    • Config File
  • Deprecated and hidden flags won't show up in the help-doc any more.
• Write Enhancements:
  • Information of newly created files is added to the type cache. This helps reduce an additional GCS list call if the bucket has been mounted with –implicit-dirs flag. (PR#2303)
• Deprecated - The flags --debug_gcs and --debug_fuse flags are now deprecated. Set --log-severity=TRACE to achieve the same behavior.

Gcsfuse v2.4.1

• Bug Fixes & Improvements:
  • Fix for os.RemoveAll(directory) failure with kernel list cache and remote file addition: PR#2163.
  • Fixed an issue in listing during creation of files in the same directory (Issue#2220, PR#2237).
  • Additional performance improvements when using Parallel downloads (PR#2287).
• Metrics:
  • Categorize the various (approximately 150) fs_errors into a handful of categories. This helps reduce the upper bound of cardinality of these error labels which avoids it from getting dropped by the Google Cloud Metrics pipeline (PR#2321, PR#2382, PR#2390).
  • Change unit of ops and file-cache latency metrics to microseconds to improve the resolution of these metrics (PR#2380).
• Dependency Upgrades / CVE fixes:
  • Upgraded dependencies for better stability (PR#2301, PR#2355, PR#2364)
  • Upgrade Golang version to 1.23.0 (PR#2365)

Gcsfuse v2.1.1

This release is built on top of v2.1.0 with the additional security fixes of Golang upgrade to 1.22.4

Gcsfuse v2.4.0

• Parallel download:
  • Accelerates reads of large files, by using the file cache directory as a prefetch buffer using multiple workers to download large files in parallel.
  • This feature is useful for single threaded read scenarios that load large (>1GiB) files, such as model serving use cases and checkpoint restores.
  • This feature is disabled by default. To enable this feature:
    • Enabling the file cache feature (GKE instructions) is a prerequisite for using the parallel download feature, which uses the cache directory as a prefetch buffer. Although a cache is typically associated with repeat reads, with parallel downloads even first reads of large files are accelerated.
      • The file being read must fit within the file cache directory’s available capacity, which can be controlled by max-size-mb. A value of “-1” allows it to use the cache volume’s entire capacity, or you can give it a value in Megabytes.
      • If the same file will be read multiple times, increase the ttl-secs value. A value of "-1" bypasses TTL expiration and serves the file from the cache if it's available.
    • Set file-cache:enable-parallel-downloads:true in the config file to enable parallel downloads. The default is false.
    • Additional optional parameters:
      • file-cache:parallel-downloads-per-file: The number of maximum workers to spawn per file to download the object from GCS into the file-cache. Default is 16.
      • file-cache:max-parallel-downloads: The number of maximum workers that can be spawned at any given time across all the download jobs of files. The default is set to 2x the number of CPU cores on the machine. A value of -1 means no limit.
      • file-cache:download-chunk-size-mb: The size of each read request in MiB that each goroutine makes to GCS when downloading the object into file-cache. Default is 50. A parallel download will only trigger if the file being read is => this value specified
    • Note: If your application does high read parallelism (>8 threads), a slight performance degradation may be observed if using this feature. High read parallelism is typically seen in training workloads so should not be used for training workloads, and is therefore recommended only for model serving and checkpoint restores, which are typically single threaded large file reads.
  • Addresses #1300
• Kernel-List-Cache
  • List responses, that happen as a part of a readdir operation, are cached in the kernel page cache. This can significantly speed up AI/ML training runs, which do full directory listing first, by serving repeat ListObjects calls locally from the kernel page cache. Due to potential coherency/consistency issues, it is recommended to use on read only volumes, specifically for serving and training.
  • This feature is disabled by default. To enable this feature:
    • Control cache invalidation via the --kernel-list-cache-ttl-secs cli flag or file-system:kernel-list-cache-ttl-secs config flag, where a value of:
      • 0 means disabled. This is the default value.
      • valid positive - represents the ttl (in seconds) to keep the directory list response in the kernel page-cache.
      • -1 to bypass a TTL expiration and serve the list response from the cache whenever it's available.
  • Addresses #184
• CLI-flags Take Precedence over Config, behavior change for logging-flags: Going forward command-line flags will always take precedence for all settings. Change in behavior for logging flags (--log-file & --log-format), where config was taking precedence but not cli-flags will take. This will affect only if the same settings are set in both CLI and config. - #2077

Dependency Upgrades / CVE fixes:

• Upgraded dependencies for better stability (PR #2209, #2196, #2195, #2086)

Gcsfuse v2.3.2

This release includes:

• Code changes/fixes to kernel list cache feature.

Full Changelog: v2.3.1...v2.3.2

Gcsfuse v2.3.1

Enhancements:

• Hotfix release to restore backward compatibility for the experimental KernelListCacheTtlSecs that got broken in version v2.3.0
  • Cache invalidation can be controlled via the --kernel-list-cache-ttl-secs cli flag or file-system:kernel-list-cache-ttl-secs config flag

Dependency Upgrades / CVE fixes:

• No Changes here.

What's Changed

• Revert "Move KernelListCacheTtlSecs to List by @kislaykishore in #2083

Full Changelog: v2.3.0...v2.3.1

Gcsfuse v2.3.0

Enhancements:

• Ignore Interrupts default true:
  • --ignore-interrupts flag is now ENABLED by default, more details on this feature added in v2.1.0.
• Experimental kernel list cache config change:
  • There is backward incompatible change in this release for configuration file, prior to this release the experimental kernel-list-cache-ttl-secs could be set using file-system:kernel-list-cache-ttl-secs, for this release the config should be list:kernel-list-cache-ttl-secs. Passing this value through cli remains the same, no change in that.
• anonymous-access config change:
  • There is backward incompatible change in this release for configuration file, prior to this release anonymous access used to access an endpoint that does not require authentication, such as a publicly accessible bucket or test endpoint could be set using auth-config:anonymous-access: true, for this release the config should be gcs-auth:anonymous-access: true. Passing this value through cli remains the same, no change in that.

Dependency Upgrades / CVE fixes:

• Upgraded go lang to 1.22.4.

What's Changed

• Update direct dependencies. by @kislaykishore in #2009
• Add all config-params to params.yaml by @kislaykishore in #1999
• Convert gcloud operations in storage client - 1 by @Tulsishah in #1959
• Add documentation to params.yaml by @kislaykishore in #2010
• Make the spreadsheetId argument optional by @Tulsishah in #2012
• Delete folder api implementation by @Tulsishah in #1954
• fix e2e test for hns bucket by @Tulsishah in #2020
• Validates formatting in CI workflow by @kislaykishore in #2016
• Run all except cache tests concurrently by @kislaykishore in #2022
• Making compute crc method context cancellable by @vadlakondaswetha in #2013
• Use 1.22 as the min Go version by @kislaykishore in #2003
• Set crc32 when converting minobject to object by @vadlakondaswetha in #2028
• Add parallel downloads job by @sethiay in #2005
• Fix copyright URL by @kislaykishore in #2031
• Convert Golang compliant flags to POSIX compliant by @kislaykishore in #2030
• Automate e2e tests on TPC environment by @Tulsishah in #1943
• Make Wait for Download false for all read scenarios when parallel downloads is true by @ankitaluthra1 in #2027
• Kokoro tpc build fix by @Tulsishah in #2033
• Support fstab mount with hyphens by @ashmeenkaur in #1993
• Fix Pytorch v2 model failure by @sethiay in #2037
• Unmarshal the config object by @kislaykishore in #2035
• Implement limit on max-concurrency. by @kislaykishore in #2032
• Add max-download-parallelism validation by @kislaykishore in #2046
• Ignore interrupts by default by @ashmeenkaur in #2034
• [Config CLI Parity] add squash to config structure tags to make decoding easier by @ashmeenkaur in #2044
• [Config-CLI Parity][Fix] Populate config object when config file is not provided by @ashmeenkaur in #2042
• adds get folder method in bucket.go by @ankitaluthra1 in #2041
• Raise error if params.yaml contains invalid fields by @kislaykishore in #2049
• fix gcloud install by @Tulsishah in #2039
• Return error if the config file is invalid. by @kislaykishore in #2048
• Use atleast 1 goroutine for async job by @sethiay in #2056
• Add @kislaykishore as owners for config by @kislaykishore in #2055
• Revert "add squash to config structure tags to make decoding possible… by @ashmeenkaur in #2058
• [Config CLI Parity] temp-dir should be of type resolved path by @ashmeenkaur in #2059
• Add more tests (unit, composite) tests for parallel downloads by @sethiay in #2019
• Remove flakiness in parallel downloads unit tests by @sethiay in #2063
• Add constraints on number of args by @kislaykishore in #2066
• Rename parallel downloads flags by @sethiay in #2064
• Fix race in test by @kislaykishore in #2067
• Use default MaxDownloadParallelism based on number of CPU cores by @sethiay in #2047
• Set default max-parallel-downloads in new config by @kislaykishore in #2072
• Setting defaults for parallel downloads by @sethiay in #2073
• Use truncate and remove function while eviction from job during CRC check by @sethiay in #2015

Full Changelog: v2.2.0...v2.3.0