Support creating fio test data using gcsfuse rather than gcloud

[gargnitingoogle]

Description

Adds the following parameters in the fio/data-loading helm chart.

• createUsingGcsfuse - If set true, instead of gcloud, a gcsfuse mount
  is used to copy fop testing data from gke pod to bucket
• gcsfuse.mountOptions - Useful only when createUsingGcsfuse is set.
  This is used to define the mount-optiosn for gcsfuse mount for copying
  data.
• preprod - Sets gcloud configuration to preprod for any gcloud runs.
  Useful when createUsingGcsfuse is false or not set.

This is needed for uploading data to zonal buckets right now which are not writeable using gcloud cli. To use this way of creating fio test data, do the following.

# 1. set up - install helm if not already installed

which helm || ( curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 && chmod 700 get_helm.sh && ./get_helm.sh )

# 2. set up - activate a gke cluster to use

gcloud container clusters get-credentials ${cluster_name} --project=${project_id} --location=${zone}
kubectl config current-context
kubectl config set-context --current --namespace=default

# 3 - create fio input data for one or more file-sizes using installing helm charts as follows:

cd ${gcsfuse}/perfmetrics/scripts/testing_on_gke/examples/fio
helm install fio-${fileSize}-data-loader data-loader --set bucketName=${bucketName} --set fio.fileSize=${fileSize} --set fio.blockSize=${blockSize} --set fio.filesPerThread=${filesPerThread} --set fio.numThreads=${numThreads} --set createUsingGcsfuse=true --set gcsfuse.mountOptions=<e.g. "implicit-dirs\,metadata-cache:ttl-secs:-1\,metadata-cache:type-cache-max-size-mb:-1\,metadata-cache:stat-cache-max-size-mb:-1\,client-protocol=grpc\,max-retry-attempts=5\,log-severity=trace">

# 4 - monitor the deployment and run

helm ls
kubectl get pods

# 5 - clean-up

helm ls | tail -n +2 | tr -s ' ' | tr -s '\t' ' ' | cut -d' ' -f1 | while read chart ; do helm uninstall $chart ; done
kubectl get pods | tail -n +2 | tr -s ' ' | tr -s '\t' ' ' | cut -d' ' -f1 | while read pod ; do kubectl delete pods/${pod}; done

Link to the issue in case of a bug fix.

NA

Testing details

1. Manual - NA
2. Unit tests - NA
3. Integration tests - NA

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 75.81%. Comparing base (ff4417a) to head (e1da780).
Report is 2 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #2836      +/-   ##
==========================================
- Coverage   75.94%   75.81%   -0.14%     
==========================================
  Files         118      118              
  Lines       16320    16375      +55     
==========================================
+ Hits        12394    12414      +20     
- Misses       3409     3435      +26     
- Partials      517      526       +9     

Flag	Coverage Δ
unittests	75.81% <ø> (-0.14%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.