Update docs

airbyte-integrations/connectors/destination-s3/src/main/java/io/airbyte/integrations/destination/s3/S3ConsumerFactory.java

@@ -100,13 +100,13 @@ private OnStartFunction onStartFunction(final BlobStorageOperations storageOpera
         if (writeConfig.getSyncMode().equals(DestinationSyncMode.OVERWRITE)) {
           final String namespace = writeConfig.getNamespace();
           final String stream = writeConfig.getStreamName();
-          final String bucketPath = writeConfig.getOutputBucketPath();
-          LOGGER.info("Clearing storage area in destination started for namespace {} stream {} bucketObject {}", namespace, stream, bucketPath);
+          final String outputBucketPath = writeConfig.getOutputBucketPath();
+          LOGGER.info("Clearing storage area in destination started for namespace {} stream {} bucketObject {}", namespace, stream, outputBucketPath);
           AirbyteSentry.executeWithTracing("PrepareStreamStorage",
-              () -> storageOperations.dropBucketObject(bucketPath),
-              Map.of("namespace", Objects.requireNonNullElse(namespace, "null"), "stream", stream, "storage", bucketPath));
+              () -> storageOperations.dropBucketObject(outputBucketPath),
+              Map.of("namespace", Objects.requireNonNullElse(namespace, "null"), "stream", stream, "storage", outputBucketPath));
           LOGGER.info("Clearing storage area in destination completed for namespace {} stream {} bucketObject {}", namespace, stream,
-              bucketPath);
+              outputBucketPath);
         }
       }
       LOGGER.info("Preparing storage area in destination completed.");

airbyte-integrations/connectors/destination-s3/src/main/resources/spec.json

@@ -41,7 +41,7 @@
         "description": "Format string on how data will be organized inside the S3 bucket directory",
         "type": "string",
         "examples": [
-          "${NAMESPACE}/${STREAM_NAME}/${YEAR}_${MONTH}_${DAY}_${EPOCH}_${PART_ID}"
+          "${NAMESPACE}/${STREAM_NAME}/${YEAR}_${MONTH}_${DAY}_${EPOCH}_"
         ],
         "order": 3
       },

docs/integrations/destinations/s3.md

@@ -22,15 +22,15 @@ Check out common troubleshooting issues for the S3 destination connector on our
 | S3 Endpoint | string | URL to S3, If using AWS S3 just leave blank. |
 | S3 Bucket Name | string | Name of the bucket to sync data into. |
 | S3 Bucket Path | string | Subdirectory under the above bucket to sync the data into. |
-| S3 Bucket Format | string | Additional string format under S3 Bucket Path. Default value is `${NAMESPACE}/${STREAM_NAME}/${YEAR}_${MONTH}_${DAY}_${EPOCH}_${PART_ID}`. |
+| S3 Bucket Format | string | Additional string format on how to store data under S3 Bucket Path. Default value is `${NAMESPACE}/${STREAM_NAME}/${YEAR}_${MONTH}_${DAY}_${EPOCH}_`. |
 | S3 Region | string | See [here](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-regions-availability-zones.html#concepts-available-regions) for all region codes. |
 | Access Key ID | string | AWS/Minio credential. |
 | Secret Access Key | string | AWS/Minio credential. |
 | Format | object | Format specific configuration. See the [spec](/airbyte-integrations/connectors/destination-s3/src/main/resources/spec.json) for details. |
 
 ⚠️ Please note that under "Full Refresh Sync" mode, data in the configured bucket and path will be wiped out before each sync. We recommend you to provision a dedicated S3 resource for this sync to prevent unexpected data deletion from misconfiguration. ⚠️
 
-The full path of the output data with S3 path format `${NAMESPACE}/${STREAM_NAME}/${YEAR}_${MONTH}_${DAY}_${EPOCH}_${PART_ID}` is:
+The full path of the output data with S3 path format `${NAMESPACE}/${STREAM_NAME}/${YEAR}_${MONTH}_${DAY}_${EPOCH}_` is:
 
 ```text
 <bucket-name>/<source-namespace-if-exists>/<stream-name>/<upload-date>_<epoch>_<partition-id>.<format-extension>
@@ -50,6 +50,22 @@ testing_bucket/data_output_path/public/users/2021_01_01_1234567890_0.csv.gz
 |              bucket path
 bucket name
 ```
+Available variable for custom s3 path format are:
+- `${NAMESPACE}`: Namespace where the stream comes from or configured by the connectionn namespace fields.
+- `${STREAM_NAME}`: Name of the stream
+- `${YEAR}`: Year in which the sync was writing the output data in.
+- `${MONTH}`: Month in which the sync was writing the output data in.
+- `${DAY}`: Day in which the sync was writing the output data in.
+- `${HOUR}`: Hour in which the sync was writing the output data in.
+- `${MINUTE}` : Minute in which the sync was writing the output data in.
+- `${SECOND}`: Second in which the sync was writing the output data in.
+- `${MILLISECOND}`: Millisecond in which the sync was writing the output data in.
+- `${EPOCH}`: Milliseconds since Epoch in which the sync was writing the output data in.
+- `${UUID}`: random uuid string
+
+Note: 
+- Multiple `/` characters in the S3 path are collapsed into a single `/` character. 
+- If the output bucket contains too many files, the part id variable is using a `UUID` instead. It uses sequential ID otherwise.
 
 Please note that the stream name may contain a prefix, if it is configured on the connection.
 A data sync may create multiple files as the output files can be partitioned by size (targeting a size of 200MB compressed or lower) .