Skip to content

Commit

Permalink
feat(glue): glue tables can include storage parameters (#24498)
Browse files Browse the repository at this point in the history
Includes a `storageParameters` property, allowing developers to access the `tableInput.storageDescriptor.parameters` property within the `CfnTable` resource. 

Closes #23132.

----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
  • Loading branch information
Rizxcviii authored Jul 31, 2023
1 parent 6c588da commit f1df9ab
Show file tree
Hide file tree
Showing 16 changed files with 964 additions and 118 deletions.
Original file line number Diff line number Diff line change
@@ -1 +1 @@
{"version":"32.0.0"}
{"version":"32.0.0"}
20 changes: 19 additions & 1 deletion packages/@aws-cdk/aws-glue-alpha/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -223,7 +223,25 @@ new glue.Table(this, 'MyTable', {
});
```

By default, an S3 bucket will be created to store the table's data and stored in the bucket root. You can also manually pass the `bucket` and `s3Prefix`:
Glue tables can be configured to contain user-defined properties, to describe the physical storage of table data, through the `storageParameters` property:

```ts
declare const myDatabase: glue.Database;
new glue.Table(this, 'MyTable', {
storageParameters: [
glue.StorageParameter.skipHeaderLineCount(1),
glue.StorageParameter.compressionType(glue.CompressionType.GZIP),
glue.StorageParameter.custom('separatorChar', ',')
],
// ...
database: myDatabase,
columns: [{
name: 'col1',
type: glue.Schema.STRING,
}],
dataFormat: glue.DataFormat.JSON,

This comment has been minimized.

Copy link
@markusl

markusl Aug 1, 2023

Contributor

Does skipHeaderLineCount work with JSON data format?

});
```

### Partition Keys

Expand Down
5 changes: 3 additions & 2 deletions packages/@aws-cdk/aws-glue-alpha/lib/index.ts
Original file line number Diff line number Diff line change
@@ -1,12 +1,13 @@
// AWS::Glue CloudFormation Resources:

export * from './code';
export * from './connection';
export * from './data-format';
export * from './data-quality-ruleset';
export * from './database';
export * from './job';
export * from './job-executable';
export * from './code';
export * from './schema';
export * from './security-configuration';
export * from './table';
export * from './storage-parameter';
export * from './table';
Loading

0 comments on commit f1df9ab

Please sign in to comment.