Skip to content

Commit

Permalink
docs(aws): 📝 Add a aws glue example
Browse files Browse the repository at this point in the history
  • Loading branch information
chrisguest75 committed Jan 12, 2024
1 parent 35c5e0b commit 541e102
Showing 1 changed file with 49 additions and 0 deletions.
49 changes: 49 additions & 0 deletions 33_awscli/GLUE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
# GLUE

Create a database source.

## Glue

```sh
export AWS_PROFILE=myprofile
export AWS_REGION=eu-west-1

# create a bucket
aws s3 mb s3://my-athena-data

# create the db
aws glue create-database --database-input '{
"Name": "athenadata",
"Description": "mydatabase"
}'

# copy the data
aws s3 cp "files.tsv" s3://my-athena-data/mydata/files.tsv

# create the table
aws glue create-table --database-name athenadata --table-input '{
"Name": "files",
"StorageDescriptor": {
"Columns": [
{"Name": "path", "Type": "string"}
],
"Location": "s3://my-athena-data/mydata",
"InputFormat": "org.apache.hadoop.mapred.TextInputFormat",
"OutputFormat": "org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat",
"SerdeInfo": {
"SerializationLibrary": "org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe",
"Parameters": {
"field.delim": "\t",
"serialization.format": "\t"
}
}
},
"TableType": "EXTERNAL_TABLE",
"Parameters": {
"classification": "csv",
"delimiter": "\t"
}
}'
```

## Resources

0 comments on commit 541e102

Please sign in to comment.