| parent | title | nav_exclude |
|---|---|---|
Infrastructure Catalog |
AWS Data-Lake |
false |
This data lake implementation creates three buckets, one each for data, logging, and metadata. The data lake also supports lambda functions which can trigger automatically when new content is added.
- Designed to be used in combination with the
aws/data-lake-usersmodule. - To add SFTP protocol support, combine this module with the
aws/sftpmodule.
No requirements.
The following providers are used by this module:
-
random
-
aws
The following input variables are required:
Description: Standard name_prefix module input. (Prefix counts towards 64-character max length for certain resource types.)
Type: string
Description: Standard environment module input.
Type:
object({
vpc_id = string
aws_region = string
public_subnets = list(string)
private_subnets = list(string)
})Description: Standard resource_tags module input.
Type: map(string)
The following input variables are optional (have default values):
Description: Optionally, you can override the default data bucket with a bucket that already exists.
Type: string
Default: null
Description: Local path to a folder containing the lambda source code (e.g. 'resources/fn_log')
Type: string
Default: null
Description: List of S3 triggers objects, for example:
[{
function_name = "fn_log"
triggering_path = "*"
lambda_handler = "main.lambda_handler"
environment_vars = {}
environment_secrets = {}
}]
Type:
map(
# function_name as map key
object({
triggering_path = string
lambda_handler = string
environment_vars = map(string)
environment_secrets = map(string)
})
)Default: {}
Description: The path to a local pip executable, used to package python dependencies for any lambda triggers.
Type: string
Default: "pip3"
The following outputs are exported:
Description: Summary of resources created by this module.
Description: The S3 bucket used for data storage.
Description: The S3 bucket used for metadata file storage.
Description: The S3 bucket used for log file storage.
Source code for this module is available using the links below.
NOTE: This documentation was auto-generated using
terraform-docs and s-infra from slalom.dataops.
Please do not attempt to manually update this file.