MDAA datawarehouse module
npm install @aws-mdaa/datawarehouseThe Data Warehouse CDK application is used to configure and deploy resources required for a secure Redshift-based Data Warehouse on AWS.
*
Warehouse KMS Key - Will be used to encrypt all warehouse data at rest (Warehouse bucket, Redshift Cluster).
Warehouse Bucket - An S3 Bucket which can be used for warehouse utility/maintenance operations.
Warehouse Logging Bucket - If 'enableAuditLoggingToS3' specified in config, an S3 Bucket specific to Redshift user activity logging will be created.
* Note - This bucket will be configured with SSE-S3 encryption due to Redshift logging requirements (KMS not supported).
Execution Roles - List of externally managed execution roles required to be associated to the Redshift Cluster. Access to these roles may be granted to Redshift users in order to interact with other AWS services through Redshift queries and commands.
Warehouse Security Group - Will control who can connect to the cluster according to the app config (CIDR ranges, other Security Groups).
* All egress permitted by default
* No ingress (to cluster) permitted by default
Warehouse Subnet Group - Controls which subnets the cluster will be deployed on.
Warehouse Parameter Group - Contains cluster config parameters required to control cluster behaviour and ensure secure operation.
* Enforces use of SSL on client connections
Warehouse Cluster - A Redshift cluster conforming to the specified configuration and security controls.
* All data encrypted at rest using warehouse KMS key
* SSL enforced on all client connections
* Network access controlled by security group.
Warehouse Cluster Scheduled Actions - Scheduled actions to automatically pause and resume Redshift cluster.
Warehouse Federation Roles - Roles which are used via IAM SAML Identity Providers to federate access to the cluster
* Establishes assume role trust (with SAML) with IAM Identity Provider
* Grants ability to dynamically generate cluster user and credentials, and join groups provided in the SAML claim by the identity provider
* Groups must pre-exist in cluster, otherwise federation will fail
Warehouse Users - Generates Redshift user credentials inside of cluster and stores them in a Secret
* Automated secret rotation can be triggered on an configurable cycle (by days)
*
Add the following snippet to your mdaa.yaml under the modules: section of a domain/env in order to use this module:
``yaml`
datawarehouse: # Module Name can be customized
module_path: "@aws-caef/datawarehouse" # Must match module NPM package name
module_configs:
- ./datawarehouse.yaml # Filename/path can be customized
`yamlSpecify the admin username to be created on the cluster. A secret will
be automatically generated containing the admin password.
adminUsername: admin
#Used to configure SAML federations
federations:
- federationName: "test" # Should be descriptive and unique
# This is the arn of the IAM Identity Provider
providerArn: arn:{{partition}}:iam::{{account}}:saml-provider/sample-saml-identity-provider
#A preferred maintenance window day/time range. Should be specified as a range ddd:hh24:mi-ddd:hh24:mi (24H Clock UTC).
#Example: 'Sun:23:45-Mon:00:15'
preferredMaintenanceWindow: Sun:23:45-Mon:00:15
databaseUsers:
- userName: "serviceuserGlue"
dbName: "default_db"
secretRotationDays: 90
secretAccessRoles:
- name: "test-arn"
- userName: "serviceuserQuicksight"
dbName: "default_db"
secretRotationDays: 90
- name: resume-cluster
# Resume cluster every Monday at 7am ET starting April 13, 2022 until Dec 31, 2099
enable: True
# Target Action must be either of: "pauseCluster" or "resumeCluster". resizeCluster is not supported yet.
targetAction: resumeCluster
# Specify the action schedule in cron format cron(Minutes Hours Day-of-month Month Day-of-week Year).
schedule: cron(0 12 ? MON )
# Start Date and Time in UTC format when the schedule becomes active. This must be a future date-time.
startTime: "2023-12-31T00:00:00Z"
# End Date and Time in UTC format after which the schedule is no longer active. This must be a future date-time later than start date.
endTime: "2099-12-31T00:00:00Z"