AWS CDK Typescript Project
Budget: £20 – £250 GBP
I am looking for a developer with experience in AWS CDK, typescript, AWS Aurora (MySQL) and S3 buckets.
I am currently writing a website and am currently looking at the database and how I will release it to a development or production environment creating the initial tables and then populating from csv files (1 of the files is 100MB).
The project is to create an AWS CDK project that consists of the following.
- VPC (with at least two subnets - one is private)
- S3 Bucket
- Secret for the database
- Aurora MySQL serverless V2 clustered database
- Lambda function
The database needs to be placed in the private subnet and the lambda in the other subnet. When the project is deployed it must create the infrastructure above.
After the infrastructure above has been created I want to copy the file attached to this project to the S3 bucket.
When I run the lambda I want it to
- Run the create table script below
- Run a LOAD DATA script that loads the data file from the S3 bucket into the table we have just created
- Run a SELECT COUNT(*) FROM geo_countries and console.log the row count.
Create Table
CREATE TABLE IF NOT EXISTS geo_countries (
country_id SMALLINT UNSIGNED NOT NULL,
country_code CHAR(2),
country VARCHAR(50),
lc_country VARCHAR(50),
PRIMARY KEY (country_id)
) ENGINE=InnoDB;
Load Data
Will be something like. It has to be the fast load (LOAD DATA) from MySQL and NOT an insert into table.
LOAD DATA INFILE 's3-bucket/geo_countries.csv'
INTO TABLE geo_countries
FIELDS TERMINATED BY ';'
ENCLOSED BY '"'
LINES TERMINATED BY '\r\n';
Database Cluster (CDK)
Should be something like this this. The database has to be MySQL and I want it to be serverless so that it scales with the load. If you disagree with the below or a database cluster then Im open to a discussion.
const dbCluster = new rds.DatabaseCluster(this, "db-cluster", {
engine: rds.DatabaseClusterEngine.auroraMysql({ version: rds.AuroraMysqlEngineVersion.VER_3_03_0 }),
credentials: rds.Credentials.fromSecret(databaseCredentialsSecret),
clusterIdentifier: "cluster",
defaultDatabaseName: dbName,
serverlessV2MaxCapacity: 4,
serverlessV2MinCapacity: 0.5,
writer: rds.ClusterInstance.serverlessV2(prefix + "-" + props.environment + "-db-writer", {}),
readers: [rds.ClusterInstance.serverlessV2(prefix + "-" + props.environment + "-db-reader", {})],
deletionProtection: false,
removalPolicy: RemovalPolicy.DESTROY,
securityGroups: [props.databaseSG],
s3ImportBuckets: [deployBucket],
s3ExportBuckets: [deployBucket],
// s3ImportRole: deployBucketRole,
vpc: props.vpc,
vpcSubnets: {
subnetType: SubnetType.PRIVATE_ISOLATED,
},
parameterGroup: parameterGroupForInstance,
});
I am currently writing a website and am currently looking at the database and how I will release it to a development or production environment creating the initial tables and then populating from csv files (1 of the files is 100MB).
The project is to create an AWS CDK project that consists of the following.
- VPC (with at least two subnets - one is private)
- S3 Bucket
- Secret for the database
- Aurora MySQL serverless V2 clustered database
- Lambda function
The database needs to be placed in the private subnet and the lambda in the other subnet. When the project is deployed it must create the infrastructure above.
After the infrastructure above has been created I want to copy the file attached to this project to the S3 bucket.
When I run the lambda I want it to
- Run the create table script below
- Run a LOAD DATA script that loads the data file from the S3 bucket into the table we have just created
- Run a SELECT COUNT(*) FROM geo_countries and console.log the row count.
Create Table
CREATE TABLE IF NOT EXISTS geo_countries (
country_id SMALLINT UNSIGNED NOT NULL,
country_code CHAR(2),
country VARCHAR(50),
lc_country VARCHAR(50),
PRIMARY KEY (country_id)
) ENGINE=InnoDB;
Load Data
Will be something like. It has to be the fast load (LOAD DATA) from MySQL and NOT an insert into table.
LOAD DATA INFILE 's3-bucket/geo_countries.csv'
INTO TABLE geo_countries
FIELDS TERMINATED BY ';'
ENCLOSED BY '"'
LINES TERMINATED BY '\r\n';
Database Cluster (CDK)
Should be something like this this. The database has to be MySQL and I want it to be serverless so that it scales with the load. If you disagree with the below or a database cluster then Im open to a discussion.
const dbCluster = new rds.DatabaseCluster(this, "db-cluster", {
engine: rds.DatabaseClusterEngine.auroraMysql({ version: rds.AuroraMysqlEngineVersion.VER_3_03_0 }),
credentials: rds.Credentials.fromSecret(databaseCredentialsSecret),
clusterIdentifier: "cluster",
defaultDatabaseName: dbName,
serverlessV2MaxCapacity: 4,
serverlessV2MinCapacity: 0.5,
writer: rds.ClusterInstance.serverlessV2(prefix + "-" + props.environment + "-db-writer", {}),
readers: [rds.ClusterInstance.serverlessV2(prefix + "-" + props.environment + "-db-reader", {})],
deletionProtection: false,
removalPolicy: RemovalPolicy.DESTROY,
securityGroups: [props.databaseSG],
s3ImportBuckets: [deployBucket],
s3ExportBuckets: [deployBucket],
// s3ImportRole: deployBucketRole,
vpc: props.vpc,
vpcSubnets: {
subnetType: SubnetType.PRIVATE_ISOLATED,
},
parameterGroup: parameterGroupForInstance,
});