Day 007 – 100DaysAWSIaCDevopsChallenge
Today in my series of the 100 Days of Code challenge, I’m going to create another infrastructure using the Cloud Development Kit (CDK) containing AWS services such as API Gateway, Lambda Functions, Layers, and DynamoDB. The project involves creating an infrastructure that exposes APIs using API Gateway. These APIs will be deployed in a Lambda function, and we will use DynamoDB to persist data. To achieve this, we will follow the steps below:
- Set up the CDK project: Initialize the CDK project and configure it for your preferred programming language.
- Define the API Gateway: Create an API Gateway to handle the HTTP requests and route them to the Lambda function.
- Create Lambda function: Write and deploy Lambda function to handle the business logic for the APIs.
- Add Lambda Layers: Define and add Lambda Layers to share common code among multiple Lambda function.
- Configure DynamoDB: Set up a DynamoDB table to store and retrieve the data needed by the APIs.
- Integrate services: Connect the API Gateway to the Lambda function and configure the Lambda function to interact with DynamoDB.
- Deploy the infrastructure: Deploy the CDK stack to AWS and test the end-to-end functionality of the APIs.
Prerequises
- AWS Cloud Developement Kit (CDK)
- AWS SDK (Javascript version)
- API Gateway, Lambda, IAM and DynamoDB
- Typescript
- esbuild ↗
Infrastructure Diagram
Set up the CDK Project
First we need to install at least Nodejs 18.x↗ or newer and configure an aws profile↗. Additionally, we need to have Docker installed on our host machine, as the NodejsFunction CDK construct requires it to build functions written in TypeScript into JavaScript using esbuild.
npm install -g aws-cdk
mdkir day_007 && cd day_007
cdk init app --language typescript
cdk bootstrap --profile cdk-user
for more details about CDK set up, go to the Docs↗
Configure DynamoDB Table
DynamoDB is a managed NoSQL database service by Amazon Web Services (AWS). We will use it to persist our data because it is very simple to handle and easy to use compared to RDS or Aurora. The table we need will have a composite key (a primary key and a sort key). Since DynamoDB is a NoSQL database, it is not necessary to specify other attributes initially beyond the key attributes; additional table attributes can be added over time as needed.
As mentioned above, our table will have one composite keys split in two key : hash key known as the partition key (primary key) and range key known as sort key(second part of primary key).
| Hash Key | Range Key | Others attributes
|———–|————–|—————————–|
|ID | TodoName | CreatedAt, UpdatedAt, Owner
CDK
Let now create the table using CDK
import { aws_dynamodb as dynamodb } from "aws-cdk-lib"
/*
const props = { ... tableName: "TodoApp"
}
*/ const table = new dynamodb.CfnTable(this, 'DynamoDBTableResource', { tableName: props.tableName, keySchema: [{ keyType: 'HASH', attributeName: 'ID' }, { keyType: 'RANGE', attributeName: 'TodoName' }], attributeDefinitions: [{ attributeName: 'ID', attributeType: dynamodb.AttributeType.STRING }, { attributeName: 'TodoName', attributeType: dynamodb.AttributeType.STRING }], billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, tableClass: dynamodb.TableClass.STANDARD, tags: [{ key: 'Name', value: `DynamoDB-Table-${props.tableName}` }]
})
// day_007/lib/cdk-stack.ts
As you can see, we don’t need to specify the _CreatedAt, _UpdatedAt, and Owner attributes in the attributeDefinitions property.
Create Lambda function
Inside the Lambda function, the code handles creating, updating, and listing the TodoList items in DynamoDB.
The Function Code
Below is the structure of Code:
apps ├── domain │ ├── abstract.domain.ts │ ├── task-status.enum.ts │ ├── task.domain.ts │ └── todo-list.domain.ts ├── index.ts ├── layer │ └── nodejs │ ├── package-lock.json │ └── package.json ├── lib │ └── utils.ts ├── package-lock.json ├── package.json ├── src │ └── infra │ ├── dto │ │ ├── lambda.response.ts │ │ └── page.request.ts │ └── storage │ └── dynamodb │ ├── abstract.repository.ts │ └── todo-list.repository.ts └── tsconfig.json
The files we need here at the moment are index.ts, todo-list.repository.ts, and all files inside the domain directory. I will provide the link to the complete GitHub code at the end of this tutorial.
apps/package.json
{ "main": "index.ts", "type": "commonjs", "dependencies": { "aws-lambda": "^1.0.7", "@aws-sdk/client-dynamodb": "^3.620.0", "@aws-sdk/lib-dynamodb": "^3.620.0", "@aws-lambda/http": "^1.0.1", "uuid": "^10.0.0", "moment": "^2.30.1" }, "devDependencies": { "@types/node": "^22.0.0", "@types/uuid": "^10.0.0", "@types/aws-lambda": "^8.10.142", "@types/moment": "^2.13.0" }
}
apps/domain/*.ts
// apps/domain/abstract.domain.ts
export interface IDomain { id?: string; createdAt?: string; updatedAt?: string;
}
// apps/domain/todo-list.domain.ts
import { IDomain } from './abstract.domain'
export interface TodoList extends IDomain { name: string; owner?: { fullName: string, email: string }, tasks?: any[]
}
apps/src/infra/storage/dynamodb/todo-list.repository.ts
import { AbstractRepository } from './abstract.repository'
import { DeleteCommand, DynamoDBDocumentClient, GetCommand, GetCommandOutput, PutCommand, PutCommandOutput, QueryCommandOutput, ScanCommand, UpdateCommand
} from '@aws-sdk/lib-dynamodb'
import { v4 } from 'uuid'
import moment from 'moment'
import { TodoList } from '../../../../domain/todo-list.domain' export class TodoListRepository implements AbstractRepository<TodoList> { constructor(private client: DynamoDBDocumentClient, private tableName: string) { } async create({ name, owner }: TodoList): Promise<TodoList> { const item = { ID: v4(), CreatedAt: Math.floor((new Date().getTime()) / 1000), UpdatedAt: Math.floor((new Date().getTime()) / 1000), TodoName: name, Owner: { Fullname: owner?.fullName, Email: owner?.email } } let response: PutCommandOutput try { response = await this.client.send(new PutCommand({ TableName: this.tableName, Item: item, ExpressionAttributeNames: { '#nameAttr': 'TodoName' }, ExpressionAttributeValues: { ':nameValue': { 'S': name } }, ConditionExpression: '#nameAttr <> :nameValue' })) } catch (e) { console.error(e) throw new Error('DataStorageException: ' + e.message) } console.log('PutCommandResponse', response) if (response.$metadata.httpStatusCode !== 200) { throw new Error('DataStorage: Something wrong, please try again') } return { id: item.ID, createdAt: this.convertTimestampToDate(item.CreatedAt), updatedAt: this.convertTimestampToDate(item.UpdatedAt), owner, name: item.TodoName } } async list(query: any): Promise<TodoList[]> { let response: QueryCommandOutput try { response = await this.client.send(new ScanCommand({ TableName: this.tableName, Limit: 25, Select: 'ALL_ATTRIBUTES' })) } catch (e) { console.error(e) throw new Error('DataStorageException: ' + e.message) } if (response.$metadata.httpStatusCode === 200 && response.Count! > 0) { return [...response.Items || []].map((value: any) => { return { id: value.ID, name: value.TodoName, createdAt: this.convertTimestampToDate(value.CreatedAt), updatedAt: this.convertTimestampToDate(value.UpdatedAt), owner: { fullName: value.Owner.Fullname, email: value.Owner.Email }, tasks: value.Tasks! } }) } return [] } async update(id: { pk: string, sk?: string }, toUpdate: TodoList): Promise<TodoList> { let response: PutCommandOutput const updateDate = Math.floor((new Date().getTime()) / 1000) try { response = await this.client.send(new UpdateCommand({ TableName: this.tableName, Key: { ID: id.pk, TodoName: id.sk }, UpdateExpression: 'SET #name=:newOrOldName, #owner.#email = :newOrOldEmail, #owner.#fullname = :newOrOldFullname', ExpressionAttributeNames: { '#name': 'TodoName', '#owner': 'Owner', '#fullname': 'FullName', '#Email': 'Email' }, ExpressionAttributeValues: { ':newOrOldName': { 'S': toUpdate.name }, ':newOrOldEmail': { 'S': toUpdate.owner?.email }, ':newOrOldFullname': { 'S': toUpdate.owner?.fullName } } })) } catch (e) { console.error(e) throw new Error('DataStorageException: ' + e.message) } console.debug('PutCommandOutput4Update', response) if (response.$metadata.httpStatusCode !== 200) { throw new Error('DataStorageException: something wrong during update !!') } return { ...toUpdate, id: id.pk, updatedAt: this.convertTimestampToDate(updateDate) } } async get(id: { pk: string, sk?: string }): Promise<TodoList | null> { let response: GetCommandOutput try { response = await this.client.send(new GetCommand({ TableName: this.tableName, Key: { ID: id.pk, TodoName: id.sk } })) } catch (e) { console.error(e) throw new Error('DataStorageException: ' + e.message) } console.debug('GetCommandOutput', response) if (response.$metadata.httpStatusCode !== 200) { throw new Error('DataStorageException: something wrong during update !!') } const value: any = response.Item return !value ? null : { id: value.ID, name: value.TodoName, createdAt: this.convertTimestampToDate(value.CreatedAt), updatedAt: this.convertTimestampToDate(value.UpdatedAt), owner: { fullName: value.Owner.Fullname, email: value.Owner.Email }, tasks: value.Tasks! } } private convertTimestampToDate = (time: number): string => moment.unix(time).format('YYYY-MM-DD HH:mm:ss')
}
apps/index.ts
import { APIGatewayProxyEvent } from 'aws-lambda'
import { isNull } from 'utils'
import { LambdaResponse } from './src/infra/dto/lambda.response'
import { TodoListRepository } from './src/infra/storage/dynamodb/todo-list.repository'
import { DynamoDBDocumentClient } from '@aws-sdk/lib-dynamodb'
import { DynamoDBClient } from '@aws-sdk/client-dynamodb'
import { AbstractRepository } from './src/infra/storage/dynamodb/abstract.repository'
import { TodoList } from './domain/todo-list.domain' const tableName = <string>process.env.TABLE_NAME
const region = process.env.REGION ?? 'us-east-1' const dynamodbClient = DynamoDBDocumentClient.from(new DynamoDBClient({ region: region
})) export const handler = async (event: APIGatewayProxyEvent, context: any) => { const method = event.httpMethod const path = event.path console.log('incoming request', { ...event }) if (isNull(method) && isNull(path)) { return { statusCode: 500, isBase64Encoded: false, body: JSON.stringify({ message: 'Something wrong !!' }) } } const repo: AbstractRepository<TodoList> = new TodoListRepository(dynamodbClient, tableName) const requestBody = (() => { if (event.isBase64Encoded) { return /* global atob */ atob(event.body!) } return typeof event.body === 'string' ? JSON.parse(event.body) : event.body })() let response: LambdaResponse if (method === 'POST' && path.endsWith('/create-todolist')) { // const existing = await repo.get?.() const result = await repo.create({ name: requestBody.name, owner: { fullName: 'Kevin L.', email: '[email protected]' } }) response = { isBase64Encoded: false, statusCode: 200, body: JSON.stringify(result, null, -2) } } else if (path === '/todo-app-api/todolists') { const todos = await repo.list('') response = { isBase64Encoded: false, statusCode: 200, body: JSON.stringify(todos, null, -2) } } else if (new RegExp('^/todo-app-api/todolists/[a-z0-9-]{36}/update-todolist$', 'gi').test(path)) { const old = await repo.get?.({ pk: event.pathParameters?.todoListId!, sk: requestBody.name }) if (!old) { response = { isBase64Encoded: false, statusCode: 404, body: JSON.stringify({ message: `The TodoList ${event.pathParameters?.todoListId!} doesn't exists` }, null, -2) } } else { const updated = await repo.update?.({ pk: old.id!, sk: old.name }, { name: requestBody.name ?? old.name, owner: { fullName: requestBody.owner?.fullName ?? old.owner?.fullName, email: requestBody.owner?.email ?? old.owner?.email } }) response = { isBase64Encoded: false, statusCode: 200, body: JSON.stringify(updated, null, -2) } } } else { return { isBase64Encoded: false, statusCode: 403, body: JSON.stringify({ message: `The API ${method} ${path} is no yet implemented in lambda side` }, null, -2) } } return { ...response, headers: { 'Content-Type': 'application/json', 'Access-Control-Allow-Origin': '*' } }
}
Add Lambda Layer
All the dependencies (node_modules) are added in the Lambda Layer. The reason I chose to add dependencies inside the Layer is that the Layer:
- Centralizes the dependencies
- Reduces the function code size by simplifying the Lambda code
- Allows for simplified update management
If we come back to the code structure, you can see there is file name apps/lib/utils.ts
import moment from 'moment'
export const DATE_FORMAT = 'YYYY-MM-DD'
export const TIME_FORMAT = 'HH:mm:ss'
export const DATETIME_FORMAT = `${DATE_FORMAT} ${TIME_FORMAT}`
export const isNull = (value: any) => value === null
export const nowDate = () => moment().format(DATETIME_FORMAT)
We need to build this utility as the library for our Lambda.
cd apps/layer/nodejs npm install # install deps based apps/layer/nodejs/package.json
# and then build the utils library using esbuild
esbuild --bundle --platform=node --sourcemap ../../lib/utils.ts --outdir=./node_modules "--external:moment"
Let now configure the Lambda Layer cdk resource
import { aws_lambda as lambda, RemovalPolicy, Stack, StackProps } from 'aws-cdk-lib'
import { Construct } from 'constructs' interface LayerStackProps extends StackProps { layerName: string
}
export class LayerStack extends Stack { public readonly layerArn: string; constructor(scope: Construct, id: string, props?: LayerStackProps) { super(scope, id, props) const layer = new lambda.LayerVersion(this, 'LayerResource', { code: lambda.Code.fromAsset('./apps/layer'), compatibleArchitectures: [lambda.Architecture.X86_64], compatibleRuntimes: [lambda.Runtime.NODEJS_20_X, lambda.Runtime.NODEJS_18_X], layerVersionName: props?.layerName, removalPolicy: RemovalPolicy.DESTROY }); this.layerArn = layer.layerVersionArn; this.exportValue(layer.layerVersionArn, { name: 'layerArn', description: 'The Lambda Layer Version ARN Value which will be used by others stack as need' }); }
} // lib/layer-stack.ts
Configure Lambda function
import * as cdk from 'aws-cdk-lib'
import {aws_lambda as lambda, Duration} from 'aws-cdk-lib'
import { NodejsFunction, SourceMapMode } from 'aws-cdk-lib/aws-lambda-nodejs'
import { dependencies } from '../apps/package.json'
....
private createLambdaFunction(props: CustomStackProps, dynamodbTableArn: string) { const lambdaRole = this.createLambdaRole(); const layerArn = cdk.Fn.importValue('layerArn') return new NodejsFunction(this, 'LambdaResource', { entry: './apps/index.ts', handler: 'index.handler', timeout: Duration.seconds(10), functionName: 'Todo-App-NodejsFunction', environment: { TABLE_NAME: props.tableName }, runtime: lambda.Runtime.NODEJS_20_X, memorySize: 128, role: lambdaRole, bundling: { externalModules: [ ...Object.keys(dependencies), 'utils' ], sourceMap: true, sourceMapMode: SourceMapMode.BOTH }, layers: [ lambda.LayerVersion.fromLayerVersionArn(this, 'UtilsAndNodeModulesResource', layerArn) ] }) }
Important to know:
-
entry: points to the file that contains the handler function, here ./apps/index.ts. That file will be converted into a bundle file named index.js, located after the build stage at<rootProject>/cdk.out/asset.<hash>/index.ts -
bundling: maps the configuration of esbuild. In our case, we need esbuild to generate the source map file and considerutilsanddependencies(list) as external mobules. Remember that these external mobules is backed by our Layer, so we don’t need to bundled them in the function code. -
layers: adds the Layer previously created.
API Gateway
| Method | Path | Payload |
|---|---|---|
GET | /todo-app-api/todolists | |
POST | /todo-app-api/todolists/create-todolist | {name: 'string'} |
PUT | /todo-app-api/todolists/{todoListId}/update-todolist | {name: 'string', owner: {fullName: "string", email: "string"}} |
We will create the above endpoints.
Lest start by creating the Rest Api Resource
const restApi = new api.CfnRestApi(this, 'ApiGatewayResource', { name: 'ApiGateway', apiKeySourceType: api.ApiKeySourceType.HEADER, disableExecuteApiEndpoint: false, endpointConfiguration: { types: [api.EndpointType.REGIONAL], tags: [{ key: 'Name', value: 'Api Gateway' }] }
})
// lib/cdk-stack.ts
We need to allow API Gateway to invoke Lambda function.
new lambda.CfnPermission(this, 'LambdaPermissionResource', { sourceAccount: props.env?.account, action: 'lambda:InvokeFunction', principal: 'apigateway.amazonaws.com', sourceArn: `arn:${Aws.PARTITION}:execute-api:${props.env?.region}:${props.env?.account}:${restApi.attrRestApiId}/*/*/*`, functionName: lambdaFunction.functionArn
})
// lib/cdk-stack.ts
Alternativelly, it is also possible to attach the above permission directly to the Lambda function previously created. To do so, we just need to call addPermission(..) method provided by the NodejsFunction class.
lambdaFunction.addPermission("LambdaPermission", { action: 'lambda:InvokeFunction', principal: new iam.ServicePrincipal("apigateway.amazonaws.com"), sourceAccount: props.env?.account, sourceArn: `arn:${Aws.PARTITION}:execute-api:${props.env?.region}:${props.env?.account}:${restApi.attrRestApiId}/*/*/*`
})
// lib/cdk-stack.ts
List TodoList method
const rootResource = new api.CfnResource(this, 'RestApiRootResource', { restApiId: restApi.attrRestApiId, parentId: restApi.attrRootResourceId, pathPart: 'todo-app-api'
}) const getTodoListsResource = new api.CfnResource(this, 'RestApiGetTodoListsResource', { restApiId: restApi.attrRestApiId, parentId: rootResource.attrResourceId, pathPart: 'todolists'
}) const getTodoListsMethod = new ApiMethod(this, 'ApiTodoListsMethodResource', { methodType: MediaType.GET, authType: api.AuthorizationType.NONE, restApiId: restApi.attrRestApiId, resourceId: getTodoListsResource.attrResourceId, operationName: 'GetAllTodoLists', integration: { connection: api.ConnectionType.INTERNET, type: api.IntegrationType.AWS_PROXY, httpMethod: MediaType.POST, uri: `arn:aws:apigateway:${props.env?.region}:lambda:path/2015-03-31/functions/${lambdaFunction.functionArn}/invocations` }
}).resource
// lib/cdk-stack.ts
Create TodoList method
const createTodoListResource = new api.CfnResource(this, 'RestApiCreateTodoListResource', { restApiId: restApi.attrRestApiId, parentId: getTodoListsResource.attrResourceId, pathPart: 'create-todolist'
})
const createTodoListMethod = new ApiMethod(this, 'ApiCreateTodoListResource', { methodType: MediaType.POST, authType: api.AuthorizationType.NONE, restApiId: restApi.attrRestApiId, resourceId: createTodoListResource.attrResourceId, operationName: 'CreateTodoList', integration: { connection: api.ConnectionType.INTERNET, type: api.IntegrationType.AWS_PROXY, httpMethod: MediaType.POST, uri: `arn:aws:apigateway:${props.env?.region}:lambda:path/2015-03-31/functions/${lambdaFunction.functionArn}/invocations` }
}).resource
// lib/cdk-stack.ts
You can notice that the parent resource of createTodoListMethod is getTodoListResource, because the path of createTodoListMethod starts with the path of getTodoListResource.
Update TodoList method
const pathVariableSegmentResource = new api.CfnResource(this, 'RestApiUpdateTodoListPathVariableResource', { restApiId: restApi.attrRestApiId, parentId: getTodoListsResource.attrResourceId, pathPart: '{todoListId}'
}) const updateTodoListLastSegmentResource = new api.CfnResource( this, 'RestApiUpdateTodoListResource', { restApiId: restApi.attrRestApiId, parentId: pathVariableSegmentResource.attrResourceId, pathPart: 'update-todolist'
})
const updateTodoListMethod = new ApiMethod(this, 'ApiUpdateTodoListResource', { methodType: MediaType.PUT, authType: api.AuthorizationType.NONE, restApiId: restApi.attrRestApiId, resourceId: updateTodoListResource.attrResourceId, operationName: 'UpdateTodoList', integration: { connection: api.ConnectionType.INTERNET, type: api.IntegrationType.AWS_PROXY, httpMethod: MediaType.POST, uri: `arn:aws:apigateway:${props.env?.region}:lambda:path/2015-03-31/functions/${lambdaFunction.functionArn}/invocations` }, requestParams: { paths: ['todoListId'] }
}).resource
// lib/cdk-stack.ts
And the ApiMethod custom resource:
import { Construct } from 'constructs'
import { aws_apigateway as api, StackProps } from 'aws-cdk-lib' export enum MediaType { GET = 'GET', POST = 'POST', PUT = 'PUT', DELETE = 'DELETE', OPTIONS = 'OPTIONS'
} interface CustomMethodProps extends StackProps { restApiId: string; resourceId: string; methodType: MediaType; integration: { uri: string; connection: api.ConnectionType; httpMethod: MediaType type: api.IntegrationType }; authType?: api.AuthorizationType, operationName?: string requestParams?: { queries?: string[] paths?: string[] headers?: string[] }
} export class ApiMethod extends Construct { public readonly resource: api.CfnMethod constructor(scope: Construct, id: string, props: CustomMethodProps) { super(scope, id) const params = {} as Record<string, boolean> const integrationReqParams = {} as Record<string, string> const integrationResParams = {} as Record<string, string> if (props.requestParams?.paths) { props.requestParams.paths.forEach(value => { const p = `method.request.path.${value}` params[p] = true integrationReqParams[`integration.request.path.${value}`] = p }) } if (props.requestParams?.queries) { props.requestParams.queries.forEach(value => { params[`method.request.queryString.${value}`] = true integrationReqParams[`integration.request.queryString.${value}`] = `method.request.queryString.${value}` }) } if (props.requestParams?.headers) { props.requestParams.headers.forEach(value => { params[`method.request.header.${value}`] = true integrationReqParams[`integration.request.header.${value}`] = `method.request.header.${value}` }) } this.resource = new api.CfnMethod(this, 'ApiTodoListsMethodResource', { apiKeyRequired: false, restApiId: props.restApiId, resourceId: props.resourceId, httpMethod: props.methodType, operationName: props.operationName || new Crypto().randomUUID(), authorizationType: props.authType || api.AuthorizationType.NONE, integration: { connectionType: props.integration.connection, integrationHttpMethod: props.integration.httpMethod, type: props.integration.type, uri: props.integration.uri, integrationResponses: [{ statusCode: '200', responseParameters: { 'method.response.header.Access-Control-Allow-Headers': '\'Content-Type,X-Amz-Date,Authorization,X-Api-key,X-Amz-Security-Token\'', 'method.response.header.Access-Control-Allow-Methods': '\'GET,OPTIONS,POST,PUT\'', 'method.response.header.Access-Control-Allow-Origin': '\'*\'' } }, { statusCode: '500', responseParameters: integrationResParams }], requestParameters: integrationReqParams }, methodResponses: [{ statusCode: '200', responseParameters: { 'method.response.header.Access-Control-Allow-Headers': true, 'method.response.header.Access-Control-Allow-Methods': true, 'method.response.header.Access-Control-Allow-Origin': true } }], requestParameters: params }) }
}
🥳✨woohaah!!!
We have reached the end of the article.
Thank you so much 🙂
Your can find the full source code on GitHub Repo
Discover more from Coursity
Subscribe to get the latest posts sent to your email.