Build a Chatbot with Amazon Bedrock: Automate API Calls Using Powertools and CDK

Bedrock and LLMs are the cool kids in town. I decided to figure out how easy it is to build a chatbot using Bedrock agents’ capability to trigger an API based on OpenAPI schema with the new Bedrock support that Powertools for AWS offers.

This post will teach you how to use Amazon Bedrock agents to automate Lambda function-based API calls using Powertools for AWS and CDK for Python. We will also discuss the limitations and gotchas of this solution with Bedrock agents.

Disclaimer: I’m not an AI guy, so if you want to learn more about how it works under the hood, this is not your blog. I will show you practical code for setting up Bedrock agents with your APIs in 5 minutes.

This blog post was originally published on my website, “Ran The Builder.”

Table of Contents

1. Bedrock Introduction

2. Bedrock Agents

3. Powertools for AWS Bedrock Support

4. What are We Building

5. Infrastructure

6. Lambda Function Handler — Powertools for Bedrock

7. Generating OpenAPI File

8. Bedrock Agent in Action

9. Limitations and Gotchas

10. Summary

Bedrock Introduction

Amazon Bedrock is a fully managed service that offers a choice of high-performing foundation models (FMs) from leading AI companies — https://aws.amazon.com/bedrock/

Bedrock introduces a bold claim:

The easiest way to build and scale generative AI applications with foundation models

However, the fact that I could build such an application within an hour or so speaks volumes about this claim. I did, however, have help; I used Powertool for AWS Lambda’s excellent documentation and new support for Bedrock agents.

The way I see it, Bedrock offers a wide range of APIs and Agents that empower you to interact with third-party LLMs and utilize them for any purpose, depending on the LLM’s expertise — whether it’s general purpose helper, writing music, creating pictures out of text or calling APIs on your behalf.

Bedrock doesn’t require any particular infrastructure for you to deploy (VPCs etc.) or manage. It is a fully managed service, and you pay you pay only for what you use, but it can get expensive. The pricing model is quite complex and varies greatly depending on the models you use and the features you select.

Highly sought-after features like guardrails (clean language filters, personal identifiable information scanners, etc.) add to the cost, but again, they are fully managed.

Bedrock Agents

Agents enable generative AI applications to execute multistep tasks across company systems and data sources

Agents are your friendly chatbots that can run multi-step tasks. In our case, they will call APIs according to user input.

Bedrock agents have several components:

1. Model — the LLM you select for the agent to use.

2. Instructions — the initial prompt that sets the context for the agent’s session. This is a classic prompt engineering practice: ‘you are a sales agent, selling X for customers’.

3. Actions groups — You define the agent’s actions for the user. You provide an OpenAPI schema and a Lambda function that implements that OpenAPI schema.

4. Knowledge bases — Optional. The agent queries the knowledge base for extra context to augment response generation and input into steps of the orchestration process.

If you want to learn how they work, check out AWS docs.

Powertools for AWS Bedrock Support

Agents for Bedrock, or just “agents,” understand the free text input, find the correct API to trigger, build the payload according to the OpenAPI, and learn whether the call was successful.

At first, I didn’t realize that Bedrock expects your APIs to change.

Usually, I serve my APIs with API Gateway, which triggers my Lambda functions. The event sent to the function has API Gateway metadata and information, and the body, comes as a JSON-encoded string.

With agents, they don’t interact with an API Gateway URL. They interact with a Lambda function (or more than one), each providing a different OpenAPI file. Agents invoke the functions directly, send a different input than API GW, and expect different response than the regular API Gateway response.

Powertools abstract these differences. I was able to take a Lambda function that worked behind an API Gateway, use Powertools’ event handler for API Gateway, and change the event handler type to Bedrock handler, and it just worked with the agents. Amazing!

Below, you can see the flow of events:

1. Agents use LLM and user input to understand what API (Lambda function) to invoke using the OpenAPI file that describes the API.

2. Powertools handles the Bedrock agent input parsing, validation, and routes to the correct inner function. Each inner function handles a different API route, thus creating a monolith Lambda.

3. Your custom business logic runs and returns a response that adheres to the OpenAPI schema.

4. Powertools returns a Bedrock format response that contains the response from section 3.

This brings me to problem number one — you can’t use the Lambda function with Bedrock agents and an API Gateway. You need to choose only one.

This is a major problem. It means I need to duplicate my APIs — one for Bedrock and another for regular customers. The inputs and responses are just too different. It’s really a shame that Bedrock didn’t extend the API Gateway model with add Bedrock agents context and headers.

If you want to see a TypeScript variation without Powertools, then I highly suggest you check out Lee Gilmore’s post.

What are We Building

We will build a Bedrock agent that will represent a seller. We will ask the agent to purchase orders on our behalf. We will use my AWS Lambda Handler Cookbook template open-source project that represents an order service. You can place orders by calling the POST ‘/api/orders’ API with a JSON payload of customer name and item counts. Orders are saved to a DynamoDB table by a Lambda function.

The Cookbook template was recently featured in an AWS article.

I altered the template and replaced the API Gateway with Bedrock agents. We will build the following parts:

1. Agents with CDK infrastructure as code

2. Generate OpenAPI schema file

3. Lambda function handler’s code to support Bedrock

All code examples can be found at the bedrock branch.

Infrastructure

We will start with the CDK code to deploy the Lambda function and the agent.

You can also use SAM according to Powertool’s documentation.

First, add the ‘cdklabs’ constructs to your poetry file.

Let’s review the Bedrock construct.

This construct is 90% the one that was shown on the Powertool’s excellent documentation:

In lines 18–24, we create the Bedrock agent.

In line 21, we select the model we wish to use.

In line 22, we supply the prompt engineering instruction.

In line 23, we prepare the agent to be used and tested immediately after deployment.

In lines 26–38, we prepare the action group and connect the Lambda function. We get input for the OpenAPI file. The OpenAPI file in this example resides in the ‘docs/swagger/openapi.json’ file.

Lambda Function Handler — Powertools for Bedrock

Let’s review the code for the Lambda function that implements the API of the orders service.

In line 17, we initiate the Bedrock Powertools event handler. Input validation is enabled by default.

In lines 27–43, we define our POST /API/orders API. This metadata helps Powertools generate an OpenAPI file for us (see next section). It defines the API description, input schema, HTTP responses, and their schemas.

In lines 59–62, we define the function’s entry point. According to the input path and HTTP command (POST), it will route the Bedrock agent request to the correct inner function. In this example, there is just one function in line 20.

In lines 49–53, we handle the input (validated automatically by Powertools!) and pass it to the inner logic handler to create the order and save it in the database. This is a hexagonal architectural implementation. You can learn more about it here.

In line 56, we return a Pydantic response object, and Powertools handles the Bedrock response format for us.

Find the complete code here.

Generating OpenAPI File

Powertools for AWS Lambda provides a way to generate the required OpenAPI file from the code.

Let’s see the simplified version below:

You can run this code and then move the output file to the folder you assign in the CDK construct at line 28.

Bedrock Agent in Action

After we deploy our service, we can enter the Bedrock console and see our new agent waiting for us:

Let’s test it out and chat with it in the console:

Success! It understood that we wanted to place an order; it built the payload input, executed the Lambda function successfully, and even displayed its output.

Let’s see what input it sent to the function (I printed it off the Lambda logs)

As you can see, it’s very different from the API Gateway schema. Line 8 contains all sorts of metadata about the agent origin, the API path, the payload, lines 9–18, and the HTTP command, which is shown in line 20.

Let’s verify the functionality of the function and the accuracy of the agents by examining the order that was successfully saved to the DynamoDB table.

As you can see, the order id matches the agent’s response and the input parameters.

Limitations and Gotchas

The Powertools’ documentation and code examples were spot on. It worked out of the box.

Powertools did an excellent job. However, I’ve had several issues with Bedrock agents:

1. Bedrock agents currently support OpenAPI schema 3.0.0 but not 3.1.0. Pydantic 2, which I use to define my models, generates the latest version. I had to change the number to 3.0.0 manually and hope that my API does not use any special 3.1.0 features. Luckily, it didn’t, but it was tough to find the error as it was raised during CloudFormation deployment (‘OpenAPI file rejected) and didn’t explain why my schema file was unsuitable. Powertool’s excellent support over their discord channel helped me. Thanks, Leandro!

2. Your Lambda needs to be monolithic and contain all the routes of your OpenAPI. An alternative would be to create multiple action groups with multiple OpenAPI files, which is doable but does not scale with a large number of routes and APIs.

3. This one’s a major issue — You can’t use the Lambda function with agents and an API Gateway. You need to choose. This means I need to duplicate my APIs — one for Bedrock, another for regular customers. The inputs and responses are just too different. It’s really a shame that Bedrock didn’t extend the API Gateway model with add Bedrock agents context and headers.

4. My agent sent an incorrect payload type. It marked the payload as an integer but kept sending the JSON object as a string instead of a number. My API has strict validation, so it didn’t convert the string to a number and failed the request. I had to debug the matter, which was not as easy as I’d hoped. Your mileage may vary with different LLMs; I chose the “simplest” one.

Summary

Chatting with an “agent” that resulted in a DynamoDB entry being created is quite amazing. Even more amazing is the fact that I was able to get this working so fast. Managed services with CDK support are the way to go forward!

I hope Bedrock makes changes according to my feedback and improves the user experience. The current implementation does not allow me to use it in production APIs without duplicating a lot of code.