Build Generative AI Applications on Amazon Bedrock with AWS SDK for Python (Boto3)

amazon Bedrock is a fully managed service that offers a selection of high-performance foundation models (FM) from leading ai companies such as AI21 Labs, Anthropic, Cohere, Meta, Mistral ai, Stability ai and amazon through a single API, along with a broad set of capabilities to build generative ai applications with security, privacy, and responsible ai. With amazon Bedrock, you can experiment and evaluate leading FMs for your use case, privately customize them with your data using techniques such as fine-tuning and recovery augmented generation (RAG), and create agents that execute tasks using the systems and your company data. sources. Because amazon Bedrock is serverless, you don't need to manage any infrastructure, and you can safely integrate and deploy generative ai capabilities into your applications using the AWS services you're already familiar with.

In this post, we demonstrate how to use amazon Bedrock with AWS SDK for Python (Boto3) to programmatically embed FM.

Solution Overview

The solution uses an AWS SDK script for Python with features that invoke Anthropic's Claude 3 Sonnet on amazon Bedrock. Using this FM, you generate an output using a message as input. The following diagram illustrates the architecture of the solution.

Prerequisites

Before invoking the amazon Bedrock API, make sure you have the following:

Implement the solution

Once you complete the prerequisites, you can start using amazon Bedrock. Get started by creating scripts with the following steps:

1. Import the required libraries:

1. Configure the Boto3 client to use the amazon Bedrock Runtime and specify the AWS region:

# Set up the amazon Bedrock client
bedrock_client = boto3.client(
    	service_name="bedrock-runtime",
    region_name="us-east-1"
)

1. Define the model to invoke using its model ID. In this example, we use Sonnet Claude 3 by Anthropic on amazon Bedrock:

# Define the model ID
model_id = "anthropic.claude-3-sonnet-20240229-v1:0"

1. Assign a message, which is your message that will be used to interact with the FM in the invocation:

# Prepare the input prompt.
prompt = "Hello, how are you?"

Quick engineering techniques can improve FM performance and improve results.

Before invoking the amazon Bedrock model, we must define a payload, which acts as a set of instructions and information that guides the model generation process. This payload structure varies depending on the model chosen. In this example, we use Sonnet Claude 3 by Anthropic on amazon Bedrock. Think of this payload as the blueprint for the model and provide it with the context and parameters necessary to generate the desired text based on your specific message. Let's look at the key elements of this payload:

• anthropic_version – This specifies the exact version of amazon Bedrock you are using.
• tokens_max – This sets a limit on the total number of tokens the model can generate in its response. Tokens are the smallest meaningful text unit (word, punctuation, subword) processed and generated by large language models (LLM).
• temperature – This parameter controls the level of randomness in the generated text. Higher values ​​lead to more creative and potentially unexpected results, and lower values ​​promote more conservative and consistent results.
• top_k – This defines the number of most likely candidate words considered at each step during the generation process.
• up_p – This influences the sampling probability distribution for selecting the next word. Higher values ​​favor frequent words, while lower values ​​allow for more diverse and potentially surprising options.
• messages – This is an array containing individual messages for the model to process.
• role – This defines the role of the sender within the message (the user of the message you provide).
• content – This array contains the text of the actual message, represented as an object of type “text”.

1. Define the payload as follows:

payload = {
    "anthropic_version": "bedrock-2023-05-31",
    "max_tokens": 2048,
    "temperature": 0.9,
    "top_k": 250,
    "top_p": 1,
    "messages": (
        {
            "role": "user",
            "content": (
                {
                    "type": "text",
                    "text": prompt
                }
            )
        }
    )
}

1. You have configured the parameters and the FM you want to interact with. Now send a request to amazon Bedrock providing the FM to interact with and the payload you defined:

# Invoke the amazon Bedrock model
response = bedrock_client.invoke_model(
    modelId=model_id,
    body=json.dumps(payload)
)

1. Once the request is processed, you can display the text output generated from amazon Bedrock:

# Process the response
result = json.loads(response("body").read())
generated_text = "".join((output("text") for output in result("content")))
print(f"Response: {generated_text}")

Let's look at our complete script:

import boto3
import json

# Set up the amazon Bedrock client
bedrock_client = boto3.client(
    service_name="bedrock-runtime",
    region_name="us-east-1"
)

# Define the model ID
model_id = "anthropic.claude-3-sonnet-20240229-v1:0"

# Prepare the input prompt
prompt = "Hello, how are you?"

# Create the request payload
payload = {
    "anthropic_version": "bedrock-2023-05-31",
    "max_tokens": 2048,
    "temperature": 0.9,
    "top_k": 250,
    "top_p": 1,
    "messages": (
        {
            "role": "user",
            "content": (
                {
                    "type": "text",
                    "text": prompt
                }
            )
        }
    )
}

# Invoke the amazon Bedrock model
response = bedrock_client.invoke_model(
    modelId=model_id,
    body=json.dumps(payload)
)

# Process the response
result = json.loads(response("body").read())
generated_text = "".join((output("text") for output in result("content")))
print(f"Response: {generated_text}")

Invoke the model with the message “Hello, how are you?” will return the result shown in the following screenshot.

Clean

When you're done using amazon Bedrock, clean up temporary resources, such as IAM users and amazon CloudWatch logs, to avoid unnecessary charges. Cost considerations depend on the frequency of use, the price of the chosen model, and the resource utilization while the script is running. See amazon Bedrock Pricing for details on pricing and cost optimization strategies, such as selecting appropriate models, optimizing indications, and monitoring usage.

Conclusion

In this post, we demonstrate how to programmatically interact with amazon Bedrock FM using Boto3. We explore the invocation of a specific FM and the processing of the generated text, showing the potential for developers to use these models in their applications for a variety of use cases, such as:

• Text generation – Generate creative content such as poems, scripts, musical pieces or even different programming languages.
• Code Completion – Improve developer productivity by suggesting relevant code snippets based on existing code or prompts
• Data summary – Extract key information and generate concise summaries from large data sets
• Conversational ai – Develop chatbots and virtual assistants that can engage in conversations in natural language.

Stay curious and explore how generative ai can revolutionize various industries. Explore the different models and APIs and make comparisons of how each model provides different results. Find the model that fits your use case and use this script as the basis for creating agents and integrations in your solution.

About the author

Merlin Naidoo is a Senior Technical Account Manager at AWS with over 15 years of experience in digital transformation and innovative technical solutions. Her passion is connecting with people of all backgrounds and leveraging technology to create meaningful opportunities that empower everyone. When he's not immersed in the world of technology, you can find him playing active sports.