Automate the process to change the background of images using Amazon Bedrock and AWS Step Functions

Many clients, including those in creative advertising, media and entertainment, e-commerce, and fashion, often need to change the background of a large number of images. Typically, this involves manually editing each image with photography software. This can require a lot of effort, especially for large batches of images. However, Amazon Bedrock and AWS Step Functions make it easy to automate this process at scale.

Amazon Bedrock offers the basic Amazon Titan Image Generator G1 generative ai model, which can automatically change the background of an image using a technique called exterior painting. Step Functions allows you to create an automated workflow that connects seamlessly with Amazon Bedrock and other AWS services. Together, Amazon Bedrock and Step Functions streamline the entire process of automatically changing backgrounds across multiple images.

This post presents a solution that simplifies the process of changing backgrounds across multiple images. By leveraging the capabilities of generative ai with Amazon Bedrock and the Titan Image Generator G1 model, combined with Step Functions, this solution efficiently generates images with the desired background. This post provides insight into the inner workings of the solution and helps you understand the design choices made to create this own custom solution.

Watch the GitHub repository for detailed instructions on how to implement this solution.

Solution Overview

Let's look at how the solution works at a high level before diving into specific elements and the AWS services used. The following diagram provides a simplified view of the solution architecture and highlights key elements.

The workflow consists of the following steps:

1. A user uploads multiple images to an Amazon Simple Storage Service (Amazon S3) bucket through a Streamlit web application.
2. The Streamlit web application calls an Amazon API Gateway REST API endpoint integrated with the Amazon Rekognition DetectLabels API, which detects labels for each image.
3. Upon submission, the Streamlit web application updates an Amazon DynamoDB table with image details.
4. Upgrading DynamoDB triggers an AWS Lambda function, which starts a Step Functions workflow.
5. The Step Functions workflow executes the following steps for each image:
   5.1 Build a request payload for Amazon Bedrock InvokeModel API.
   5.2 Summons the Amazon bedrock InvokeModel API action.
   5.3 Parses an image of the response and saves it to an S3 location.
   5.4 Update the image state in a DynamoDB table.
6. The Step Functions workflow invokes a Lambda function to generate a status report.
7. The workflow sends an email using Amazon Simple Notification Service (Amazon SNS).

As shown in the screenshot below, the Streamlit web application allows you to upload images and enter text messages to specify the desired backgrounds, negative messages, and exterior paint mode for image generation. You can also view and remove unwanted tags associated with each uploaded image that you do not want to retain in the final generated images.

In this example, the message for the background is “London City Background.” The automation process generates new images based on the original images uploaded with London as the background.

Uploading Streamlit images and web applications

A Streamlit web application serves as the interface for this solution. To protect the application from unauthorized access, it integrates with an Amazon Cognito user pool. API Gateway uses an Amazon Cognito authorizer to authenticate requests. The web application completes the following steps:

1. For each selected image, retrieve tags through Amazon Rekognition using an API Gateway REST API endpoint.
2. Upon submission, the application uploads images to an S3 bucket.
3. The application updates a DynamoDB table with relevant parameters, image names, and associated tags for each image using another API Gateway REST API endpoint.

Image processing workflow

When the DynamoDB table is updated, DynamoDB Streams triggers a Lambda function to start a new Step Functions workflow. The following is a sample request for the workflow:

{
  "Id": "621fa85a-38bb-4d98-a656-93bbbcf5477f",
  "S3Bucket": "<Image Bucket>",
  "InputS3Prefix": "image-files/<year>/<month>/<day>/<timestamp>",
  "OutputS3Prefix": "generated-image-files/<year>/<month>/<day>/<timestamp>",
  "StatusS3Prefix": "status-report-files/<year>/<month>/<day>/<timestamp>",
  "Prompt": "london city background",
  "NegativePrompt": "low quality, low resolution",
  "Mode": "PRECISE",
  "Images": (
    {
      "ImageName": "bus.png",
      "Labels": "Bus, Person"
    },
    {
      "ImageName": "cop.png",
      "Labels": "Person, Adult, Male, Man, Helmet, Jacket"
    },
    {
      "ImageName": "iguana-2.png",
      "Labels": "Lizard”
    },
    {
      "ImageName": "dog.png",
      "Labels": "Dog"
    }
  )
}

The Step Functions workflow then performs the following three steps:

1. Replace the background of all images.
2. Generate a status report.
3. Send email via Amazon SNS.

The following screenshot illustrates the Step Functions workflow.

Let's look at each step in more detail.

Replace background of all images

Step Functions uses a distributed map to process each image in parallel child workflows. The distributed map allows for high concurrency processing. Each child workflow has its own execution history independent of that of the parent workflow.

Step Functions uses an optimized InvokeModel API action for Amazon Bedrock. The API accepts requests and responses of up to 25 MB. However, Step Functions has a 256 KB limit on state payload input and output. To support larger images, the solution uses an S3 bucket where InvokeModel API reads data and writes the result. The following is the configuration for the InvokeModel API for Amazon Bedrock integration:

{
    "ModelId": "arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-image-generator-v1",
    "ContentType": "application/json",
    "Input": {  
        "S3Uri": “s3://<Image Bucket>/image-files/<year>/<month>/<day>/<timestamp>/<Image name>.json",
    },  
    "Output": {  
        "S3Uri": “s3://<Image Bucket>/generated-image-files/<year>/<month>/<day>/<timestamp>/<Image name>.json”
    } 
}

He Input S3Uri The parameter specifies the source location to retrieve the input data. He Output S3Uri The parameter specifies the destination to write the API response.

A Lambda function saves the request payload as a JSON file in the specified format. Input S3Uri location. He InvokeModel The API uses this input payload to generate images with the specified background:

{
    "taskType": "OUTPAINTING",
    "outPaintingParams": {
        "text": "london city background",
        "negativeText": "low quality, low resolution",        
        "image": "<base64-encoded string>",                         
        "maskPrompt": "Bus",                      
        "maskImage": "base64-encoded string",                             
        "outPaintingMode": "DEFAULT | PRECISE"                 
    },                                                 
    "imageGenerationConfig": {
        "numberOfImages": 1,
        "quality": "premium",
        "height": 1024,
        "width": 1024,
        "cfgScale": 8.0
    }
}

The Titan Image Generator G1 model supports the following parameters for image generation:

• type of task – Specifies the painting method to replace the image background.
• text – A text message to define the background.
• negative text – A text message to define what not to include in the image.
• quick mask – A text message that defines the mask. Corresponds to tags that you want to preserve in the final generated images.
• maskImage – The JPEG or PNG image encoded in base64.
• outPaintingMode – Specifies whether or not modification of pixels within the mask is allowed. DEFAULT allows you to modify the image within the mask to keep it consistent with the reconstructed background. PRECISE prevents image modification within the mask.
• numberofimages – The number of images to generate.
• quality – The quality of the images generated: standard either premium.
• cfgScale – Specifies how strongly the generated image should adhere to the message.
• height – The height of the image in pixels.
• broad – The width of the image in pixels.

The bedrock of the Amazon InvokeModel API generates a response with an encoded image in the Output S3Uri location. Another Lambda function parses the response image, decodes it from base64, and saves the image file to the following location: s3://<Image Bucket>/generated-image-file/<year>/<month>/<day>/<timestamp>/.

Finally, a secondary workflow updates a DynamoDB table with the imaging state, marking it as was successful either Failedand including details such as ImageName, Cause, Errorand Status.

Generate a status report

After the imaging process, a Lambda function retrieves the state details from DynamoDB. Dynamically compiles these details into a complete status report in JSON format. Then save the generated status report to a JSON file in the following location: s3://<Image Bucket>/status-report-files/<year>/<month>/<day>/<timestamp>/. The ITOps team can integrate this report with their existing notification system to track whether image processing completed successfully. For business users, you can extend this further to generate a report in CSV format.

Send an email via Amazon SNS

Step Functions invoke an Amazon SNS API action to send an email. The email contains details including the S3 location for the status report and final image files. The following is the notification email example.

Conclusion

In this post, we provide an overview of a sample solution that demonstrates automating changing image backgrounds at scale using Amazon Bedrock and Step Functions. We also explain each element of the solution in detail. Using Step Functions' optimized integration with Amazon Bedrock, Distributed Map, and the Titan Image Generator G1 model, the solution efficiently replaces image backgrounds in parallel, improving productivity and scalability.

To implement the solution, see the instructions in the GitHub repository.

Resources

To learn more about Amazon Bedrock, see the following resources:

For more information on the Titan Image Generator G1 model, see the following resources:

For more information about using Amazon Bedrock with Step Functions, see the following resources:

About the Author

Chetan Makvana is a Senior Solutions Architect at Amazon Web Services. She works with AWS partners and customers to provide architectural guidance to create a scalable architecture and implement strategies to drive adoption of AWS services. He is a technology enthusiast and builder with a primary area of ​​interest in generative ai, serverless and DevOps. Outside of work, he enjoys watching shows, traveling and music.