Unlock cost-effective AI inference using Amazon Bedrock's serverless capabilities with a trained model in Amazon SageMaker

In this post, I'll show you how to use amazon Bedrock, with its fully managed on-demand API, with your trained or tuned amazon SageMaker model.

amazon Bedrock is a fully managed service that offers a selection of high-performance foundation models (FM) from leading ai companies such as AI21 Labs, Anthropic, Cohere, Meta, Mistral ai, Stability ai and amazon through a single API, along with a broad set of capabilities to build generative ai applications with security, privacy, and responsible ai.

Previously, if you wanted to use your own custom, tuned models in amazon Bedrock, you had to self-manage your inference infrastructure in SageMaker or train the models directly within amazon Bedrock, which requires expensive provisioned performance.

With amazon Bedrock Custom Model Import, you can use new or existing models that have been trained or tuned within SageMaker using amazon SageMaker JumpStart. You can import supported architectures into amazon Bedrock, allowing you to access them on demand through amazon Bedrock's fully managed invocation model API.

Solution Overview

At the time of writing, amazon Bedrock supports importing custom models from the following architectures:

• Mistral
• Custard
• Meta Llama 2 and Llama 3

For this post, we used a Hugging Face Flan-T5 base model.

In the following sections, I'll walk you through the steps to train a model in SageMaker JumpStart and import it into amazon Bedrock. You will then be able to interact with your custom model through amazon Bedrock playgrounds.

Prerequisites

Before you begin, verify that you have an AWS account with access to amazon SageMaker Studio and amazon Bedrock.

If you don't already have a SageMaker Studio instance, see Start amazon SageMaker Studio for instructions on creating one.

Train a model in SageMaker JumpStart

Complete the following steps to train a Flan model in SageMaker JumpStart:

1. Open the AWS Management Console and go to SageMaker Studio.

<img class="alignnone wp-image-96820 size-full" style="margin: 10px 0px 10px 0px;border: 1px solid #CCCCCC" src="https://technicalterrence.com/wp-content/uploads/2025/01/Unlock-cost-effective-AI-inference-using-Amazon-Bedrock39s-serverless-capabilities-with.png" alt="amazon SageMaker Console” width=”1428″ height=”309″/>

1. In SageMaker Studio, choose Begin in the navigation panel.

With SageMaker JumpStart, machine learning (ML) professionals can choose from a wide selection of publicly available FMs using pre-built machine learning solutions that can be deployed with just a few clicks.

1. Search and choose the Hugging Face Flan-T5 Base

<img loading="lazy" class="alignnone wp-image-96821 size-full" style="margin: 10px 0px 10px 0px;border: 1px solid #CCCCCC" src="https://technicalterrence.com/wp-content/uploads/2025/01/1736427724_598_Unlock-cost-effective-AI-inference-using-Amazon-Bedrock39s-serverless-capabilities-with.png" alt="amazon SageMaker Home Page” width=”624″ height=”378″/>

On the model details page, you can review a brief description of the model, how to implement it, how to tune it, and what format your training data should be in to customize the model.

1. Choose Train to begin fitting the model on your training data.

Create the training job using the default settings. The default values ​​complete the training job with the recommended settings.

1. The example in this post uses a previously populated example data set. When using your own data, enter your location in the Data section, making sure it meets the formatting requirements.

1. Configure security settings, such as the AWS Identity and Access Management (IAM) role, virtual private cloud (VPC), and encryption.
2. Note the value of Output artifact location (S3 URI) to use later.
3. Submit the job to start training.

You can monitor your work by selecting Training in it Jobs dropdown menu. When the status of the training job is displayed as Finishedthe work is finished. With default settings, the workout takes about 10 minutes.

Import the model to amazon Bedrock

Once your model has completed training, you can import it to amazon Bedrock. Complete the following steps:

1. In the amazon Bedrock console, choose Imported models low Foundation models in the navigation panel.
2. Choose Import model.

<img loading="lazy" class="alignnone wp-image-96825 size-full" style="margin: 10px 0px 10px 0px;border: 1px solid #CCCCCC" src="https://technicalterrence.com/wp-content/uploads/2025/01/1736427726_767_Unlock-cost-effective-AI-inference-using-Amazon-Bedrock39s-serverless-capabilities-with.png" alt="amazon Bedrock – Importing Custom Models” width=”624″ height=”199″/>

1. For Model nameEnter a recognizable name for your model.
2. Low Model import settingsselect amazon SageMaker Model and select the radio button next to your model.

<img loading="lazy" class="alignnone wp-image-96826 size-full" style="margin: 10px 0px 10px 0px;border: 1px solid #CCCCCC" src="https://technicalterrence.com/wp-content/uploads/2025/01/1736427726_529_Unlock-cost-effective-AI-inference-using-Amazon-Bedrock39s-serverless-capabilities-with.png" alt="Import a model from amazon SageMaker” width=”624″ height=”464″/>

1. Low Service accessselect Create and use a new service role and enter a name for the role.
2. Choose Import model.

1. The model import will complete in about 15 minutes.

1. Low Playgrounds In the navigation pane, choose Text.
2. Choose Select model.

<img loading="lazy" class="alignnone wp-image-96829 size-full" style="margin: 10px 0px 10px 0px;border: 1px solid #CCCCCC" src="https://technicalterrence.com/wp-content/uploads/2025/01/1736427728_869_Unlock-cost-effective-AI-inference-using-Amazon-Bedrock39s-serverless-capabilities-with.png" alt="Using the model in the amazon Bedrock text playground” width=”624″ height=”279″/>

1. For Categorychoose Imported models.
2. For Modelchoose flan-t5-tuned.
3. For Performancechoose on demand.
4. Choose Apply.

You can now interact with your custom model. In the following screenshot, we use our example custom model to summarize a description about amazon Bedrock.

Clean

Complete the following steps to clean up your resources:

1. If you are no longer using SageMaker, delete your domain from SageMaker.
2. If you no longer want to maintain your model artifacts, delete the amazon Simple Storage Service (amazon S3) bucket where your model artifacts are stored.
3. To delete your imported amazon Bedrock model, in the Imported models page in the amazon Bedrock console, select your model and then choose the options menu (three dots) and select Delete.

Conclusion

In this post, we explore how the Custom Model Import feature in amazon Bedrock allows you to use your own trained or tuned custom models for cost-effective, on-demand inference. By integrating SageMaker's model training capabilities with amazon Bedrock's fully managed, scalable infrastructure, you now have a seamless way to deploy your specialized models and make them accessible through a simple API.

Whether you prefer the easy-to-use SageMaker Studio console or the flexibility of SageMaker laptops, you can train and import your models to amazon Bedrock. This allows you to focus on developing innovative applications and solutions, without the burden of managing a complex machine learning infrastructure.

As the capabilities of large language models continue to evolve, the ability to integrate custom models into your applications becomes increasingly valuable. With amazon Bedrock's Custom Model Import feature, you can now unlock the full potential of your specialized models and deliver personalized experiences to your customers, while benefiting from the scalability and cost-effectiveness of a fully managed service.

For a deeper dive into fine-tuning SageMaker, see Fine-Tuning Instructions for FLAN T5 XL with amazon SageMaker Jumpstart. For more hands-on experience with amazon Bedrock, check out our <a target="_blank" href="https://catalog.workshops.aws/building-with-amazon-bedrock/en-US” target=”_blank” rel=”noopener”>Building with amazon Bedrock workshop.

About the author

Jose Sadler is a Senior Solutions Architect on the AWS Global Public Sector team specializing in cybersecurity and machine learning. With experience in the public and private sector, he has expertise in cloud security, artificial intelligence, threat detection and incident response. His diverse experience helps him design robust, secure solutions that use cutting-edge technologies to safeguard mission-critical systems.