Implement personalized recommendations in real time using Amazon Personalize

At a basic level, machine learning (ML) technology learns from data to make predictions. Companies use their data with an ML-based personalization service to improve their customers’ experience. This approach allows businesses to use data to gain actionable insights and help increase their revenue and brand loyalty.

Amazon Personalize accelerates your digital transformation with ML, making it easy to integrate personalized recommendations into existing websites, apps, email marketing systems, and more. Amazon Personalize allows developers to quickly deploy a custom personalization engine, without requiring machine learning experience. Amazon Personalize provides the necessary infrastructure and manages the entire machine learning (ML) process, including processing data, identifying features, using the best algorithms, and training, optimizing, and hosting models. You receive results through an API and pay only for what you use, with no minimum fees or upfront commitments.

The post Designing near-real-time personalized recommendations with Amazon Personalize shows how to design near-real-time personalized recommendations using Amazon Personalize and AWS-specific data services. In this post, we’ll walk you through a reference implementation of a real-time personalized recommender system using Amazon Personalize.

Solution Overview

The real-time personalized recommendations solution is implemented using Amazon Personalize, Amazon Simple Storage Service (Amazon S3), Amazon Kinesis Data Streams, AWS Lambda, and Amazon API Gateway.

The architecture is implemented as follows:

1. Data preparation – Start by creating a group of data sets, schemas, and data sets that represent your elements, interactions, and user data.
2. Train the model – After importing your data, select the recipe that matches your use case and then create a solution to train a model by creating a version of the solution. When your solution version is ready, you can create a campaign for your solution version.
3. Get recommendations in near real time – When you have a campaign, you can integrate campaign calls into your app. This is where calls are made to the GetRecommendations or GetPersonalizedRanking APIs to request near real-time recommendations from Amazon Personalize.

For more information, see Design near-real-time personalized recommendations with Amazon Personalize.

The following diagram illustrates the architecture of the solution.

Implementation

We demonstrate this implementation with a use case on how to make real-time movie recommendations to an end user based on their interactions with the movie database over time.

The solution is implemented through the following steps:

1. Prerequisite (data preparation)
2. Set up your development environment
3. Implement the solution
4. Create a solution version
5. Create a campaign
6. Create an event tracker
7. Get recommendations
8. Ingest real-time interactions
9. Validate recommendations in real time
10. Clean

Previous requirements

Before you begin, make sure you have the following prerequisites:

• Prepare your training data – Prepare and upload data to an S3 bucket using the instructions. For this particular use case, you will load interaction data and item data. An interaction is an event that you record and then import as training data. Amazon Personalize generates recommendations based primarily on interaction data that you import into an Interactions data set. You can record various types of events, such as clicking, watching, or liking. Although the model created by Amazon Personalize can make suggestions based on a user’s past interactions, the quality of these suggestions can be improved when the model has data about associations between users or items. If a user has interacted with movies categorized as Drama in the item’s data set, Amazon Personalize will suggest movies (items) with the same genre.
• Set up your development environment – Install the AWS Command Line Interface (AWS CLI).
• Set up CLI with your Amazon account – Configure the AWS CLI with your AWS account information.
• Install and boot the AWS Cloud Development Kit (AWS CDK)

Implement the solution

To deploy the solution, do the following:

• Clone the repository to a new folder on your desktop.
• Deploy the stack to your AWS environment.

Create a solution version

A solution refers to the combination of an Amazon Personalize recipe, custom parameters, and one or more solution versions (trained models). When you deploy the CDK project in the previous step, a solution with a user customization recipe is automatically created. A solution version refers to a trained machine learning model. Create a solution version for implementation.

Create a campaign

A campaign deploys a version of the solution (trained model) with a provisioned transaction capability to generate real-time recommendations. Create a Campaign for implementation.

Create an event tracker

Amazon Personalize can make recommendations based on real-time event data only, historical event data only, or both. Record real-time events to build your interaction data and let Amazon Personalize learn from your user’s most recent activity. This keeps your data up to date and improves the relevance of Amazon Personalize recommendations. Before you can log events, you must create an event tracker. An event tracker directs new event data to the Interactions dataset in its dataset group. Create and event tracker for implementation.

Get recommendations

In this use case, the interaction data set is made up of movie IDs. Consequently, the recommendations presented to the user will consist of movie identifications that most closely align with their personal preferences, determined from their historical interactions. You can use the getRecommendations API to retrieve personalized recommendations for a user by sending their partner userID, the number of results for the recommendations you need for the user, as well as the campaign ARN. You can find the campaign ARN in the Amazon Personalize console menu.

For example, the following request will retrieve 5 recommendations for the user whose userId is 429:

curl --location 'https://{your-api-id}.execute-api.{your-region}.amazonaws.com/prod/getRecommendations?campaignArn={campaignArn}&userId=429&numResults=5'

The response of the request will be:

{
	"$metadata": {
		"httpStatusCode": 200,
		"requestId": "7159c128-4e16-45a4-9d7e-cf19aa2256e8",
		"attempts": 1,
		"totalRetryDelay": 0
	},
	"itemList": (
	{
		"itemId": "596",
		"score": 0.0243044
	},
	{
		"itemId": "153",
		"score": 0.0151695
	},
	{
		"itemId": "16",
		"score": 0.013694
	},
	{
		"itemId": "261",
		"score": 0.013524
	},
	{
		"itemId": "34",
		"score": 0.0122294
	}
	),
	"recommendationId": "RID-1d-40c1-8d20-dfffbd7b0ac7-CID-06b10f"
}

The items returned by the API call are the movies that Amazon Personalize recommends to the user based on their historical interactions.

Score values ​​given in this context represent floating point numbers ranging from zero to 1.0. These values ​​correspond to the current campaign and associated recipes for this use case. They are determined based on the collective scores assigned to all elements present in your entire data set.

Ingest real-time interactions

In the example above, recommendations were obtained for the user with an ID of 429 based on their historical interactions with the movie database. To get real-time recommendations, user interactions with items must be fed into Amazon Personalize in real time. These interactions are incorporated into the recommendation system through Amazon Personalize Event Tracker. The type of interaction, also called EventTypeis given by the column of the same name in the interaction data set (EVENT_TYPE). In this example, the events can be of type “view” or “click”, but you can have your own event types depending on the needs of your application.

In this example, the exposed API that generates the user events with the items receives the “interactions” parameter that corresponds to the number of events (interactions) of a user (UserId) with a single element (itemId) right now. He trackingId The parameter can be found in the Amazon Personalize console and in the response to the Event Tracker creation request.

This example shows a putEvent request: Generate 1 click interaction, with an item id of ‘185’ for user id ‘429’, using the current timestamp. Note that in production, ‘sentAt’ must be set at the time of user interaction. In the following example, we set this to the time in the epoch format in which we wrote the API request for this post. Events are sent to Amazon Kinesis Data Streams through an API gateway, so you must send the stream-name and PartitionKey parameters.

curl --location 'https://iyxhva3ll6.execute-api.us-west-2.amazonaws.com/prod/data' --header 'Content-Type: application/json' --data '{ "stream-name": "my-stream","Data": {"userId" : "429", "interactions": 1, "itemId": "185", "trackingId" : "c90ac6d7-3d89-4abc-8a70-9b09c295cbcd", "eventType": "click", "sentAt":"1698711110"},"PartitionKey":"userId"}'

You will receive a confirmation response similar to the following:

{
	"Message": "Event sent successfully",
	"data": {
		"EncryptionType": "KMS",
		"SequenceNumber": "49..........1901314",
		"ShardId": "shardId-xxxxxxx"
	}
}

Validate recommendations in real time

Because the interaction data set has been updated, the recommendations will be automatically updated to consider new interactions. To validate the updated recommendations in real time, you can call the getRecommendations API again for the same user ID 429 and the result should be different from before. The following results show a new recommendation with an ID of 594 and recommendations with IDs of 16, 596, 153, and 261 changed their scores. These elements brought new movie genres (‘Animation|Children|Drama|Fantasy|Musical’) the top 5 recommendations.

Order:

curl --location 'https://{your-api-id}.execute-api.{your-region}.amazonaws.com/prod/getRecommendations?campaignArn={campaignArn} &userId=429&numResults=5'

Answer:

{
	"$metadata": {
		"httpStatusCode": 200,
		"requestId": "680f2be8-2e64-47d7-96f7-1c4aa9b9ac9d",
		"attempts": 1,
		"totalRetryDelay": 0
	},
	"itemList": (
	{
		"itemId": "596",
		"score": 0.0288085
	},
	{
		"itemId": "16",
		"score": 0.0134173
	},
	{
		"itemId": "594",
		"score": 0.0129357
	},
	{
		"itemId": "153",
		"score": 0.0129337
	},
	{
		"itemId": "261",
		"score": 0.0123728
	}
	),
	"recommendationId": "RID-dc-44f8-a327-482fb9e54921-CID-06b10f"
}

The response shows that the recommendation provided by Amazon Personalize was updated in real time.

Clean

To avoid unnecessary charges, clean up your solution deployment using Cleaning resources.

Conclusion

In this post, we show you how to implement a real-time personalized recommendation system using Amazon Personalize. Interactions with Amazon Personalize to ingest real-time interactions and get recommendations were executed through a command-line tool called curl, but these API calls can be integrated into an enterprise application and achieve the same result.

To choose a new recipe for your use case, see Real-Time Personalization. To measure the impact of recommendations made by Amazon Personalize, see Measuring the impact of recommendations.

About the authors

Cristian Marquez is a senior cloud application architect. He has extensive experience designing, building and delivering enterprise-grade software, distributed and high-load systems, and cloud-native applications. He has experience in backend and frontend programming languages, as well as system design and implementation of DevOps practices. He actively helps clients build and secure innovative cloud solutions, solving their business problems and achieving their business objectives.

Anand Komandooru is a Senior Cloud Architect at AWS. He joined the AWS Professional Services organization in 2021 and helps customers build cloud-native applications in the AWS Cloud. He has over 20 years of experience building software and his favorite Amazon leadership principle is “The leaders are very right.“