The last thing we will discuss is the process of deploying each of the components in AWS. The data pipeline, backend, and frontend are each contained within their own CloudFormation stacks (collections of AWS resources). Allowing them to be deployed in isolation in this way ensures that the entire application is not redeployed unnecessarily during development. I use AWS SAM (Serverless Application Model) to implement the infrastructure for each component as code, leveraging the SAM template specification and the CLI:
- The SAM Template Specification: A shorthand syntax that serves as an extension to AWS CloudFormation for defining and configuring collections of AWS resources, how they should interact, and the required permissions.
- SAM CLI: A command-line tool used, among other things, to create and deploy resources as defined in a SAM template. Handles packaging of application code and dependencies, converting the SAM template to CloudFormation syntax and deploying templates as individual stacks in CloudFormation.
Instead of including the full templates (resource definitions) for each component, I'll highlight specific areas of interest for each service that we've discussed throughout the post.
Pass sensitive environment variables to AWS resources:
The entire application relies heavily on external components such as Youtube Data API, OpenAI API, and Pinecone API. Although it is possible to hard-code these values in CloudFormation templates and pass them as 'parameters', a safer method is to create secrets for each in AWS SecretsManager and reference these secrets in the template like this:
Parameters:
YoutubeDataAPIKey:
Type: String
Default: '{{resolve:secretsmanager:youtube-data-api-key:SecretString:youtube-data-api-key}}'
PineconeAPIKey:
Type: String
Default: '{{resolve:secretsmanager:pinecone-api-key:SecretString:pinecone-api-key}}'
OpenaiAPIKey:
Type: String
Default: '{{resolve:secretsmanager:openai-api-key:SecretString:openai-api-key}}'
Definition of a Lambda function:
These serverless code units form the backbone of the data pipeline and serve as the entry point to the web application backend. To implement them using SAM, it is as simple as defining the path to the code that the function should execute when invoked, along with the necessary permissions and environment variables. Below is an example of one of the functions used in the data pipeline:
FetchLatestVideoIDsFunction:
Type: AWS::Serverless::Function
Properties:
CodeUri: ../code_uri/.
Handler: chatytt.youtube_data.lambda_handlers.fetch_latest_video_ids.lambda_handler
Policies:
- AmazonS3FullAccess
Environment:
Variables:
PLAYLIST_NAME:
Ref: PlaylistName
YOUTUBE_DATA_API_KEY:
Ref: YoutubeDataAPIKey
Retrieving the data pipeline definition in the amazon states language:
To use Step Functions as an orchestrator for individual Lambda functions in the data pipeline, we must define the order in which each one should be executed, as well as settings such as the maximum number of retries in the amazon state language. A simple way to do this is by using the amazon.com/step-functions/latest/dg/workflow-studio.html” rel=”noopener ugc nofollow” target=”_blank”>Workflow study in the Step Functions console to schematically create the workflow and then take the automatically generated ASL definition of the workflow as a starting point that can be modified appropriately. This can then be linked in the CloudFormation template instead of defining it in-place:
EmbeddingRetrieverStateMachine:
Type: AWS::Serverless::StateMachine
Properties:
DefinitionUri: statemachine/embedding_retriever.asl.json
DefinitionSubstitutions:
FetchLatestVideoIDsFunctionArn: !GetAtt FetchLatestVideoIDsFunction.Arn
FetchLatestVideoTranscriptsArn: !GetAtt FetchLatestVideoTranscripts.Arn
FetchLatestTranscriptEmbeddingsArn: !GetAtt FetchLatestTranscriptEmbeddings.Arn
Events:
WeeklySchedule:
Type: Schedule
Properties:
Description: Schedule to run the workflow once per week on a Monday.
Enabled: true
Schedule: cron(0 3 ? * 1 *)
Policies:
- LambdaInvokePolicy:
FunctionName: !Ref FetchLatestVideoIDsFunction
- LambdaInvokePolicy:
FunctionName: !Ref FetchLatestVideoTranscripts
- LambdaInvokePolicy:
FunctionName: !Ref FetchLatestTranscriptEmbeddings
See here for the ASL definition used for the data pipeline discussed in this post.
API resource definition:
Since the API for the web application will be hosted separately from the front-end, we need to enable CORS (Cross-Origin Resource Sharing) support when defining the API resource:
ChatYTTApi:
Type: AWS::Serverless::Api
Properties:
StageName: Prod
Cors:
AllowMethods: "'*'"
AllowHeaders: "'*'"
AllowOrigin: "'*'"
This will allow the two resources to communicate freely with each other. The various endpoints that can be accessed through a Lambda function can be defined as follows:
ChatResponseFunction:
Type: AWS::Serverless::Function
Properties:
Runtime: python3.9
Timeout: 120
CodeUri: ../code_uri/.
Handler: server.lambda_handler.lambda_handler
Policies:
- AmazonDynamoDBFullAccess
MemorySize: 512
Architectures:
- x86_64
Environment:
Variables:
PINECONE_API_KEY:
Ref: PineconeAPIKey
OPENAI_API_KEY:
Ref: OpenaiAPIKey
Events:
GetQueryResponse:
Type: Api
Properties:
RestApiId: !Ref ChatYTTApi
Path: /get-query-response/
Method: post
GetChatHistory:
Type: Api
Properties:
RestApiId: !Ref ChatYTTApi
Path: /get-chat-history/
Method: get
UpdateChatHistory:
Type: Api
Properties:
RestApiId: !Ref ChatYTTApi
Path: /save-chat-history/
Method: put
React App Resource Definition:
AWS Amplify can build and deploy applications using a reference to the relevant Github repository and an appropriate access token:
AmplifyApp:
Type: AWS::Amplify::App
Properties:
Name: amplify-chatytt-client
Repository:
AccessToken: '{{resolve:secretsmanager:github-token:SecretString:github-token}}'
IAMServiceRole: !GetAtt AmplifyRole.Arn
EnvironmentVariables:
- Name: ENDPOINT
Value: !ImportValue 'chatytt-api-ChatYTTAPIURL'
Once the repository is accessible, Ampify will look for a configuration file with instructions on how to build and deploy the app:
version: 1
frontend:
phases:
preBuild:
commands:
- cd client
- npm ci
build:
commands:
- echo "VITE_ENDPOINT=$ENDPOINT" >> .env
- npm run build
artifacts:
baseDirectory: ./client/dist
files:
- "**/*"
cache:
paths:
- node_modules/**/*
As an added benefit, it is also possible to automate the continuous deployment process by defining a branch resource that will be monitored and used to redeploy the application automatically on future commits:
AmplifyBranch:
Type: AWS::Amplify::Branch
Properties:
BranchName: main
AppId: !GetAtt AmplifyApp.AppId
EnableAutoBuild: true
Once the deployment is complete in this way, anyone who has the link available from the AWS Amplify console will be able to access it. A recorded demo of the application can be found accessed this way. here:
