When creating voice-enabled chatbots with amazon Lex, one of the biggest challenges is accurately capturing the user's voice input for slot values. For example, when a user needs to provide their account number or confirmation code, the accuracy of voice recognition becomes crucial. This is where transcription confidence scores come in to help ensure reliable gap filling.
What are transcription confidence scores?
Transcription confidence scores indicate how confident amazon Lex is at converting speech to text for slot values. These scores range from low to high and are separate from the intent/entity recognition scores. For each spoken space value, Lex provides a confidence score that you can use to:
- Validate if the value of a spoken space was understood correctly
- Decide whether to request confirmation or request again
- Branching conversation flows based on confidence in recognition
Here are some ways to leverage trust scores for better slot management:
- Progressive confirmation
- High Confidence (>0.9): Accept the space value and continue
- Medium Confidence (0.6-0.9): Ask the user to confirm (“Did you say 12345?”)
- Low confidence (<0.6): Re-request the space value
- Adaptive Replay
- Customize snooze messages based on trust level
- Provide more specific guidance for low confidence inputs.
- Offer alternative entry methods when necessary
- Branching logic
- Route to human agent if there are multiple low confidence attempts
- Skip commit for consistently high confidence inputs
- Adjust validation rules based on confidence thresholds.
The post includes an AWS CloudFormation template to demonstrate these patterns that you can deploy in your AWS account.
Strategic use of trust scores helps create stronger occupancy experiences that:
- Reduce errors in capturing critical information
- Improve self-service contention rates
- Provide better handling of challenging audio conditions
- Enable smarter conversation flows
By implementing these patterns, you can significantly improve the accuracy and reliability of gap filling in your Lex voice robots while maintaining a natural conversation flow.
Solution Overview
The following diagram illustrates the overall architecture of the solution.
We use an amazon Lex bot integrated with an amazon Connect contact flow to deliver the conversational experience.
amazon Lex asks the caller: “In a nutshell, how can I help you today?” amazon Lex reviews the confidence score of the transcription of slot values to influence the commit routine.
Interaction model
The following are different customer experiences a caller may have based on what they say and the transcription confidence scores returned by amazon Lex:
Virtual Agent:
Thank you for calling Acme Travel.
Virtual agent:
…In a nutshell, what is the reason for your call today?
Customer:
I would like to check my reservation.
Virtual agent:
That's great, please say your 5 character booking reference, you'll find it at the top of the information pack we send you.
Customer:
1 9 AKB (Recognised with low transcription confidence: 0.4)
Virtual Agent:
Okay, let's try it again. What is your reservation reference?
Customer:
1 9 AAB (More clearly pronounced: recognized with 0.8)
Virtual Agent:
Your 1 9 AAB reservation is currently in process.
Virtual agent:
Thank you for calling Acme Travel.
Virtual agent:
…In a nutshell, what is the reason for your call today?
Customer:
I would like to check my reservation 1 9 AZ Y.
Virtual agent:
Your 1 9 AZY reservation is currently in progress.
Virtual agent:
Thank you for calling Acme Travel.
Virtual agent:
…In a nutshell, what is the reason for your call today?
Customer:
I would like to check my reservation.
Virtual agent:
That's great, please say your 5 character booking reference, you'll find it at the top of the information pack we send you.
Customer:
1 9 AZY (recognized with a transcription confidence of 0.6)
Virtual agent:
Did you say 1 9 AZY? Please say yes or no.
Customer:
Yeah
Virtual agent:
Your 1 9 AZY reservation is currently in progress.
In the example conversations, the IVR requests the reservation reference from the customer. Once received, the transcript confidence score is evaluated by enabling conditional branching in amazon Lex based on the speech confidence scores. These conditions compare the value with specific thresholds. If the transcript confidence score exceeds the high threshold (for example, greater than 0.7), the conversation advances to the next state. If the score falls in the medium confidence range (for example, between 0.4 and 0.7), the user is prompted to confirm the interpreted entry. Finally, if the score falls below a minimum threshold (e.g., less than 0.4), the user is prompted to try again and provide the information again. This approach optimizes conversation flow based on the quality of captured input and avoids erroneous or redundant slot capture, leading to an improved user experience while increasing self-service contention rates.
Prerequisites
You must have an AWS account and an AWS Identity and Access Management (IAM) user and role with permissions to create and manage the resources and components required for this application. If you don't have an AWS account, see How do I create and activate a new amazon Web Services account?
Additionally, you need an amazon Connect instance; You will use the amazon Resource Name (ARN) of the instance in a later step.
Deploy the amazon Lex bot and amazon Connect flow
To create the sample bot and configure phrase suggestions at runtime, complete the following steps. For this example, we created an amazon Lex bot called disambiguation-bot, an intent (CheckBooking), and a slot type (BookingRef).
- Sign in to your AWS account, and then choose launch stack To deploy the CloudFormation template:
- For First nameenter a name, for example
contact-center-transcription-confidence-scores. - Choose Next.
- Provide the following parameters:
- For Robot nameenter disambiguation-bot.
- For ConnectInstanceRNAEnter the ARN of your amazon Connect instance.
- For Contact flow nameEnter a name for your amazon Connect contact flow (for example,
lex-check-booking-sample-flow). - For Record group nameEnter the name of the amazon CloudWatch log group where conversation logs are stored.
- Choose Next.
- Leave all remaining settings as default and choose Next.
- Select I recognize that AWS CloudFormation could create IAM resources.
- Choose Deliver.
- Wait for the CloudFormation stack to deploy successfully.
- In the amazon Connect console, assign the contact flow to a claimed amazon Connect number.
Configure transcript confidence scoring logic
After creating your intent (CheckBooking), use the visual conversation builder to configure transcript confidence scoring logic.
The following figure is an example of how we add logic to the intent. Highlighted in red is the branch condition where we use the transcript confidence score to dynamically change the customer experience and improve accuracy.
If you choose node, you will be presented with the following configuration options, which is where you can configure the branch condition.
Try the solution
To test the solution, we examine a conversation with words that may not be clearly understood.
amazon Connect will ask “Thank you for calling Acme Travel. In a nutshell, what is the reason for your call today?”
- Reply “I want to check my reservation.”
- When asked for your booking reference, say two numbers followed by three letters (for example, “1 9 AZY”).
This test checks the trust score and will say “your 1 9 AZY reservation is currently in progress” or ask you to confirm “1 9 AZY”.
Limitations
Audio transcription confidence scores are available in English (GB) only (en_GB) and English (US) (en_US) languages. Confidence scores are only supported for 8 kHz audio inputs. Transcription confidence scores are not provided for the audio input from the amazon Lex V2 console test window because it uses a 16 kHz audio input.
Clean
To delete the infrastructure created by the CloudFormation template, open the AWS CloudFormation console and delete the stack. This will remove the services and configuration installed as part of this deployment process.
Conclusion
Optimizing the user experience is at the top of any amazon Lex conversational designer's priority list, as is capturing information accurately. This new feature allows designers to have options around confirmation routines that drive a more natural dialogue between the customer and the bot. While confirming every entry can slow down the user experience and cause frustration, failing to confirm when transcript confidence is low can compromise accuracy. These improvements allow you to create a more natural and efficient experience.
To learn more about creating effective conversations in amazon Lex with intent confidence scores, see Create more effective conversations in amazon Lex with confidence scores and higher accuracy.
About the authors
Alex Buckhurst is a Senior amazon Connect Consultant at amazon Web Services focusing on innovation and creating customer-centric designs. In his free time, Alex enjoys playing squash, honing his barbecue skills, and cherishing moments with his family.
Kai Loreck is a senior professional services consultant for amazon Connect. Work on the design and implementation of scalable customer experience solutions. In his free time, he can be found playing sports, snowboarding or hiking in the mountains.
Neel Kapadia is a Senior Software Engineer at AWS, where he works on designing and building scalable ai/ML services using large language models and natural language processing. He has been at amazon for over five years and has worked on amazon Lex and amazon Bedrock. In her free time she likes to cook, read and travel.
Anand Jumnani is a DevOps consultant at amazon Web Services based in the UK. Outside of work, he is passionate about club cricket and enjoys spending quality time with family and friends.
