Optimize reasoning models such as Deepseek with immediate optimization at Amazon Bedrock

Deepseek-R1 Models, Now Available on amazon Bedrock Marketplace, amazon SageMaker Jumstart, As Well As a Serverless Model on amazon Bedrock, Were Recently Popularized by Their Long and Elaborate Thinking Style, WHICH, ACCORDING TO DEEPSEEK'S PUBLISHED Impressive Performance on Highly Challenging Math Benchmarks Like Aime-2024 and Math-500, as well as competitive performance compared to The State-Of-The-Art Models Like Anthropic's Claude Sonnet 3.5, GPT 4o and OpenAi O1 (more details in this paper).

During the training, researchers showed how Deepseek-R1-Zero naturally learns to solve tasks with more time of thought, leading to an impulse in performance. However, what is often ignored is the amount of thought tokens required at the time of inference, and the time and cost of generating these tokens before answering the original question.

In this publication, we demonstrate how to optimize reasoning models such as Deepseek-R1 using immediate optimization at amazon Bedrock.

Long reasoning chains and challenges with maximum token limits

Let's try a direct question about Deepseek-R1:

For the given mathematical problem: the Nate dog can dig six holes a day. Cava for 14 days while Nate is on vacation. When Nate arrives home, 9 holes begins to fill up, but the dog continues to dig 6 new holes every night. How many weeks do you have to complete all the holes? Write the steps you would take to solve it.

In the Patio de Chat/Text of amazon Bedrock, you can continue choosing the new Deepseek-R1 model, as shown in the next screenshot.

You can see that sometimes, depending on the question, the reasoning models do not finish thinking about the general budget of Token Máximo.

The increase in the output token budget allows the model to think for longer. With the maximum increased tokens from 2,048 to 4.096, you must see the reasoning of the model for a while before printing the final response.

The appendix at the end of this publication provides the full answer. You can also collapse the reasoning steps to see only the final response.

As we can see in the case with the budget of 2,048 token, the thought process did not end. This not only cost us 2,048 tokens of time and money, but we did not get the final answer! This observation of high tokens to think generally leads to some monitoring questions, such as:

Is it possible to reduce thought sheets and still get a correct answer?
Can thought be restricted to a maximum number of thought tokens or a thought budget?
In a high level, should intensive models such as Deepseek be used in real -time applications?

In this publication, we show you how you can optimize thought models such as Deepseek-R1 using rapid optimization at amazon Bedrock, resulting in more succinct thought traces without sacrificing precision.

Optimize Deepseek-R1 indications

To start with rapid optimization, select Deepseek-R1 in the model's recreation courtyard at amazon Bedrock, enter your warning and choose the magic wand icon, or use the amazon rock bed optimize_prompt() API. You can also use the optimization immediately in the console, add variables if necessary, establish your model in Deepseek-R1 parameters and model, and click “optimize”:

To demonstrate how rapid optimization at amazon Bedrock can help with reasoning models, we first need a challenging data set. The last humanity exam (HLE), a reference point for extremely challenging questions of dozens of thematic areas, is designed to be the “final” closed reference point of the wide academic capabilities. HLE is multimodal, with questions that are only text or accompanied by an image reference, and includes multiple and exact combat questions for the verification of automated answers. The questions require a deep knowledge of the domain in several verticals; They are unequivocal and resistant to simple internet searches or the recovery of the database. For the context, several avant -garde models (including thought models) work badly at the reference point (see the table of results in this Complete paper).

Let's look at an example question of this data set:

In an alternate universe where the mass of the electron was 1% heavier and the charges of the
electron and proton were both 1% smaller, but all other fundamental constants stayed the same, 
approximately how would the speed of sound in diamond change? 

Answer Choices:
A. Decrease by 2%
B. Decrease by 1.5%
C. Decrease by 1%
D. Decrease by 0.5%
E. Stay approximately the same 
F. Increase by 0.5%
G. Increase by 1%
H. Increase by 1.5%
I. Increase by 2%

The question requires a deep understanding of physics, that most large language models (LLM) will fail today. Our goal with rapid optimization at amazon Bedrock for reasoning models is to reduce the amount of thinking tokens but not the precision of the sacrifice. After using rapid optimization, the optimized warning is as follows:

## Question
In an alternate universe where the mass of the electron was 1% heavier 
and the charges of the electron and proton were both 1% smaller, but all other fundamental constants
stayed the same, approximately how would the speed of sound in diamond change?

Answer Choices:
A. Decrease by 2%
B. Decrease by 1.5%
C. Decrease by 1%
D. Decrease by 0.5%
E. Stay approximately the same
F. Increase by 0.5%
G. Increase by 1%
H. Increase by 1.5%
I. Increase by 2%

## Instruction
Read the question above carefully and provide the most accurate answer possible.
If multiple choice options are provided within the question, respond with the entire text of the
correct answer option, not just the letter or number. Do not include any additional explanations or
preamble in your response.

Remember, your goal is to answer as precisely and accurately as possible!

The following figure shows how, for this specific case, the number of thought tokens was reduced by 35%, while obtaining the correct final response (B. decreases by 1.5%). Here, the number of thought tokens was reduced from 5,000 to 3,300. We also notice that in this and other examples with the original indications, part of the reasoning is summarized or repeated before the final response. As we can see in this example, the optimized notice provides clear instructions, separates different application sections and provides additional guidance based on the type of question and how to answer. This leads to traces of shorter and more clear reasoning and a directly removable final response.

Optimized indications can also lead to correct answers instead of incorrect after thinking long, because thinking does not guarantee a correct final response. In this case, we see that the number of thought tokens was reduced from 5,000 to 1,555, and the answer is obtained directly, instead of another long explanation and after thought. The following figure shows an example.

The two previous examples demonstrate ways in which rapid optimization can improve the results while shortening the output tokens for models such as Deepseek R1. Rapid optimization was also applied to 400 HLE questions. The following table summarizes the results.

Experiment	General precision	Average number of rapid tokens	Average number of completion tokens (Thought + answer)	Average tokens number (answer Only)	Average number of tokens (just thinking)	Completed Thought Percentage (6,000 maximum output token)
Baseline Deepseek	8.75	288	3334	271	3063	80.0%
Optimized depth information	11	326	1925	27	1898	90.3%

As we can see, the general precision increases to 11% in this subset of the HLE data set, the number of thought and exit tokens is reduced (therefore, it reduces the time for the last token and cost), and the rate of completing the thought increased to 90% in general. From our experiments, we see that, although there is no explicit reference to reduce thought tokens, the clearest and detailed instructions on the task in question after rapid optimization could reduce the additional effort involved for models such as Deepseek-R1 to make a self-clarification or an understanding of deeper problems. The rapid optimization for reasoning models ensures that the quality of thought and general flow, which is self -adaptive and depends on the question, is not largely affected, which leads to better final responses.

Conclusion

In this publication, we demonstrate how rapid optimization at amazon Bedrock can effectively improve the performance of intensive thought models such as Deepseek-R1. Through our experiments with the HLE data set, we show that the optimized indications not only reduced the number of thought tokens in a significant margin, but also improved the general accuracy of 8.75% to 11%. Optimization resulted in more efficient reasoning paths without sacrificing the quality of the answers, which led to faster response times and lower costs. This improvement in both efficiency and effectiveness suggests that rapid optimization can be a valuable tool to implement heavy models of reasoning in production environments where precision and computational resources should be carefully balanced. As the ai field continues to evolve with more sophisticated thinking models, techniques such as rapid optimization will be increasingly important for practical applications.

To start with the rapid optimization at amazon Bedrock, see optimizing a notice and improving the performance of your generative IA applications with rapid optimization at amazon Bedrock.

Appendix

The next is the complete answer of the question about the Nate dog:


Okay, let's see. So the problem is about Nate's dog digging holes and Nate trying to fill them.
I need to figure out how many weeks it takes Nate to fill all the holes once he starts. Let me break
it down step by step.\n\nFirst, the dog digs 6 holes a day. He does this for 14 days while Nate is on
vacation. So when Nate comes back, there are a certain number of holes already dug. Let me calculate
that first. If the dog digs 6 holes each day for 14 days, that's 6 multiplied by 14. Let me do that
math: 6*14=84. So there are 84 holes when Nate gets home.\n\nNow, once Nate starts filling the holes,
he fills 9 holes per day. But here's the catch: the dog keeps digging 6 new holes every night. So 
each day, Nate fills 9 holes, but the dog adds 6 more overnight. I need to figure out the net change
per day. That would be the number of holes filled minus the new holes dug. So 9 filled - 6 dug = 3 
holes net reduction per day. Wait, but does this happen every day? Let me make sure.\n\nSo Nate works
during the day, filling 9 holes. Then, during the night, the dog digs 6 new holes. So each full day
(day and night) results in a net decrease of 3 holes. So starting from 84 holes, each day the total
number decreases by 3. But I need to check if the process is that Nate fills 9, then the dog digs 6,
so the next day starts with 84 -9 +6 = 84 -3 =81, and so on. So yes, each day the net is -3.\n\nBut
wait, maybe the timing is important. Let's think: when Nate starts filling, does he fill 9 holes on
the first day, and then the dog digs 6 that night? So each day after Nate fills, the dog adds 6. So
the sequence is: Day 1: Nate fills 9, holes become 84-9=75. Then night: dog digs 6, holes become
75+6=81. Day 2: Nate fills 9, holes 81-9=72. Night: +6=78. So each full day cycle (day and night)
reduces the total by 3. So yes, net 3 per day.\n\nBut wait, maybe on the last day, if Nate fills the
remaining holes and the dog doesn't get to dig that night because all holes are filled. Hmm, that's a
crucial point. For example, if on a certain day, the number of holes left is less than or equal to 9,
Nate can fill them all in that day, and the dog doesn't add any more because there are no holes left
to dig. So we need to check if on the day when Nate fills the remaining holes, the total would reach
zero, and the dog doesn't dig that night. So we can't just do total holes divided by 3, because the
last day might not have the dog digging.\n\nSo let's model this properly. Let's denote the number of
holes each day.\n\nStarting with 84 holes.\n\nEach day:\n\n1. Nate fills 9 holes. Subtract 9.\n\n2.
If the result is zero or negative, then done.\n\n3. Otherwise, the dog digs 6 holes. Add 6.\n\nSo we
need to iterate this until the number of holes is zero or less after Nate fills them.\n\nLet me try
to calculate day by day.\n\nStart: 84\n\nDay 1:\n\nAfter Nate fills: 84 -9 =75\n\nSince 75 >0, dog
digs 6: 75 +6=81\n\nDay 2:\n\nAfter Nate fills:81-9=72\n\nDog digs 6:72+6=78\n\nDay3:\n\n78-
9=69\n\n+6=75\n\nDay4:75-9=66+6=72\n\nDay5:72-9=63+6=69\n\nDay6:69-9=60+6=66\n\nDay7:66-
9=57+6=63\n\nDay8:63-9=54+6=60\n\nDay9:60-9=51+6=57\n\nDay10:57-9=48+6=54\n\nDay11:54-
9=45+6=51\n\nDay12:51-9=42+6=48\n\nDay13:48-9=39+6=45\n\nDay14:45-9=36+6=42\n\nDay15:42-
9=33+6=39\n\nDay16:39-9=30+6=36\n\nDay17:36-9=27+6=33\n\nDay18:33-9=24+6=30\n\nDay19:30-
9=21+6=27\n\nDay20:27-9=18+6=24\n\nDay21:24-9=15+6=21\n\nDay22:21-9=12+6=18\n\nDay23:18-
9=9+6=15\n\nDay24:15-9=6+6=12\n\nDay25:12-9=3+6=9\n\nDay26:9-9=0\n\nAt this point, after Nate fills
on day26, the holes are 0. Since there are no holes left, the dog can't dig any more that night. So
the process stops here.\n\nSo total days:26 days.\n\nBut the question asks for how many weeks. Since
a week is 7 days, we divide 26 by7.\n\n26 divided by7 is 3 with a remainder of5. So 3 weeks and5
days. But the question says \"how many weeks does it take him to fill in all the holes\". Depending
on whether they want it rounded up to full weeks or expressed as a fraction. Let me check the problem
statement again.\n\n\"How many weeks does it take him to fill in all the holes?\"\n\nTypically, such
problems expect the answer in whole weeks, rounded up if there's a remainder. Because you can't have
a fraction of a week in practical terms. So 26 days is 3 weeks and5 days, which is more than3 weeks
but less than4. So they might expect 4 weeks. But let me verify.\n\nAlternatively, maybe they want
the exact number in weeks as a fraction. 26 days is 26/7 weeks \u22483.714 weeks. But the problem
might expect an integer number of weeks, rounded up. Let's see.\n\nBut let's check the problem again.
It says \"how many weeks does it take him to fill in all the holes?\" So if it takes 26 days, which
is 3 weeks and5 days, then the answer would be 4 weeks if we're counting full weeks needed. Because
after 3 weeks (21 days), he still has 5 days left. So total weeks is4.\n\nBut maybe the answer is
26/7 weeks, which is approximately3.714, but since the problem might expect an exact answer in weeks,
possibly as a fraction. However, the problem might be designed to have a whole number. Wait, let's
check the math again.\n\nWait, when I did the day-by-day count, I got 26 days. Let me recount to make
sure I didn't make a mistake.\n\nStarting from day1 to day26. Let's check day26:\n\nStart with
day1:84\n\nAfter day1:81\n\n...\n\nLet me check day25:\n\nAfter day24:12\n\nDay25:12-9=3, then dog
digs6:3+6=9\n\nDay26:9-9=0. So day26 is the day when Nate fills the last 9 holes. So yes, 26
days.\n\n26 days divided by7 days/week=3.714... weeks. So the answer is 26/7 weeks, which is3 and5/7
weeks. But the problem might want it expressed as a mixed number or an exact number. However, the
question says \"how many weeks does it take him\", so likely they expect the answer in weeks,
possibly as a fraction. But let's see if there's a better way to model this without counting each
day.\n\nAlternative approach:\n\nTotal holes after vacation:84\n\nEach day, Nate fills9, dog digs6,
net -3 per day, except possibly the last day.\n\nBut if on the last day, Nate fills the remaining
holes without the dog digging, then we need to adjust.\n\nLet\u2019s model it as:\n\nLet H be the
number of holes. Start with H=84.\n\nEach day, H decreases by9, then if H>0, increases by6.\n\nWe
need to find the number of days until H<=0.\n\nThis can be modeled as:\n\nEach full cycle (day and
night) reduces H by3, except the last day where if H<=9, then H becomes0 and no addition.\n\nSo
the number of full cycles (days where H>9 after filling) plus the final day.\n\nLet\u2019s
denote:\n\nLet\u2019s say after n days, the holes would be:\n\nEach day, H = H -9 +6 = H -3, but only
if after subtracting9, H >=1 (since if H-9 <=0, then the dog doesn\u2019t dig). Wait, no. Wait,
the dog digs6 only if after Nate fills9, there are still holes left. Wait, no. Wait, the problem says
the dog keeps digging6 new holes every night. So regardless of whether there are holes left, the dog
digs6 every night. Wait, but if all holes are filled, can the dog dig new ones? The problem says
\"fill in all the holes\", so once all holes are filled, the dog can't dig new ones because there are
no existing holes? Wait, the problem might be implying that the dog digs new holes regardless, but
Nate is filling the existing ones. Wait, the problem says \"the dog keeps digging6 new holes every
night\". So perhaps every night, regardless of the current state, the dog digs6 new holes. But that
would mean that even if Nate fills all holes on a day, the dog digs6 new ones that night, making it
impossible to ever finish. But that can't be, because the problem asks how long to fill all holes,
which implies that eventually Nate can catch up.\n\nWait, this is a critical point. Let me re-examine
the problem statement:\n\n\"When Nate gets home, he starts filling in 9 holes a day, but the dog
keeps digging 6 new holes every night. How many weeks does it take him to fill in all the holes?
\"\n\nSo the key is: every night, the dog digs6 new holes. So even if Nate fills all the holes during
the day, the dog will dig6 new ones that night. But that would mean the process never ends. But the
problem must assume that once all holes are filled, the dog stops. But the wording is ambiguous.
However, logically, if all holes are filled, there's nothing for the dog to dig. So perhaps the dog
only digs6 holes if there's ground to dig, but the problem states \"keeps digging6 new holes every
night\", implying that regardless of the current state, the dog digs6 each night. This is a problem
because if Nate ever fills all holes, the next night the dog digs6, so Nate would have to fill those6
the next day, but then the dog digs6 again, leading to an infinite loop. But the problem must have a
solution, so likely the dog stops digging once all holes are filled. Therefore, the correct
interpretation is that each night, the dog digs6 new holes, but if there are no holes (i.e., all
filled), the dog can't dig. Therefore, the last day when Nate fills the remaining holes, the dog
doesn't dig that night.\n\nTherefore, the correct model is:\n\nEach day:\n\n1. Nate fills9
holes.\n\n2. If holes remain after filling, the dog digs6 that night.\n\nSo the process is:\n\nStart
with H=84.\n\nWhile H>0:\n\n H = H -9\n\n if H >0:\n\n H = H +6\n\n days +=1\n\nSo let's model
this with equations.\n\nLet\u2019s find the number of days required.\n\nThis is similar to the
classic problem where each day you have a net change, but with a different condition on the last
day.\n\nLet\u2019s denote that each \"cycle\" (day and night) reduces H by3, except possibly the last
day where H <=9, so after filling, H becomes0, and no addition.\n\nSo the total number of days is
the number of full cycles (each reducing H by3) plus the final day.\n\nBut how to calculate this?
\n\nLet\u2019s think of it as:\n\nLet\u2019s suppose that after n days, the number of holes is H_n
=84 -3n.\n\nBut this is only true if every day the net change is -3. However, this would be the case
until H_n -9 <=0. Wait, no. Wait, the H after filling is H -9. If H -9 <=0, then the dog
doesn't dig, so the total reduction is H (to 0). So we need to find the day when H -9 <=0.\n\nSo
let\u2019s solve for when H -9 <=0:\n\nH =84 -3(n-1) -9 <=0\n\nWait, this is getting
complicated. Let me use another approach.\n\nLet\u2019s consider that each day before the final day,
the net change is -3. On the final day, the net change is -9 (since the dog doesn\u2019t add
6).\n\nLet\u2019s let x be the number of full days (with net -3), and 1 final day (net -9).\n\nTotal
reduction: 3x +9 =84\n\nSo 3x =84 -9=75\n\nx=75/3=25\n\nSo x=25 days of net -3, and 1 final day of 
-9.\n\nTotal days=25+1=26 days.\n\nYes, this matches the day-by-day count. Therefore, 26 
days.\n\nConvert to weeks:26/7=3.714... weeks. But the problem asks \"how many weeks does it take
him\". If they want a whole number of weeks, rounded up, it's4 weeks. If they accept fractions,
it's26/7\u22483.714. But the problem might expect the answer in weeks as a mixed number, like3 and5/7
weeks. However, typically, such problems expect the answer in whole weeks, rounded up. Let me check
the problem statement again.\n\nThe problem says \"how many weeks does it take him to fill in all the
holes\". The answer should be in weeks, and since it's more than3 weeks but less than4, but you can't
have a fraction of a week in practical terms when scheduling, so they might expect4 weeks. However,
mathematically, it's26/7 weeks. Let me see if the problem expects an exact answer or a whole 
number.\n\nLooking back at the problem statement, it's a math problem, so likely expects the exact
value, which is26/7 weeks, which can be written as3 5/7 weeks. However, sometimes problems like this
expect the answer in whole weeks, rounding up. Let's check both possibilities.\n\nIf the answer is26
days, which is3 weeks and5 days, then depending on the context:\n\n- If partial weeks count as a full
week, then4 weeks.\n\n- If exact, then3 5/7 weeks.\n\nBut the problem doesn't specify, so as a math
problem, the precise answer is26/7 weeks, which is3 and5/7 weeks. However, the question might want it
in weeks as a whole number, rounded up. Let's see.\n\nBut let's check the problem again. The problem
says \"how many weeks does it take him to fill in all the holes?\" The answer choices (if any) aren't
given, but since the user hasn't provided options, we need to infer. In math problems like this,
unless specified otherwise, the answer is expected in the most precise form, which would be26/7 weeks
or3 5/7 weeks. However, sometimes problems expect whole numbers, so maybe the answer is4 weeks. But
let's verify with the day count.\n\n26 days is exactly3 weeks and5 days. If the question asks for the
number of weeks, and if part of a week counts as a full week, then it's4. But if they want the exact
number, it's3 5/7. However, the problem might have an integer answer. Wait, let's check the
calculations again.\n\nWait, when I did the day-by-day count, I got26 days. Let me verify that again
quickly.\n\nStart with84.\n\nEach day:\n\n1. Subtract9.\n\n2. If result >0, add6.\n\nRepeat until
H<=0.\n\nLet\u2019s model this with a formula.\n\nLet\u2019s denote that each day, except the
last, the net change is -3. The last day is -9.\n\nSo total holes =84 =3x +9\n\nSo3x=75 \u2192x=25
days of net -3, plus1 day of -9. Total26 days.\n\nYes, that's correct. So26 days.\n\n26 days divided
by7 days/week=3.714... weeks.\n\nBut the problem might want the answer as a mixed number, so3 and5/7
weeks. However, in some contexts, especially in school problems, they might expect the answer in
whole weeks, rounded up, so4 weeks. But since the problem is from Nate's perspective, he would count
the exact days, so converting26 days to weeks gives3 weeks and5 days. But the question asks for
weeks, so unless specified, it's better to present it as26/7 weeks, which is3 5/7 weeks. However,
let's check if there's a different approach that gives a whole number.\n\nAlternatively, maybe I made
a mistake in the day count. Let me recount the days more carefully.\n\nStarting with84
holes.\n\nDay1:\n\nFill9:84-9=75. Since75>0, dog digs6:75+6=81.\n\nDay2:\n\n81-9=72>0
\u2192+6=78.\n\nDay3:78-9=69>0 \u2192+6=75.\n\nDay4:75-9=66>0 \u2192+6=72.\n\nDay5:72-9=63>0
\u2192+6=69.\n\nDay6:69-9=60>0 \u2192+6=66.\n\nDay7:66-9=57>0 \u2192+6=63.\n\nDay8:63-9=54>0
\u2192+6=60.\n\nDay9:60-9=51>0 \u2192+6=57.\n\nDay10:57-9=48>0 \u2192+6=54.\n\nDay11:54-
9=45>0 \u2192+6=51.\n\nDay12:51-9=42>0 \u2192+6=48.\n\nDay13:48-9=39>0 
\u2192+6=45.\n\nDay14:45-9=36>0 \u2192+6=42.\n\nDay15:42-9=33>0 \u2192+6=39.\n\nDay16:39-
9=30>0 \u2192+6=36.\n\nDay17:36-9="

About the authors

Shreyas Subramanian It is a main data scientist and helps customers through the use of generative ai and deep learning to solve their commercial challenges using AWS services. Shreyas has a history of large -scale and ML optimization and the use of ML and reinforcement learning to accelerate optimization tasks.

Zhengyuan Shen He is a scientist applied at amazon Bedrock, specializing in fundamental models and ML modeling for complex tasks that include natural language and understanding structured data. He is passionate about ML innovative solutions to improve products or services, thus simplifying customer life through a perfect combination of science and engineering. External work, enjoys sport and kitchen.

Xuan Qi He is a scientist applied at amazon Bedrock, where he applies his background in physics to face complex challenges in automatic learning and artificial intelligence. Xuan is passionate to translate scientific concepts into practical applications that promote tangible improvements in technology. His work focuses on creating more intuitive and efficient systems that can understand and interact better with the world. Out of their professional activities, Xuan finds balance and creativity through his love for dancing and interpreting the violin, bringing the precision and harmony of these arts to their scientific efforts.

Shuai Wang He is a scientist and manager applied at amazon Bedrock, specialized in natural language procedures, automatic learning, large language modeling and other related areas.