Choice of tools with Amazon Nova models

In many generative applications of ai, a large language model (LLM) such as amazon Nova is used to respond to a user consultation based on knowledge or context of the model provided. However, as the use cases have matured, the ability of a model of having access to tools or structures that would be inherently outside the model reference frame has become essential. This could be API, code functions or schemes and structures required by its final application. This capacity has become what is known as Use of the tool either Functions called.

To add a fine grain control to how the tools are used, we have launched a function for the choice of tools for amazon Nova models. Instead of trusting fast engineering, the choice of the tool forces the model to adhere to the configuration in place.

In this publication, we discussed the use of tools and the new function of choice of tools, with cases of example use.

Use of the tool with amazon Nova

To illustrate the concept of use of tools, we can imagine a situation in which we provide amazon Nova access to some different tools, such as a calculator or a meteorological API. According to the user's consultation, amazon Nova will select the appropriate tool and tell you how to use it. For example, if a user asks “What is the weather in Seattle?” amazon Nova will use the weather tool.

The following diagram illustrates an example workflow between a amazon Nova model, its available tools and related external resources.

The use of the tool in the nucleus is the selection of the tool and its parameters. The responsibility of executing external functionality is left to the application or the developer. After the application executes the tool, you can return the results to the model for the generation of the final response.

Let's explore some examples in more detail. The following diagram illustrates the workflow of a amazon Nova model using a function call to access a weather API and return the response to the user.

The following diagram illustrates the workflow of a amazon Nova model using a function call to access a calculator tool.

Choice of tools with amazon Nova

He toolChoice The API parameter allows you to control when a tool is called. There are three compatible options for this parameter:

Any – With the choice of tools AnyThe model will select at least one of the tools available each time:
```
{
   "toolChoice": {
        "any": {}
    }
}
```

Tool – With the choice of tools ToolThe model will always use the requested tool:

{
   "toolChoice": {
        "tool": {
            "name": "name_of_tool"
        }
    }
}

Car – Choice of tools Auto It is the predetermined behavior and will leave the selection of the tool completely to the model:
```
{
   "toolChoice": {
        "auto": {}
    }
}
```

A popular tactics to improve the reasoning capabilities of a model is to use the chain of thought. When using the choice of the tool autoamazon Nova will use the chain of thought and the model response will include both the reasoning and the tool that was selected.

This behavior will differ according to the case of use. When tool either any They are selected as the choice of the tool, amazon Nova will generate only the tools and not the exit thinking chain.

Use cases

In this section, we explore different cases of use for the choice of tools.

Structured output/JSON mode

In certain scenarios, you may want amazon Nova to use a specific tool to answer the user's question, even if amazon Nova believes you can provide an answer without the use of a tool. A common use case for this approach is to enforce the structured output/JSON mode. It is often essential to have the structured output of LLMS return, because this allows the cases of downstream use to consume and process the generated outputs more effectively. In these cases, the tools used do not necessarily need to be functions on the client's side: they can be used when the model is required to return the JSON output that adheres to a predefined scheme, which forces amazon Nova to use the specified tool.

By using tools to enforce the structured output, it provides a unique tool with a descriptive JSON inputSchema. Specify the tool with {"tool" : {"name" : "Your tool name"}}. The model will pass the entrance to the tool, so the name of the tool and its description must be from the perspective of the model.

For example, consider a food website. When a description of the dish is provided, the website can extract the details of the recipe, such as cooking time, ingredients, the name of the dish and the level of difficulty, to facilitate the search and filtering capabilities of users. See the following example code:

import boto3
import json

tool_config = {
    "toolChoice": {
        "name": { "tool" : "extract_recipe"}
    },
    "tools": (
        {
            "toolSpec": {
                "name": "extract_recipe",
                "description": "Extract recipe for cooking instructions",
                "inputSchema": {
                    "json": {
                        "type": "object",
                        "properties": {
                            "recipe": {
                                "type": "object",
                                "properties": {
                                    "name": {
                                        "type": "string",
                                        "description": "Name of the recipe"
                                    },
                                    "description": {
                                        "type": "string",
                                        "description": "Brief description of the dish"
                                    },
                                    "prep_time": {
                                        "type": "integer",
                                        "description": "Preparation time in minutes"
                                    },
                                    "cook_time": {
                                        "type": "integer",
                                        "description": "Cooking time in minutes"
                                    },
                                    "servings": {
                                        "type": "integer",
                                        "description": "Number of servings"
                                    },
                                    "difficulty": {
                                        "type": "string",
                                        "enum": ("easy", "medium", "hard"),
                                        "description": "Difficulty level of the recipe"
                                    },
                                    "ingredients": {
                                        "type": "array",
                                        "items": {
                                            "type": "object",
                                            "properties": {
                                                "item": {
                                                    "type": "string",
                                                    "description": "Name of ingredient"
                                                },
                                                "amount": {
                                                    "type": "number",
                                                    "description": "Quantity of ingredient"
                                                },
                                                "unit": {
                                                    "type": "string",
                                                    "description": "Unit of measurement"
                                                }
                                            },
                                            "required": ("item", "amount", "unit")
                                        }
                                    },
                                    "instructions": {
                                        "type": "array",
                                        "items": {
                                            "type": "string",
                                            "description": "Step-by-step cooking instructions"
                                        }
                                    },
                                    "tags": {
                                        "type": "array",
                                        "items": {
                                            "type": "string",
                                            "description": "Categories or labels for the recipe"
                                        }
                                    }
                                },
                                "required": ("name", "ingredients", "instructions")
                            }
                        },
                        "required": ("recipe")
                    }
                }
            }
        }
    )
}

messages = ({
    "role": "user",
    "content": (
        {"text": input_text},
    )
})

inf_params = {"topP": 1, "temperature": 1}

client = boto3.client("bedrock-runtime", region_name="us-east-1")

response = client.converse(
    modelId="us.amazon.nova-micro-v1:0",
    messages=messages,
    toolConfig=tool_config,
    inferenceConfig=inf_params,
    additionalModelRequestFields= {"inferenceConfig": { "topK": 1 } }
)
print(json.dumps(response('output')('message')('content')(0)(), indent=2))

We can provide a detailed description of a dish as a text input:

Legend has it that this decadent chocolate lava cake was born out of a baking mistake in New York's Any Kitchen back in 1987, when chef John Doe pulled a chocolate sponge cake out of the oven too early, only to discover that the dessert world would never be the same. Today I'm sharing my foolproof version, refined over countless dinner parties. Picture a delicate chocolate cake that, when pierced with a fork, releases a stream of warm, velvety chocolate sauce – it's pure theater at the table. While it looks like a restaurant-worthy masterpiece, the beauty lies in its simplicity: just six ingredients (good quality dark chocolate, unsalted butter, eggs, sugar, flour, and a pinch of salt) transform into individual cakes in under 15 minutes. The secret? Precise timing is everything. Pull them from the oven a minute too late, and you'll miss that magical molten center; too early, and they'll be raw. But hit that sweet spot at exactly 12 minutes, when the edges are set but the center still wobbles slightly, and you've achieved dessert perfection. I love serving these straight from the oven, dusted with powdered sugar and topped with a small scoop of vanilla bean ice cream that slowly melts into the warm cake. The contrast of temperatures and textures – warm and cold, crisp and gooey – makes this simple dessert absolutely unforgettable.

We can force amazon Nova to use the tool extract_recipewhich will generate a structured JSON output that adheres to the predefined scheme provided as the tool input scheme:

 {
  "toolUseId": "tooluse_4YT_DYwGQlicsNYMbWFGPA",
  "name": "extract_recipe",
  "input": {
    "recipe": {
      "name": "Decadent Chocolate Lava Cake",
      "description": "A delicate chocolate cake that releases a stream of warm, velvety chocolate sauce when pierced with a fork. It's pure theater at the table.",
      "difficulty": "medium",
      "ingredients": (
        {
          "item": "good quality dark chocolate",
          "amount": 125,
          "unit": "g"
        },
        {
          "item": "unsalted butter",
          "amount": 125,
          "unit": "g"
        },
        {
          "item": "eggs",
          "amount": 4,
          "unit": ""
        },
        {
          "item": "sugar",
          "amount": 100,
          "unit": "g"
        },
        {
          "item": "flour",
          "amount": 50,
          "unit": "g"
        },
        {
          "item": "salt",
          "amount": 0.5,
          "unit": "pinch"
        }
      ),
      "instructions": (
        "Preheat the oven to 200\u00b0C (400\u00b0F).",
        "Melt the chocolate and butter together in a heatproof bowl over a saucepan of simmering water.",
        "In a separate bowl, whisk the eggs and sugar until pale and creamy.",
        "Fold the melted chocolate mixture into the egg and sugar mixture.",
        "Sift the flour and salt into the mixture and gently fold until just combined.",
        "Divide the mixture among six ramekins and bake for 12 minutes.",
        "Serve straight from the oven, dusted with powdered sugar and topped with a small scoop of vanilla bean ice cream."
      ),
      "prep_time": 10,
      "cook_time": 12,
      "servings": 6,
      "tags": (
        "dessert",
        "chocolate",
        "cake"
      )
    }
  }
}

API generation

Another common scenario is to demand that amazon Nova select a tool of the available options regardless of the user consultation context. An example of this is with the API final point selection. In this situation, we do not know the specific tool to use, and we allow the model to choose among those available.

With the choice of the tool anyYou can make sure the model always uses at least one of the available tools. Because of this, we provide a tool that can be used for when an API is not relevant. Another example would be to provide a tool that allows explanatory questions.

In this example, we provide the two different APIs and a non -compatible API tool that will select based on the user consultation:

import boto3
import json

tool_config = {
    "toolChoice": {
        "any": {}
    },
    "tools": (
         {
            "toolSpec": {
                "name": "get_all_products",
                "description": "API to retrieve multiple products with filtering and pagination options",
                "inputSchema": {
                    "json": {
                        "type": "object",
                        "properties": {
                            "sort_by": {
                                "type": "string",
                                "description": "Field to sort results by. One of: price, name, created_date, popularity",
                                "default": "created_date"
                            },
                            "sort_order": {
                                "type": "string",
                                "description": "Order of sorting (ascending or descending). One of: asc, desc",
                                "default": "desc"
                            },
                        },
                        "required": ()
                    }
                }
            }
        },
        {
            "toolSpec": {
                "name": "get_products_by_id",
                "description": "API to retrieve retail products based on search criteria",
                "inputSchema": {
                    "json": {
                        "type": "object",
                        "properties": {
                            "product_id": {
                                "type": "string",
                                "description": "Unique identifier of the product"
                            },
                        },
                        "required": ("product_id")
                    }
                }
            }
        },
        {
            "toolSpec": {
                "name": "unsupported_api",
                "description": "API to use when the user query does not relate to the other available APIs",
                "inputSchema": {
                    "json": {
                        "type": "object",
                        "properties": {
                            "reasoning": {
                                "type": "string",
                                "description": "The reasoning for why the user query did not have a valid API available"
                            },
                        },
                        "required": ("reasoning")
                    }
                }
            }
        }
    )
}


messages = ({
    "role": "user",
    "content": (
        {"text": input_text},
    )
})

inf_params = {"topP": 1, "temperature": 1}

client = boto3.client("bedrock-runtime", region_name="us-east-1")

response = client.converse(
    modelId="us.amazon.nova-micro-v1:0",
    messages=messages,
    toolConfig=tool_config,
    inferenceConfig=inf_params,
    additionalModelRequestFields= {"inferenceConfig": { "topK": 1 } }
)

print(json.dumps(response('output')('message')('content')(0), indent=2))

A user entry of “Can you get all available products?” I would issue the following:

{
  "toolUse": {
    "toolUseId": "tooluse_YCNbT0GwSAyjIYOuWnDhkw",
    "name": "get_all_products",
    "input": {}
  }
}

While “Can you get my most recent orders?” I would issue the following:

{
  "toolUse": {
    "toolUseId": "tooluse_jpiZnrVcQDS1sAa-qPwIQw",
    "name": "unsupported_api",
    "input": {
      "reasoning": "The available tools do not support retrieving user orders. The user's request is for personal order information, which is not covered by the provided APIs."
    }
  }
}

Chat

The final option for choosing the tool is auto. This is the predetermined behavior, so it is consistent with not providing any tool option at all.

The use of this tool option will allow the option to use the tool or simply text output. If the model selects a tool, there will be a block of tools and a block of text. If the model responds without tool, only a block of text is returned. In the following example, we want to allow the model to respond to the user or call a tool if necessary:

import boto3
import json

tool_config = {
    "toolChoice": {
        "auto": {}
    },
    "tools": (
         {
            "toolSpec": {
                "name": "search",
                "description": "API that provides access to the internet",
                "inputSchema": {
                    "json": {
                        "type": "object",
                        "properties": {
                            "query": {
                                "type": "string",
                                "description": "Query to search by",
                            },
                        },
                        "required": ("query")
                    }
                }
            }
        }
    )
}

messages = ({
    "role": "user",
    "content": (
        {"text": input_text},
    )
})

system = ({
    "text": "ou are a helpful chatbot. You can use a tool if necessary or respond to the user query"
})

inf_params = {"topP": 1, "temperature": 1}

client = boto3.client("bedrock-runtime", region_name="us-east-1")

response = client.converse(
    modelId="us.amazon.nova-micro-v1:0",
    messages=messages,
    toolConfig=tool_config,
    inferenceConfig=inf_params,
    additionalModelRequestFields= {"inferenceConfig": { "topK": 1 } }
)


if (response("stopReason") == "tool_use"):
    tool_use = next(
        block("toolUse")
        for block in response("output")("message")("content")
            if "toolUse" in block
    )
   print(json.dumps(tool_use, indent=2))
 else:
    pattern = r'.*?\\n\\n|.*?'
    text_response = response("output")("message")("content")(0)("text")
    stripped_text = re.sub(pattern, '', text_response, flags=re.DOTALL)
    
    print(stripped_text)

A user entry of “What is the weather in San Francisco?” would result in a tool call:

{
  "toolUseId": "tooluse_IwtBnbuuSoynn1qFiGtmHA",
  "name": "search",
  "input": {
    "query": "what is the weather in san francisco"
  }
}

While the model gives a direct question like “How many months are there in a year?” I would answer with a text response to the user:

There are 12 months in a year.

Considerations

There are some best practices that are required for tools calls with Nova models. The first is to use greedy decoding parameters With the amazon Nova models, that requires establishing a temperature, Top P and Top K of 1. can consult the previous code examples on how to establish them. The use of greedy decoding parameters forces the models to produce deterministic responses and improve the success rate of the tool call.

The second consideration is the JSON scheme that is using for the consideration of the tool. At the time of writing this article, amazon Nova Models supports a limited subset of JSON schemes, so it is possible that the model does not collect them as expected. The common fields would be $def and $ref fields. Make sure your scheme has the following set of upper level fields: type (It must be object), propertiesand required.

Finally, for the greatest impact on the success of tool calls, you must optimize the configurations of your tool. Descriptions and names must be very clear. If there are nuances when one should be called to the other, be sure to include it concisely in the tool descriptions.

Conclusion

The use of the choice of tools in the workflows of tool calls is a scalable way of controlling how a model invokes tools. Instead of trusting fast engineering, the choice of the tool forces the model to adhere to the configuration in place. However, there are complexities for tool calls; For more information, see the use of the tool (functions calls) with amazon Nova, call tool calling systems.

Explore how amazon Nova models can improve their generative cases of use of today.

About the authors

Jean farmer It is a generative architect of Ama's General Intelligence (AGI) teams of amazon, specialized in agent applications. Based in Seattle, Washington, works at the intersection of self -employed systems and practical commercial solutions, helping to shape the AGI's future in amazon.

Sharon Li He is an IA/ML specialized solution architect at amazon Web Services (AWS) based in Boston, Massachusetts. With a passion for taking advantage of avant -garde technology, Sharon is at the forefront of developing and implementing innovative generative solutions on the Aws Cloud platform.

Lulu Wong He is an ai UX designer in the amazon artificial general intelligence team (AGI). With experience in computer science, learning design and experience of the user, unites the technical domains and user experience by shaping how IA systems interact with humans, refining the entry-salary behaviors of the model and the creation of resources to make the products of ai be more accessible to users.

Choice of tools with Amazon Nova models

Technical Terrence Team

The best -selling Walmart $ 900 section is for sale for only $ 431, and buyers call it 'very comfortable'

Leave a Reply Cancel reply

Recommended.

Bitcoin price is heading towards the weekly low of $22,300: will we see a bounce?

MicroStrategy Buys $2 Billion in Bitcoin, Increases Holdings to $23 Billion

The Direct-To-Consumer 'Buy with Prime' program of Amazon to include Adidas

YouTube is testing a new subscription option that could remove most ads

Los streams están hechos de esto: ¿las plataformas digitales cambiarán nuestros recuerdos musicales? | Música

Categories

Important Links

Choice of tools with Amazon Nova models

Use of the tool with amazon Nova

Choice of tools with amazon Nova

Use cases

Structured output/JSON mode

API generation

Chat

Considerations

Conclusion

About the authors

Related

Technical Terrence Team

The best -selling Walmart $ 900 section is for sale for only $ 431, and buyers call it 'very comfortable'

Leave a Reply Cancel reply

Recommended.

Bitcoin price is heading towards the weekly low of $22,300: will we see a bounce?

MicroStrategy Buys $2 Billion in Bitcoin, Increases Holdings to $23 Billion

The Direct-To-Consumer 'Buy with Prime' program of Amazon to include Adidas

YouTube is testing a new subscription option that could remove most ads

Los streams están hechos de esto: ¿las plataformas digitales cambiarán nuestros recuerdos musicales? | Música

Categories

Important Links

Get daily news updates to your inbox!