Mistral ai’s Mistral Large 2 (24.07) Base Model (FM) is now generally available on amazon Bedrock. Mistral Large 2 is the latest version of Mistral Large and, according to Mistral ai, offers significant improvements in multilingual capabilities, mathematics, reasoning, coding, and more.
In this post we discuss the benefits and capabilities of this new model with some examples.
Mistral Large 2 Overview
Mistral Large 2 is an advanced large language model (LLM) with state-of-the-art reasoning, knowledge, and coding capabilities according to Mistral ai. It is multilingual by design, supporting dozens of languages including English, French, German, Spanish, Italian, Chinese, Japanese, Korean, Portuguese, Dutch, Polish, Arabic, and Hindi. According to Mistral ai, significant effort was also put into improving the model’s reasoning capabilities. One of the key focuses during training was to minimize the model’s tendency to hallucinate, or generate information that sounds plausible but is incorrect or irrelevant. This was achieved by fine-tuning the model to be more cautious and insightful in its responses, ensuring that it delivers reliable and accurate results. Additionally, the new Mistral Large 2 is trained to recognize when it cannot find solutions or does not have enough information to provide a confident answer.
According to Mistral ai, the model is also proficient in coding, having been trained in over 80 programming languages such as Python, Java, C, C++, JavaScript, Bash, Swift, and Fortran. With its first-class agent capabilities, it can natively call functions and generate JSON, allowing for seamless interaction with external systems, APIs, and tools. Additionally, Mistral Large 2 (24.07) boasts advanced mathematical and reasoning capabilities, making it a powerful resource for tackling complex logical and computational challenges.
Mistral Large 2 also offers an expanded context window of 128,000 tokens. At the time of writing, the model (mistral.mistral-large-2407-v1:0) is available on us-west-2
AWS Region.
Get started with Mistral Large 2 on amazon Bedrock
If you are new to Mistral ai models, you can request access to the models in the amazon Bedrock console. For more details, see Managing Access to amazon Bedrock Core Models.
To try out Mistral Large 2 on the amazon Bedrock console, choose Text either Chat low Children's parks in the navigation pane. Then select Select model and choose Mistral as the category and Mistral Grande 24.07 as model.
By choice See API request, you can also access the model using code examples in the AWS Command Line Interface (AWS CLI) and the AWS SDKs. You can use model identifiers such as mistral.mistral-large-2407-v1:0
as shown in the following code:
$ aws bedrock-runtime invoke-model \
--model-id mistral.mistral-large-2407-v1:0 \
--body "{\"prompt\":\"(INST) this is where you place your input text (/INST)\", \"max_tokens\":200, \"temperature\":0.5, \"top_p\":0.9, \"top_k\":50}" \
--cli-binary-format raw-in-base64-out \
--region us-west-2 \
invoke-model-output.txt
In the following sections, we will delve deeper into the capabilities of Mistral Large 2.
Expanded context window
Mistral Large 2 supports a context window of 128,000 tokens, compared to Mistral Large (24.02), which had a context window of 32,000 tokens. This larger context window is important for developers because it allows the model to process and understand longer pieces of text, such as entire documents or code files, without losing context or coherence. This can be particularly useful for tasks such as code generation, documentation analysis, or any application that requires understanding and processing large amounts of text data.
Generating JSON and using tools
Mistral Large 2 now offers a native JSON output mode. This feature allows developers to receive model responses in a structured, easy-to-read format that can be easily integrated into various applications and systems. As JSON is a widely adopted data exchange standard, this capability simplifies the process of working with model outputs, making it more accessible and practical for developers across different domains and use cases. For more information on generating JSON with the Converse API, see amazon-bedrock-converse-api?lang=en” target=”_blank” rel=”noopener”>Generating JSON with amazon Bedrock's Converse API.
To generate JSON with the Converse API, you must define a toolSpec
In the following code, we present an example for a travel agency company that will take passenger information and requests and convert them to JSON:
# Define the tool configuration
import json
tool_list = (
{
"toolSpec": {
"name": "travel_agent",
"description": "Converts trip details as a json structure.",
"inputSchema": {
"json": {
"type": "object",
"properties": {
"origin_airport": {
"type": "string",
"description": "Origin airport (IATA code)"
},
"destination_airport": {
"type": "boolean",
"description": "Destination airport (IATA code)"
},
"departure_date": {
"type": "string",
"description": "Departure date",
},
"return_date": {
"type": "string",
"description": "Return date",
}
},
"required": (
"origin_airport",
"destination_airport",
"departure_date",
"return_date"
)
}
}
}
}
)
content = """
I would like to book a flight from New York (JFK) to London (LHR) for a round-trip.
The departure date is June 15, 2023, and the return date is June 25, 2023.
For the flight preferences, I would prefer to fly with Delta or United Airlines.
My preferred departure time range is between 8 AM and 11 AM, and my preferred arrival time range is between 9 AM and 1 PM (local time in London).
I am open to flights with one stop, but no more than that.
Please include non-stop flight options if available.
"""
message = {
"role": "user",
"content": (
{ "text": f"{content}" },
{ "text": "Please create a well-structured JSON object representing the flight booking request, ensuring proper nesting and organization of the data. Include sample data for better understanding. Create the JSON based on the content within the tags." }
),
}
# Bedrock client configuration
response = bedrock_client.converse(
modelId=model_id,
messages=(message),
inferenceConfig={
"maxTokens": 500,
"temperature": 0.1
},
toolConfig={
"tools": tool_list
}
)
response_message = response('output')('message')
response_content_blocks = response_message('content')
content_block = next((block for block in response_content_blocks if 'toolUse' in block), None)
tool_use_block = content_block('toolUse')
tool_result_dict = tool_use_block('input')
print(json.dumps(tool_result_dict, indent=4))
We received the following response:
{
"origin_airport": "JFK",
"destination_airport": "LHR",
"departure_date": "2023-06-15",
"return_date": "2023-06-25"
}
Mistral Large 2 was able to successfully take our user query and convert the appropriate information to JSON.
Mistral Large 2 also supports the Converse API and the use of tools. You can use the amazon Bedrock API to give a model access to tools that help it generate responses for messages you send to it. For example, you might have a chat application that allows users to find the most popular song playing on a radio station. To respond to a request for the most popular song, a model needs a tool that can query and return the song information. The following code shows an example for getting the correct train schedule:
# Define the tool configuration
toolConfig = {
"tools": (
{
"toolSpec": {
"name": "shinkansen_schedule",
"description": "Fetches Shinkansen train schedule departure times for a specified station and time.",
"inputSchema": {
"json": {
"type": "object",
"properties": {
"station": {
"type": "string",
"description": "The station name."
},
"departure_time": {
"type": "string",
"description": "The departure time in HH:MM format."
}
},
"required": ("station", "departure_time")
}
}
}
}
)
}
# Define shikansen schedule tool
def shinkansen_schedule(station, departure_time):
schedule = {
"Tokyo": {"09:00": "Hikari", "12:00": "Nozomi", "15:00": "Kodama"},
"Osaka": {"10:00": "Nozomi", "13:00": "Hikari", "16:00": "Kodama"}
}
return schedule.get(station, {}).get(departure_time, "No train found")
def prompt_mistral(prompt):
messages = ({"role": "user", "content": ({"text": prompt})})
converse_api_params = {
"modelId": model_id,
"messages": messages,
"toolConfig": toolConfig,
"inferenceConfig": {"temperature": 0.0, "maxTokens": 400},
}
response = bedrock_client.converse(**converse_api_params)
if response('output')('message')('content')(0).get('toolUse'):
tool_use = response('output')('message')('content')(0)('toolUse')
tool_name = tool_use('name')
tool_inputs = tool_use('input')
if tool_name == "shinkansen_schedule":
print("Mistral wants to use the shinkansen_schedule tool")
station = tool_inputs("station")
departure_time = tool_inputs("departure_time")
try:
result = shinkansen_schedule(station, departure_time)
print("Train schedule result:", result)
except ValueError as e:
print(f"Error: {str(e)}")
else:
print("Mistral responded with:")
print(response('output')('message')('content')(0)('text'))
prompt_mistral("What train departs Tokyo at 9:00?")
We received the following response:
Mistral wants to use the shinkansen_schedule tool
Train schedule result: Hikari
Mistral Large 2 was able to correctly identify the shinkansen tool and demonstrate its use.
Multilingual support
Mistral Large 2 now supports a host of character-based languages, including Chinese, Japanese, Korean, Arabic, and Hindi. This expanded language support enables developers to build applications and services that can meet the needs of users from diverse linguistic backgrounds. With multilingual capabilities, developers can create localized user interfaces, provide language-specific content and resources, and deliver a seamless experience for users regardless of their native language.
In the following example, we translate the author-generated customer emails into different languages such as Hindi and Japanese:
emails= """
"I recently bought your RGB gaming keyboard and absolutely love the customizable lighting features! Can you guide me on how to set up different profiles for each game I play?"
"I'm trying to use the macro keys on the gaming keyboard I just purchased, but they don't seem to be registering my inputs. Could you help me figure out what might be going wrong?"
"I'm considering buying your gaming keyboard and I'm curious about the key switch types. What options are available and what are their main differences?"
"I wanted to report a small issue where my keyboard's space bar is a bit squeaky. However, your quick-start guide was super helpful and I fixed it easily by following the lubrication tips. Just thought you might want to know!"
"My new gaming keyboard stopped working within a week of purchase. None of the keys respond, and the lights don't turn on. I need a solution or a replacement as soon as possible."
"I've noticed that the letters on the keys of my gaming keyboard are starting to fade after several months of use. Is this covered by the warranty?"
"I had an issue where my keyboard settings would reset every time I restarted my PC. I figured out it was due to a software conflict and resolved it by updating the firmware. Just wanted to ask if there are any new updates coming soon?"
"I've been having trouble with the keyboard software not saving my configurations, and it's starting to get frustrating. What can be done to ensure my settings are saved permanently?"
"""
def converse(prompt, inference_config):
messages = ({"role": "user", "content": ({"text": prompt})})
response = bedrock_client.converse(
messages=messages,
modelId=model_id,
inferenceConfig=inference_config
)
generated_text = response('output')('message')('content')(0)('text')
print(generated_text)
return generated_text
prompt=f"""
emails={emails}
Translate the following customer emails into these languages:
1. Hindi
2. Japanese
Label each language section accordingly""".format(emails=emails)
inference_config = {"temperature": 0.0, "maxTokens": 4000, "topP": 0.1}
response = converse(prompt, inference_config)
We received the following response:
1. "मैंने हाल ही में आपका RGB गेमिंग कीबोर्ड खरीदा और कस्टमाइजेबल लाइटिंग फीचर्स से बहुत प्रेम करता हूँ! क्या आप मुझे बता सकते हैं कि मैं हर गेम के लिए अलग-अलग प्रोफाइल कैसे सेट कर सकता हूँ?"
2. "मैं अपने नए गेमिंग कीबोर्ड पर मैक्रो कीज का उपयोग करने की कोशिश कर रहा हूँ, लेकिन वे मेरे इनपुट्स को रजिस्टर नहीं कर रहे हैं। क्या आप मुझे बता सकते हैं कि क्या गलत हो सकता है?"
3. "मैं आपका गेमिंग कीबोर्ड खरीदने के बारे में सोच रहा हूँ और मुझे की स्विच टाइप्स के बारे में जानना है। क्या विकल्प उपलब्ध हैं और उनके मुख्य अंतर क्या हैं?"
4. "मैं यह रिपोर्ट करना चाहता था कि मेरे कीबोर्ड का स्पेस बार थोड़ा सा चरमरा रहा है। हालाँकि, आपका क्विक-स्टार्ट गाइड बहुत मददगार था और मैंने लुब्रिकेशन टिप्स का पालन करके इसे आसानी से ठीक कर दिया। बस यह जानना चाहता था कि शायद आपको पता चलना चाहिए!"
5. "मेरा नया गेमिंग कीबोर्ड खरीद के एक सप्ताह के भीतर काम करना बंद हो गया। कोई भी की जवाब नहीं दे रहा है, और लाइट्स भी नहीं चालू हो रहे हैं। मुझे एक समाधान या एक रिप्लेसमेंट जितनी जल्दी हो सके चाहिए।"
6. "मैंने नोट किया है कि मेरे गेमिंग कीबोर्ड के कीज पर अक्षर कुछ महीनों के उपयोग के बाद फेड होने लगे हैं। क्या यह वारंटी के तहत कवर है?"
7. "मेरे कीबोर्ड सेटिंग्स हर बार मेरे पीसी को रीस्टार्ट करने पर रीसेट हो जाती थीं। मैंने पता लगाया कि यह एक सॉफ्टवेयर कॉन्फ्लिक्ट के कारण था और फर्मवेयर अपडेट करके इसे सुलझा दिया। बस पूछना चाहता था कि क्या कोई नए अपडेट आने वाले हैं?"
8. "मेरे कीबोर्ड सॉफ्टवेयर मेरी कॉन्फ़िगरेशन को सेव नहीं कर रहे हैं, और यह अब परेशान करने लगा है। मेरे सेटिंग्स को स्थायी रूप से सेव करने के लिए क्या किया जा सकता है?"
### Japanese
1. "最近、あなたのRGBゲーミングキーボードを購入し、カスタマイズ可能なライティング機能が大好きです! 各ゲームごとに異なるプロファイルを設定する方法を教えていただけますか?"
2. "新しく購入したゲーミングキーボードのマクロキーを使おうとしていますが、入力が認識されません。何が問題か教えていただけますか?"
3. "あなたのゲーミングキーボードを購入しようと考えていますが、キースイッチの種類について知りたいです。どのようなオプションがあり、その主な違いは何ですか?"
4. "キーボードのスペースバーが少しきしむようになりました。ただし、クイックスタートガイドが非常に役立ち、潤滑のヒントに従って簡単に修理できました。ただ、知っておいてほしいと思いました!"
5. "新しいゲーミングキーボードが購入後1週間で動作しなくなりました。どのキーも反応せず、ライトも点きません。できるだけ早く解決策や交換をお願いします。"
6. "ゲーミングキーボードのキーの文字が数ヶ月の使用後に消え始めました。これは保証に含まれますか?"
7. "キーボードの設定がPCを再起動するたびにリセットされていました。ソフトウェアの競合が原因であることがわかり、ファームウェアを更新して解決しました。新しいアップデートが近々来るのか教えていただけますか?"
8. "キーボードソフトウェアが設定を保存しないため、非常に困っています。設定を永続的に保存するために何ができますか?"
Coding tasks
Mistral Large 2 has been trained on over 80 coding languages, including popular ones like Python, Java, C, C++, JavaScript, and Bash, as well as more specialized languages like Swift and Fortran. This comprehensive language support enables developers to tackle a wide range of coding tasks and projects across multiple domains and platforms. Whether you work in web development, mobile apps, scientific computing, or systems programming, Mistral Large 2 can help you with code generation, debugging, refactoring, and other coding-related tasks. For example, the following code asks the model to generate a Python function:
prompt = """
Write a Python function called palindrome_prime_finder that finds all prime numbers within a given range that are also palindromes when written in base 10 (decimal system).
"""
inference_config = {"temperature": 0.0, "maxTokens": 1000, "topP": 0.1}
response = converse(prompt, inference_config)
We received the following response:
Certainly! Below is a Python function called `palindrome_prime_finder` that finds all prime numbers within a given range that are also palindromes when written in base 10.
```python
def is_prime(n):
"""Check if a number is prime."""
if n <= 1:
return False
if n <= 3:
return True
if n % 2 == 0 or n % 3 == 0:
return False
i = 5
while i * i <= n:
if n % i == 0 or n % (i + 2) == 0:
return False
i += 6
return True
def is_palindrome(n):
"""Check if a number is a palindrome."""
return str(n) == str(n)(::-1)
def palindrome_prime_finder(start, end):
"""Find all prime numbers within a given range that are also palindromes."""
palindrome_primes = ()
for num in range(start, end + 1):
if is_prime(num) and is_palindrome(num):
palindrome_primes.append(num)
return palindrome_primes
# Example usage:
start_range = 1
end_range = 1000
print(palindrome_prime_finder(start_range, end_range))
```
### Explanation:
1. **is_prime(n)**: This function checks if a number `n` is prime. It uses a basic primality test that eliminates even numbers and numbers divisible by 3 early, then checks divisibility up to the square root of `n`.
2. **is_palindrome(n)**: This function checks if a number `n` is a palindrome by converting it to a string and comparing it to its reverse.
3. **palindrome_prime_finder(start, end)**: This function iterates through all numbers in the given range `(start, end)`. For each number, it checks if it is both a prime and a palindrome. If both conditions are met, the number is added to the list `palindrome_primes`.
### Example Usage:
The example usage finds all palindromic prime numbers between 1 and 1000. You can adjust the `start_range` and `end_range` variables to find palindromic primes in any desired range.
Conclusion
Mistral Large 2 FM by Mistral ai is now available on amazon Bedrock in the US West (Oregon) region. To get started with Mistral Large 2 on amazon Bedrock, visit the amazon Bedrock console.
Interested in diving deeper? Check out Mistral Repository on AWSTo learn more about Mistral ai on amazon Bedrock, see Mistral ai Models Now Available on amazon Bedrock.
About the authors
Nithiyn Vijeaswaran Niithiyn is a Solutions Architect at AWS. His area of expertise is Generative ai and AWS ai Accelerators. He holds a BS in Computer Science and Bioinformatics. Niithiyn works closely with the GTM Generative ai team to help AWS customers on multiple fronts and accelerate their adoption of Generative ai. He is an avid Dallas Mavericks fan and enjoys collecting sneakers.
Armando Diaz Armando is a Solutions Architect at AWS. He focuses on generative ai, ai/ML, and data analytics. At AWS, Armando helps customers integrate cutting-edge generative ai capabilities into their systems, driving innovation and competitive advantage. When not working, he enjoys spending time with his wife and family, hiking, and traveling the world.
Preston Tuggle is a Senior Solutions Architect specializing in Generative ai.