Falcon 3 models now available at Amazon SageMaker Jumstart

Today, we are pleased to announce that the family of Falcon 3 models of Tii is available at amazon Sagemaker Jumstart. In this publication, we explore how to implement this model efficiently at amazon Sagemaker ai.

General description of the Falcon 3 models family

The Falcon 3 family, developed by technology Innovation Institute (TII) in Abu Dhabi, represents a significant advance in open source language models. This collection includes five base models ranging from 1 billion to 10 billion parameters, with an approach in improving science, mathematics and coding capabilities. The family consists of Falcon3-1b-Base, Falcon3-3b-Base, Falcon3-Mamba-7b-Base, Falcon3-7b-Base and Falcon3-10b-base along with its instructions variants.

These models show innovations, such as efficient priority techniques, scale for better reasoning and knowledge distillation for better performance in smaller models. In particular, the FALCON3-10B-BASE model achieves a state-of-the-art performance for models of less than 13 billion parameters in zero shooting tasks and few shots. The Falcon 3 family also includes several versions adjusted as instruction models and admits different quantification formats, which makes them versatile for a wide range of applications.

Currently, SageMaker JUMPTart offers the base versions of Falcon3-3b, Falcon3-7B and Falcon3-10B, along with its corresponding instructions variants, as well as Falcon3-1b-Instruct.

Start with SageMaker JUMPTart

SageMaker Jumpstart is an automatic learning center (ML) that can help accelerate its ML trip. With SageMaker JUMPTart, you can evaluate, compare and select previously trained base models (FMS), including Falcon 3 models. These models are totally customizable for their use case with their data.

The implementation of a FALCON 3 model through SageMaker JUMPTART offers two convenient approaches: using the intuitive user interface of SageMaker JUMPTart or implement programmatically through the SageMaker Python SDK. Let's explore both methods to help you choose the approach that best suits your needs.

Implement Falcon 3 using the SageMaker JUMPTart user

Complete the following steps to implement Falcon 3 through the JUMPTart user interface:

1. To access SageMaker JUMPTart, use one of the following methods:
   1. In amazon Sagemaker Unified Studio, in the Build Menu, choose JUMPTart models low Model development.
      
   2. Alternatively, at amazon Sagemaker Studio, choose Splash In the navigation panel.
      

1. Look for the Falcon3-10b base in the model browser.
   
2. Choose the model and choose Deploy.
   
3. For Type of instanceUse the default instance or choose a different instance.
4. Choose Deploy.
   After a while, the end point state will be shown as Intermediate And you can execute inference against her.
   

Implement Falcon 3 by programming using the sagemaker python SDK

For equipment that seeks to automate the implementation or integrate with existing mlops pipes, you can use the SDK sagemaker:

from sagemaker.serve.builder.model_builder import ModelBuilder
from sagemaker.serve.builder.schema_builder import SchemaBuilder
from sagemaker.jumpstart.model import ModelAccessConfig
from sagemaker.session import Session
import logging

sagemaker_session = Session()

artifacts_bucket_name = sagemaker_session.default_bucket()
execution_role_arn = sagemaker_session.get_caller_identity_arn()


js_model_id = "huggingface-llm-falcon-3-10B-base"

gpu_instance_type = "ml.g5.12xlarge"  

response = "Hello, I'm a language model, and I'm here to help you with your English."

sample_input = {
    "inputs": "Hello, I'm a language model,",
    "parameters": {"max_new_tokens": 128, "top_p": 0.9, "temperature": 0.6},
}

sample_output = ({"generated_text": response})

schema_builder = SchemaBuilder(sample_input, sample_output)

model_builder = ModelBuilder(
    model=js_model_id,
    schema_builder=schema_builder,
    sagemaker_session=sagemaker_session,
    role_arn=execution_role_arn,
    log_level=logging.ERROR
)

model= model_builder.build()

predictor = model.deploy(model_access_configs={js_model_id:ModelAccessConfig(accept_eula=True)}, accept_eula=True)

Execute inference in the predictor:

predictor.predict(sample_input)

If you want to configure the ability to climb to zero after the implementation, see the savings of unlock costs with the new zero scale function in SageMaker's inference.

Clean

To clean the model and the end point, use the following code:

predictor.delete_model()
predictor.delete_endpoint()

Conclusion

In this publication, we explore how SageMaker JUMPTart empowers data scientists and ML engineers to discover, access and execute a wide range of FMS previously trained for inference, including the family of Falcon Models 3. Visit SageMaker Jumstart in SageMaker Studio Now to start. For more information, see the SageMaker JUMPTART PRETRANED, amazon SageMaker Jumstart Foundation Models models and starting with amazon Sagemaker Jumstart.

About the authors

Niithiyn Vijeasswan It is a generative architect of solutions specialized in ai with the third -party model science team in AWS. Its focus area are the generative accelerators of ai and AWS ai. It has a degree in computer science and bioinformatics.

Marc Karp He is an ML architect with the amazon Sagemaker service team. It focuses on helping customers design, deploying and managing ML workloads at scale. In his free time, he likes to travel and explore new places.

Ragu Ramesha He is a senior architect of ML solutions with the amazon Sagemaker service team. It focuses on helping customers build, deploy and migrate ML ML production to SageMaker on a scale. He specializes in automatic learning, ai and computer vision domains, and has a master's degree in UT Dallas. In his free time, he likes to travel and photograph.

Banu Nagasundaram He leads products, engineering and strategic associations for Sagemaker Jumstart, Sagemaker's Machine Learning and Genai Hub. He is passionate to build solutions that help customers accelerate their ai trip and unlock commercial value.