Deploy Meta Llama 3.1-8B on AWS Inferentia using Amazon EKS and vLLM
With the rise of large language models (LLMs) like Meta Llama 3.1, there is an increasing need for scalable, reliable, ...
Tag: EKS
Introducing Amazon EKS support in Amazon SageMaker HyperPod
We are thrilled to introduce amazon Elastic Kubernetes Service (amazon EKS) support in amazon SageMaker HyperPod, a purpose-built infrastructure engineered ...
Detecting and recovering from node issues for AWS Neuron nodes within Amazon EKS clusters
Implementing hardware resilience in your training infrastructure is critical to mitigating risks and enabling uninterrupted model training. By implementing features ...
Accelerate your generative AI distributed training workloads with the NVIDIA NeMo Framework on Amazon EKS
In today’s rapidly evolving landscape of artificial intelligence (ai), training large language models (LLMs) poses significant challenges. These models often ...
Scale and simplify ML workload monitoring on Amazon EKS with AWS Neuron Monitor container
amazon Web Services is excited to announce the launch of the AWS Neuron Monitor container, an innovative tool designed to ...
Scale AI training and inference for drug discovery through Amazon EKS and Karpenter
This is a guest post co-written with the leadership team of Iambic Therapeutics. ai/" target="_blank" rel="noopener">Iambic Therapeutics is a drug ...
Scale LLMs with PyTorch 2.0 FSDP on Amazon EKS – Part 2
This is a guest post co-written with Meta’s PyTorch team and is a continuation of Part 1 of this series, ...
Federated learning on AWS using FedML, Amazon EKS, and Amazon SageMaker
This post is co-written with Chaoyang He, Al Nevarez and Salman Avestimehr from FedML. Many organizations are implementing machine learning ...
Recommended.
