Octo: An Open Source Large Transformer-Based Generalist Robot Policy Trained on 800,000 Trajectories from the Open X-Embodiment Dataset

When it comes to robot learning, standard practice is to use data sets tailored to the particular robot and job to train policies. Starting from scratch in this way requires a substantial amount of data collection for each activity, and the policies that are produced generally show little generalization. In theory, data collected from robots and previous work could be a solution; Training models on various control issues could improve their ability to generalize and perform better on subsequent tasks. In contrast to the ubiquity of general-purpose models in computer vision and natural language processing, creating a “general-purpose robot model” capable of controlling multiple robots has proven to be a formidable challenge. Dealing with robot realizations, sensor configurations, action spaces, task specifications, environments, and computing budgets are unique issues when training a unified control strategy in robotics.

Several publications have proposed basic robotics models that do just that (directly translate robot observations into actions) and offer generalization to new domains and robots with little or no triggering. Due to their versatility in low-level visuomotor control in activities, environments, and robotic systems, these models are generally called “generalist robotic policies” (GRP). While there has been progress toward a “general purpose robot model,” these models still have a long way to go. For example, they do not allow effective adjustment of new domains; the largest ones are not even available to the public. Another problem is that they limit downstream users to a predefined and often restrictive set of input observations, such as a single camera stream.

To better accommodate the variety of user interfaces found in robotic applications later, researchers from UC Berkeley, Stanford, Carnegie Mellon University, and Google Deepmind provide a method to pre-train generalist robot policies.

Octo is a pre-trained transformer-based strategy using 800,000 robot demos from the Open x-Embodiment dataset, the largest data set on robot manipulation. Octo is the first generalist robot manipulation policy that is completely open source, including the data, model checkpoints, and training process. It is also the first GRP to adapt effectively to new observations and spaces of action.

When trained on a diverse dataset of robots and tasks, the model is a transformative architecture that can convert any number of input tokens (generated from observations and tasks) into actions. This policy can be trained once and used for multiple robots, different camera configurations (e.g., wrist or workspace cameras), and other input methods (e.g., language commands, target images) by simply changing the tokens provided in the model. The model can be easily adjusted to fit other robot configurations, sensory inputs, action spaces or morphologies by incorporating the necessary adapters and refining it using a small data set from the target domain and a reasonable computing budget.

Previous research has delved into individual components of Octo, such as a transformer backbone, support for target image specification, and a diffusion head for modeling expressive action distributions. However, the true power of this combination as a generalist robot policy is a new and innovative concept. The researchers conducted extensive experiments with nine robots from four different universities, demonstrating that their integrated system achieves state-of-the-art results in out-of-the-box multirobot control for one- and two-arm manipulation tasks. They also demonstrated that Octo can be effectively used as an initialization to fit new observation and action spaces in unseen configurations. Throughout these experiments, they analyzed the impact of various design options on the quality of the pre-trained GRP, including data distribution, model architecture, and policy formulation. The evaluation highlighted the importance of scale and flexibility to achieve optimal performance.

In addition to this release, the team is making available all the resources necessary to train, use, reproduce and refine an Octo model. With parameters of 27M and 93M, respectively, its pre-trained Octo model checkpoints enable specification of out-of-the-box language and target image tasks and multiple RGB camera inputs. In addition to their entire pre-training process, which includes optimal data loaders, transformer implementations for multi-modal inputs, and tools to monitor training progress, they also offer scripts to tune these models in new domains.

While the team acknowledges there is still room for improvement in the model, such as language conditioning, support for wrist cameras, and incorporating data beyond ideal demonstrations, Octo represents a significant step toward robotic policymaking. generalists that are compatible with a variety of robot configurations. Octo aims to provide a practical platform where researchers and professionals can access larger data sets related to robotics. They anticipate that their work will enable the use of pre-trained models for rapid learning and generalization of tasks, thus advancing the field of robotics and machine learning.

Review the Paper and Project. All credit for this research goes to the researchers of this project. Also, don't forget to follow us on twitter.com/Marktechpost”>twitter. Join our Telegram channel, Discord channeland LinkedIn Grabove.

If you like our work, you will love our Newsletter..

Don't forget to join our 42k+ ML SubReddit