Amazon Robotics Open-Sources ARMBench: a large open-source dataset for training robots

Picking, sorting and packing are just a few of the many warehouse operations that can be automated using robotic object handling systems. It is not easy to build robust and scalable robotic systems for use in handling objects in warehouses. Warehouses now handle millions of items that vary greatly in size, shape, material, and other physical characteristics. Disorganized arrangements of these things within containers make robotic perception and planning difficult. More fundamental work is required to make visual perception algorithms such as segmentation and object identification work with novel elements and configurations. New challenges (such as defect identification) and metrics (such as prediction uncertainty assessment) need to be established to capture the size and high-accuracy needs of such systems.

A large-scale benchmark dataset for perceptual and manipulation problems in robotic pick-and-place tasks is available through the Amazon Robotic Manipulation Benchmark, ARMBench. The data set contains many products and configurations collected from an Amazon warehouse. Includes high-quality annotated photos and videos for the many steps of robotic manipulation, such as picking, transferring, and positioning.

ARMBench Datasets features:

1. A treasure trove of sensor readings collected by a pick and place robotic handling work cell

2. Containerized Object Metadata and Visual References

3. Annotations are collected automatically (due to system architecture) or manually (by humans).

4. Robotic Manipulation Perception Benchmarking Tasks and Metrics

Amazon product categories and physical attributes such as size, shape, material, deformability, appearance, brittleness, etc., are represented in the data set.

The data collection platform is a warehouse pick and place work cell using robotic handling. The robotic arm in the work cell is equipped with a vacuum end effector. It shows several items with different properties and arrangements inside a box. The robotic arm’s job is to individualize the contents of the bin, removing one item at a time and placing it on rotating trays.

When a bin is empty, the work cell is released and a new bin is pushed in to take its place. While the process is fully automated, a human is still involved to check the progress of each pick-and-place task, provide any necessary annotations, and troubleshoot any issues that arise. Many image sensors are installed in the work cell to facilitate and confirm the pick and place operation.

The dataset has annotated data for the three most important computer vision tasks: object segmentation, object identification, and defect detection in video and still images.

Object segmentation

Object instance segmentation is the process of separating individual items in a warehouse’s containerized storage system. Instance segmentation guides subsequent robotic operations in handling objects, including grip creation, motion planning, and placement. The success of the selection, the recognition of objects and the introduction of failures depend on the precision of the segmentation of instances.

Object segments of more than 50,000 photos have been manually tagged to a high-grade level. Instance segmentation algorithms face a new problem when it comes to variations in object types and clutter levels.

object identification

The process of correctly assigning an image region to one of the items in a database is known as object identification (ID). This work can be done before or after a robot picks up an object. During the pre-picking phase, placing a part of an item in the bag provides quick access to any previously purchased item model or property for handling planning.

For robotic manipulation, this poses a problem of open-set object recognition and confidence estimation. The data set, which includes more than 190,000 distinct objects in various configurations, will evaluate and compare several few-shot classification and image retrieval approaches that incorporate uncertainty estimates.

Defect detection

The defect identification task is to determine whether or not a defect was introduced during robotic manipulation. The data set contains two types of robot-induced defects: multiple choice and package defect. A “packaging defect” occurs when the item’s packaging is damaged in some way, either because it was opened or broken into several pieces. When many items need to be moved from one bin to another, this process, known as “multiple pick,” is employed.

Rare but costly robot-induced failures, such as those seen during the picking and packing of multiple items, are detected using human-issued tags. The collection includes more than 100,000 actions without failures and more than 19,000 photos and 4,000 videos of activities with problems.

In conclusion

Process automation in contemporary warehouses requires a robotic manipulator to cope with a wide range of products, unstructured storage and dynamically changing inventory. In such environments, it is difficult to perceive the identity, physical properties, and state of objects during handling. There needs to be more diversity in element attributes, disorder, and interactions in existing robotic manipulation datasets, as they only consider a small subset of objects or rely on 3D models to create synthetic configurations. Using a robotic manipulator to perform the separation of objects from containers containing various contents, the researchers present a large-scale dataset obtained from an Amazon warehouse. Images, videos, and information representing more than 235,000 pick-and-place operations on 190,000 different items are available on ARMBench. Information is recorded prior to selection, during transfer, and after placement. Learning generalizable representations that can be transferred to other visual perception tasks will be made possible by the availability of large amounts of sensor data and detailed object properties. Additionally, the researchers want to add 3D data and annotations to the data set and suggest further benchmarking activities.

review the Paper and Project. All credit for this research goes to the researchers of this project. Also, don’t forget to join our 18k+ ML SubReddit, discord channeland electronic newsletterwhere we share the latest AI research news, exciting AI projects, and more.

🚀 Check out 100 AI tools at AI Tools Club