The release of RFM-1, the first commercial Robotics Foundation Model, set a new horizon for AI by giving robots the human-like ability to use language and reason about the physical world. Leveraging a vast real-world robotics dataset from Covariant robots operating in warehouses around the world, RFM-1 is a multimodal any-to-any sequence model that enables robots to adapt and handle unexpected or new scenarios seamlessly.

Learning and adapting on the fly

Learning and adapting on the fly

With the latest scaling update of RFM-1, robots can now learn how to adapt to their environment and improve on the fly when presented with a physical limitation by reflecting upon their most recent actions and the outcomes of those actions. This goes beyond relying on a model trained on a dataset because now robots can reason about their actions. Using this reasoning, robots can, for example, come up with a different picking strategy if recent attempts at picking an item fail.

Robot learning from its own actions and coming up with new strategies.

The above video shows a scenario in which the robot is asked to pick brand new items it has never seen before — seasonal promotional socks. It attempts to pick up the socks by grasping on the fabric a few times and fails. RFM-1 enables robots to learn from their most recent actions and outcomes. The right side of the video shows the robot doing this — having an internal dialogue, hypothesizing that its current gripper is not suited for fabric, and then coming up with a new strategy to pick up the socks by the paper label instead.

This degree of reasoning can help robots react to unforeseen situations that arise in real-world settings. Other such examples can include a robot having to adjust behavior due to damaged products, SKUs with manufacturing issues, never-seen-before seasonal items with very complicated packaging or shape, and much more.

RFM-1’s emergent in-context reasoning capabilities enabled the robot to learn and improve based on self-reflection — drastically increasing performance and reducing ramp time for a new system, scenario, or item.

Continuous innovation: Scaling up and pushing the boundaries of AI and robotics

Continuous innovation: Scaling up and pushing the boundaries of AI and robotics

Our growing datasets from real-world robots allow us to continue to scale up RFM-1 for additional capabilities. Leveraging this dataset, the Covariant AI research team continues to conduct active research to expand the capabilities of robots in real-world applications, helping customers further reduce their operational costs and increase the performance of Covariant robots in their warehouses.

Interested to learn how RFM-1 can increase the reliability and flexibility of your fulfillment network? Contact us.

If you would like to join our world-leading team that is shaping the future of AI and robotics, take a look at our open positions.

Related RFM-1: Allowing robots and people to communicate in natural language