The release of RFM-1, the first commercial Robotics Foundation Model, set a new horizon for AI by giving robots the human-like ability to use language and reason about the physical world. Leveraging a vast real-world robotics dataset from Covariant robots operating in warehouses around the world, RFM-1 is a multimodal any-to-any sequence model that enables robots to adapt and handle unexpected or new scenarios seamlessly.
Learning and adapting on the fly
With the latest scaling update of RFM-1, robots can now learn how to adapt to their environment and improve on the fly when presented with a physical limitation by reflecting upon their most recent actions and the outcomes of those actions. This goes beyond relying on a model trained on a dataset because now robots can reason about their actions. Using this reasoning, robots can, for example, come up with a different picking strategy if recent attempts at picking an item fail.
The above video shows a scenario in which the robot is asked to pick brand new items it has never seen before — seasonal promotional socks. It attempts to pick up the socks by grasping on the fabric a few times and fails. RFM-1 enables robots to learn from their most recent actions and outcomes. The right side of the video shows the robot doing this — having an internal dialogue, hypothesizing that its current gripper is not suited for fabric, and then coming up with a new strategy to pick up the socks by the paper label instead.
This degree of reasoning can help robots react to unforeseen situations that arise in real-world settings. Other such examples can include a robot having to adjust behavior due to damaged products, SKUs with manufacturing issues, never-seen-before seasonal items with very complicated packaging or shape, and much more.
RFM-1’s emergent in-context reasoning capabilities enabled the robot to learn and improve based on self-reflection — drastically increasing performance and reducing ramp time for a new system, scenario, or item.
Continuous innovation: Scaling up and pushing the boundaries of AI and robotics
Our growing datasets from real-world robots allow us to continue to scale up RFM-1 for additional capabilities. Leveraging this dataset, the Covariant AI research team continues to conduct active research to expand the capabilities of robots in real-world applications, helping customers further reduce their operational costs and increase the performance of Covariant robots in their warehouses.
