Skip to content

Virtual worlds used to train advanced robots

Cambridge start-up Vsim has announced a significant advancement in accelerating robot training, stating its “Freddo” robot can pick up new abilities within minutes—far outpacing rival systems, which may require several days for a similar process.

Thanks to their proprietary simulator, which is optimized to take full advantage of high-powered computing hardware, Vsim is able to evaluate tens of thousands of hypothetical scenarios each second. Co-founders Kier Storey and Michelle Lu assert that this technology may be critical for developing robots that handle complex, dynamic situations in unpredictable, real-world settings.

Leveraging fast-paced virtual simulations for robot learning

Using Vsim’s advanced simulation platform, Freddo—the company’s prototype—has been taught to cross an office, identify a plastic bottle, and pick it up. All of these skills were developed and installed within a matter of minutes. Lu remarked, “Eighteen months in and we actually have a completely functional, super high-performance simulator.”

Central to this system is a virtual space in which robots can rehearse tasks millions of times. When the simulation finds an optimal strategy, that set of instructions is transferred directly to Freddo’s hardware. As Storey highlights, “While core robotics simulation algorithms originated in the 1970s and 1980s and weren’t designed with today’s GPUs in mind, starting from scratch allowed us to calibrate our code for modern AI-centric chips.”

Freddo’s system lets it forecast over 20,000 likely outcomes a second in advance, which is vital as the robot maneuvers through environments that may include people or animals. Lu emphasized, “Unexpected scenarios can occur rapidly, so robots must rapidly respond to ensure their behavior remains safe and aligned with the task.”

Bigger players and open-source developments

Across the robotics field, virtual simulation is on the rise. Industry leader Nvidia—recognized for its dominance in AI hardware—provides its own Isaac Sim and Cosmos world model platforms to facilitate robot education. Although these tools allow robots to interpret aspects of real-world physics, they are still imperfect. Spencer Huang, who directs product development for robotics at Nvidia, has noted, “Manipulation—where I grab a bottle—isn’t too hard. The challenge comes from long-horizon tasks: say, fetch the bottle, fill it, and pour it out.”

To expedite the traditionally tedious process of constructing virtual environments, Nvidia now utilizes AI agents. Huang observed, “We’re just throwing agents at it… it’s basically given us a huge workforce.”

Meanwhile, open-source tools are also attracting attention. Rika Antonova, an associate professor at the University of Cambridge, leverages MuJoCo—a robotics simulator made open-source following its 2021 acquisition by Google’s DeepMind. Its accessible nature is especially valuable for innovative start-ups and small labs, facilitating rapid test cycles.

Antonova expressed support for Vsim’s innovation: “If you have a very, very fast simulator, you can simulate hundreds of millions of samples in the few seconds your robot is thinking about how to adjust its motion, allowing real-time adaptation.” She also advised caution about the limitations of simulation technologies: “Tasks such as manipulating soft or deformable items, or executing precision operations like cutting, are tough to mimic virtually.”

Refining virtual to real-world transfer

Vsim is working to decrease the gap between simulation and reality by boosting the fidelity of their digital training environments. Lu reports that the company’s improvements have reduced inaccuracies, making it possible for robots to transition skills from the simulator directly into real-world tasks. As part of ongoing research, a second robot called Nacho will soon be introduced to help test and broaden the software’s effectiveness later this year.

Operating with a team of just 10 engineers, Vsim is competing against industry giants and research juggernauts in the quest to develop nimble and flexible service robots for homes and businesses. Should these simulation breakthroughs meet expectations, features such as assembling IKEA furniture or moving around crowded living spaces could soon be within easy reach for robots.