Transfer simulation-trained AI to real-world robotic platforms effectively
Training AI in simulation offers efficiency and safety advantages, but deploying that intelligence on physical platforms introduces challenges that can undermine performance. Artificial Intelligence: From Simulation to Reality addresses this critical gap directly. Compiled by researchers from DARPA, Johns Hopkins, Penn, and Oregon State, this volume provides the methodologies needed to successfully transfer simulated learning to real-world autonomous systems.
The book covers diverse simulation environments, AI techniques for sim-to-real transfer, and a variety of exciting and relevant application domains, including autonomous vehicle drifting, bipedal locomotion, control of humanoid robots, human-in-the loop robotics, quadruped autonomy, and superhuman drone racing. This book also presents a modern treatment of classical concepts in robotics, including how large language models and vision-language-action model training techniques can be adapted to train robots in simulation for real-world transfer. Each chapter addresses specific sim-to-real challenges with proven solutions.
Readers will also explore:
- Coverage of multiple simulation platforms, environments, and techniques enabling practitioners to select the right tools for their specific robotics applications
- Domain randomization and system dynamics techniques that improve the robustness of AI models when transitioning from simulated to physical environments
- Machine learning methods for predicting robot trajectories that account for real-world uncertainties absent from idealized simulation training scenarios
- Techniques adapted from large language model development showing how transformer-based approaches can enhance sim-to-real transfer for autonomous robots
- Practical guidance on addressing the quintessential challenges that arise when deploying simulation-trained intelligence on real autonomous platforms
Research scientists, applied scientists, and engineers working in AI, machine learning, or robotics will find this an authoritative resource for sim-to-real transfer. Professors teaching robotics, transfer learning, reinforcement learning, or AI for control courses will find material suitable for advanced undergraduate and graduate curricula.
Alvaro Velasquez, PhD, is with the University of Colorado Boulder. In his former position as program manager at the Defense Advanced Research Projects Agency (DARPA), Alvaro secured and led $200 million in programs on efficient neurosymbolic AI and robust autonomy. He also served as the technical lead for AI at the Air Force Research Laboratory (AFRL). Alvaro's research has received best paper awards from the flagship conference of the Association for the Advancement of Artificial Intelligence (AAAI), the IEEE Computational Intelligence Society (CIS), and AFRL.
Vishal Patel, PhD, is an Associate Professor of Electrical and Computer Engineering at Johns Hopkins University and member of the Vision and Image Understanding Lab. He serves as Associate Editor of IEEE Transactions on Pattern Analysis and Machine Intelligence and on the IEEE Signal Processing Society's MLSP Committee.
Antonio Loquercio, PhD, is an Assistant Professor at the University of Pennsylvania. His research interests include learning-based robotics and computer vision. His work includes seminal results on simulation-to-real-world transfer in sensorimotor control. He is the recipient of several awards (2017 ETH Medal, the 2022 Georges Giralt PhD Award, and the 2025 ISNAFF Mario Gerla Award). Additionally, he has won several awards for his publications (2018 CORL Best Systems Paper, 2020 RSS Best Paper Honorable Mention, and the 2020 T-RO Best Paper Honorable Mention). His article on superhuman drone racing was featured on the cover of Nature.
Alan Fern, PhD, is a Professor at Oregon State University. His research interests span a variety of topics with a particular emphasis on building systems that can learn from experience. He directs the Dynamic Robotics and AI Laboratory (DRAIL), which studies AI for enabling humanoid robots to perform real-world work. He is a fellow of the Association for the Advancement of AI (AAAI) and Associate Editor for the Journal of Artificial Intelligence Research.
