Sim-to-Real Transfer in Production: Domain Randomization & Policy Deployment with MuJoCo & ROS 2

1. The Anatomy of the Sim-to-Real Gap

In simulation, a neural policy can collect 100,000 hours of robotic experience overnight across thousands of parallel GPU environments in NVIDIA Isaac Sim or MuJoCo. Yet when that identical checkpoint is flashed onto physical hardware, it often violently oscillates, stumbles, or collapses.

This failure is driven by three distinct gaps:

• The Visual Gap: Differences in real-world ambient lighting, surface reflections, shadow occlusions, rolling shutter artifacts, and camera lens distortions compared to synthetic rasterizers.
• The Dynamics Gap: Unmodeled physics: cable drag, non-linear joint stiction, gearbox backlash, thermal motor weakening, compliance in 3D-printed brackets, and non-rigid ground interactions.
• The Latency & Discretization Gap: Simulation runs in lockstep with zero clock jitter. Real robots experience asynchronous packet arrivals over Ethernet/CAN bus, OS context switching, sensor exposure latencies, and motor controller delays.

2. Mathematical Formulation of Domain Randomization (DR)

Rather than attempting to construct a microscopically perfect simulation of one specific physical robot, Domain Randomization models physics parameters \(\xi\) as random variables drawn from a wide distribution \(P(\Xi)\):

If the randomized simulation distribution \(P(\Xi)\) is broad enough to enclose the real world's actual physical parameters (\(\xi_{real} \in \text{support}(P(\Xi))\)), the real robot appears to the policy simply as another random simulation rollout.

3. Dynamics Randomization Parameter Ranges

Production teams at Boston Dynamics, Unitree, and Tesla utilize rigorous randomization ranges during Isaac Lab training runs:

4. Avoiding the Over-Randomization Trap

A common novice mistake in Sim-to-Real is randomizing parameters too aggressively (e.g. \(\pm 80\%\) mass variation or extreme friction swings).

When faced with physically impossible parameter extremes, the neural network learns an ultra-conservative policy: it locks its knees, takes microscopic 2-centimeter shuffling steps, or refuses to swing its arm with momentum. To maintain agile, dynamic behavior, teams combine narrow Domain Randomization with System Identification (SysID):

5. Residual Reinforcement Learning

Instead of forcing a neural network to learn physics from scratch, production systems often adopt Residual RL. A classical, verified Model Predictive Controller (MPC) handles macroscopic balance and trajectory tracking, while a neural policy outputs fine corrective torques:

If the neural network encounters an unexpected observation, \(\Delta\tau_{neural}\) can be clipped to \(\pm 5\text{ Nm}\) by a hardware supervisor, ensuring the robot never destabilizes completely.

6. Production ROS 2 Real-Time Deployment Pipeline

Once trained in PyTorch, the policy is compiled to an ONNX Runtime / TensorRT C++ engine and executed inside a real-time ROS 2 node graph:

// C++ ROS 2 Node executing learned policy with PREEMPT_RT deterministic timer
#include <rclcpp/rclcpp.hpp>
#include <sensor_msgs/msg/joint_state.hpp>
#include <trajectory_msgs/msg/joint_trajectory.hpp>
#include <onnxruntime_cxx_api.h>

class RealTimePolicyNode : public rclcpp::Node {
public:
    RealTimePolicyNode() : Node("rt_policy_node") {
        // Enforce 100 Hz deterministic execution loop
        timer_ = this->create_wall_timer(
            std::chrono::milliseconds(10),
            std::bind(&RealTimePolicyNode::control_step, this));
            
        joint_sub_ = this->create_subscription<sensor_msgs::msg::JointState>(
            "/joint_states", 1, 
            std::bind(&RealTimePolicyNode::joint_callback, this, std::placeholders::_1));
            
        torque_pub_ = this->create_publisher<trajectory_msgs::msg::JointTrajectory>(
            "/joint_torque_commands", 1);
    }

private:
    void control_step() {
        // 1. Pack observation vector (joint positions, velocities, past actions)
        auto obs_tensor = build_observation_tensor();

        // 2. Evaluate TensorRT / ONNX engine (deterministic < 2.5 ms)
        auto action_tensor = policy_session_->Run(obs_tensor);

        // 3. Safety Watchdog Filter: Clamp torques & check thermal limits
        auto safe_torques = enforce_hardware_safety(action_tensor);

        // 4. Dispatch over CAN bus / EtherCAT to motor controllers
        publish_commands(safe_torques);
    }
};

7. Pre-Flight Hardware Deployment Checklist

1. Zero Joint Torque Test (Gravity Off): Hang the robot from an overhead gantry; execute the policy in air to verify joint coordinate frame conventions match simulation signs.
2. Watchdog Heartbeat: Verify that if the policy node drops a frame or takes > 15 ms, the motor controllers immediately switch to passive damping (\(K_d\) only).
3. Thermal Logging: Monitor winding temperatures across a continuous 30-minute stress test; ensure stator coils stay below \(90^\circ\text{C}\).
4. Sim-to-Real Tracking Variance: Log joint trajectories; ensure physical Root-Mean-Square Error (RMSE) against simulation is \(< 0.05\text{ rad}\).