Skip to main content

NORD: NVIDIA Omniverse Robot Definition

Overview​

NVIDIA Omniverse Robot Definition (NORD) represents a comprehensive framework for defining, simulating, and deploying robotic systems within the NVIDIA Omniverse platform. NORD bridges the gap between robot design and simulation, providing a standardized approach to create digital twins of robotic systems that can be used for development, testing, and validation before physical deployment.

Architecture and Components​

NORD Framework Structure​

NORD is built on several key components that work together to create comprehensive robot definitions:

┌─────────────────────────────────────────┐
│              NORD Framework             │
├─────────────────────────────────────────┤
│  ┌───────────────────────────────────┐  │
│  │    Robot Description Format       │  │
│  │  (Extended URDF/SDF Integration)  │  │
│  └───────────────────────────────────┘  │
├─────────────────────────────────────────┤
│  ┌─────────────────┐ ┌─────────────────┐│
│  │   Physics       │ │   Materials     ││
│  │   Simulation    │ │   Definition    ││
│  └─────────────────┘ └─────────────────┘│
├─────────────────────────────────────────┤
│  ┌─────────────────┐ ┌─────────────────┐│
│  │   Sensor        │ │   Actuator      ││
│  │   Modeling      │ │   Integration   ││
│  └─────────────────┘ └─────────────────┘│
├─────────────────────────────────────────┤
│  ┌───────────────────────────────────┐  │
│  │    Omniverse Integration Layer    │  │
│  └───────────────────────────────────┘  │
└─────────────────────────────────────────┘

Core Components​

1. Extended Robot Description​

NORD extends traditional robot description formats with Omniverse-specific capabilities:

{
  "robot": {
    "name": "example_robot",
    "version": "1.0",
    "description": "Example robot for NORD demonstration",
    "links": [
      {
        "name": "base_link",
        "visual": {
          "mesh": "package://robot_description/meshes/base_link.stl",
          "material": "metal_chrome"
        },
        "collision": {
          "mesh": "package://robot_description/meshes/base_link_collision.stl"
        },
        "inertial": {
          "mass": 10.0,
          "inertia": {
            "ixx": 0.4,
            "ixy": 0.0,
            "ixz": 0.0,
            "iyy": 0.4,
            "iyz": 0.0,
            "izz": 0.2
          }
        }
      }
    ],
    "joints": [
      {
        "name": "joint1",
        "type": "revolute",
        "parent": "base_link",
        "child": "link1",
        "origin": {
          "xyz": [0.0, 0.0, 0.1],
          "rpy": [0.0, 0.0, 0.0]
        },
        "axis": [0, 0, 1],
        "limits": {
          "lower": -3.14,
          "upper": 3.14,
          "effort": 100,
          "velocity": 1.0
        }
      }
    ]
  },
  "omniverse": {
    "articulation": {
      "enable_self_collision": true,
      "fix_root_link": false
    },
    "materials": {
      "metal_chrome": {
        "albedo": [0.8, 0.8, 0.9],
        "metallic": 0.9,
        "roughness": 0.1,
        "specular": 0.5
      }
    },
    "sensors": [
      {
        "name": "camera_sensor",
        "type": "camera",
        "parent_link": "sensor_mount",
        "position": [0.1, 0.0, 0.05],
        "orientation": [0.0, 0.0, 0.0, 1.0],
        "parameters": {
          "resolution": [640, 480],
          "fov": 1.047,
          "clipping_range": [0.1, 100.0]
        }
      }
    ]
  }
}

2. Physics Simulation Integration​

NORD provides advanced physics simulation capabilities:

class NORDPhysicsIntegrator:
    def __init__(self, robot_config):
        self.robot = self.load_robot(robot_config)
        self.physics_scene = self.create_physics_scene()
        self.contact_manager = self.setup_contact_manager()

    def load_robot(self, config):
        """Load robot with NORD extensions"""
        # Load base URDF/SDF
        robot = self.load_base_description(config['urdf_path'])

        # Apply NORD-specific physics properties
        for link in robot.links:
            self.apply_material_properties(link, config['materials'])
            self.setup_collision_properties(link, config['collision'])

        return robot

    def create_physics_scene(self):
        """Create physics scene with Omniverse integration"""
        scene = PhysicsScene(
            gravity=[0, 0, -9.81],
            solver_type='tgs',  # Time-stepping Gauss-Seidel
            num_position_iterations=4,
            num_velocity_iterations=1
        )

        # Add NORD-specific constraints
        scene.set_contact_offset(0.001)
        scene.set_rest_offset(0.0)

        return scene

    def setup_contact_manager(self):
        """Setup advanced contact handling"""
        contact_manager = ContactManager()

        # Register contact callbacks for NORD-specific behaviors
        contact_manager.register_callback(
            'robot_environment',
            self.handle_robot_environment_contact
        )
        contact_manager.register_callback(
            'robot_robot',
            self.handle_robot_robot_contact
        )

        return contact_manager

3. Material and Appearance System​

NORD includes advanced material definitions for photorealistic rendering:

class NORDMaterialSystem:
    def __init__(self):
        self.material_library = {}
        self.pbr_properties = {}

    def define_material(self, name, properties):
        """Define material with Physically Based Rendering properties"""
        material = {
            'name': name,
            'albedo': properties.get('albedo', [0.8, 0.8, 0.8]),
            'metallic': properties.get('metallic', 0.0),
            'roughness': properties.get('roughness', 0.5),
            'normal_map': properties.get('normal_map'),
            'occlusion_map': properties.get('occlusion_map'),
            'emissive': properties.get('emissive', [0.0, 0.0, 0.0]),
            'transmission': properties.get('transmission', 0.0),
            'ior': properties.get('ior', 1.5)
        }

        self.material_library[name] = material
        return material

    def apply_material_to_link(self, link, material_name):
        """Apply material to robot link"""
        if material_name in self.material_library:
            material = self.material_library[material_name]
            # Apply to visual mesh in Omniverse
            self.set_visual_material(link.visual_mesh, material)
            return True
        return False

NORD Integration with Omniverse​

USD (Universal Scene Description)​

NORD leverages USD for scene representation:

from pxr import Usd, UsdGeom, Sdf

class NORDUSDExporter:
    def __init__(self, stage_path):
        self.stage = Usd.Stage.CreateNew(stage_path)
        self.root_prim = self.stage.GetPseudoRoot()

    def export_robot(self, robot_config):
        """Export robot to USD format"""
        robot_prim = UsdGeom.Xform.Define(self.stage, '/Robot')

        # Export links as Xform prims
        for link_config in robot_config['links']:
            link_prim = UsdGeom.Xform.Define(
                self.stage,
                f'/Robot/{link_config["name"]}'
            )

            # Set transform
            xform_api = UsdGeom.XformCommonAPI(link_prim)
            xform_api.SetTranslate(link_config['origin']['xyz'])

            # Add mesh
            if 'mesh' in link_config['visual']:
                mesh_prim = UsdGeom.Mesh.Define(
                    self.stage,
                    f'/Robot/{link_config["name"]}/visual'
                )
                self.set_mesh_properties(mesh_prim, link_config['visual'])

        # Export joints as articulation joints
        self.export_joints(robot_config['joints'])

        self.stage.GetRootLayer().Save()

    def export_joints(self, joints):
        """Export joints with articulation properties"""
        for joint_config in joints:
            joint_prim = UsdGeom.Xform.Define(
                self.stage,
                f'/Robot/Joints/{joint_config["name"]}'
            )

            # Apply articulation joint properties
            articulation_joint = PhysxSchema.Joint.Define(
                self.stage,
                joint_prim.GetPath()
            )
            self.set_joint_properties(articulation_joint, joint_config)

Sensor Integration​

NORD provides comprehensive sensor modeling:

class NORDSensorIntegrator:
    def __init__(self):
        self.sensor_types = {
            'camera': self.create_camera_sensor,
            'lidar': self.create_lidar_sensor,
            'imu': self.create_imu_sensor,
            'force_torque': self.create_force_torque_sensor
        }

    def create_camera_sensor(self, config):
        """Create Omniverse-compatible camera sensor"""
        camera = omni.replicator.core.Camera()
        camera.set_resolution(config['parameters']['resolution'])
        camera.set_fov(config['parameters']['fov'])
        camera.set_clipping_range(config['parameters']['clipping_range'])

        return {
            'sensor': camera,
            'config': config,
            'data_provider': omni.replicator.core.RandomIntrinsicsCameraProvider()
        }

    def create_lidar_sensor(self, config):
        """Create Omniverse-compatible LIDAR sensor"""
        lidar = omni.isaac.range_sensor.acquire_lidar_sensor_interface()

        return {
            'sensor': lidar,
            'config': config,
            'spec': {
                'min_range': config['parameters'].get('min_range', 0.1),
                'max_range': config['parameters'].get('max_range', 10.0),
                'rotation_frequency': config['parameters'].get('rotation_frequency', 10),
                'channels': config['parameters'].get('channels', 16)
            }
        }

    def integrate_sensors(self, robot_prim, sensor_configs):
        """Integrate sensors into robot structure"""
        for sensor_config in sensor_configs:
            sensor_type = sensor_config['type']
            if sensor_type in self.sensor_types:
                sensor_instance = self.sensor_types[sensor_type](sensor_config)

                # Attach sensor to parent link
                parent_link_path = f'/Robot/{sensor_config["parent_link"]}'
                sensor_prim = self.stage.DefinePrim(
                    f'{parent_link_path}/{sensor_config["name"]}',
                    'Xform'
                )

                # Set sensor transform
                transform_api = UsdGeom.XformCommonAPI(sensor_prim)
                transform_api.SetTranslate(sensor_config['position'])
                transform_api.SetRotate Orient(sensor_config['orientation'])

NORD Replay System​

The NORD Replay system enables recording and playback of robot simulation data for analysis and debugging.

Data Recording​

class NORDReplayRecorder:
    def __init__(self, robot, output_path):
        self.robot = robot
        self.output_path = output_path
        self.recording_data = {
            'timestamps': [],
            'joint_states': [],
            'sensor_data': [],
            'environment_state': [],
            'control_commands': []
        }
        self.is_recording = False

    def start_recording(self):
        """Start recording robot simulation data"""
        self.is_recording = True
        self.start_time = time.time()
        self.record_initial_state()

    def record_frame(self):
        """Record current simulation frame"""
        if not self.is_recording:
            return

        current_time = time.time() - self.start_time
        self.recording_data['timestamps'].append(current_time)

        # Record joint states
        joint_states = {}
        for joint in self.robot.joints:
            joint_states[joint.name] = {
                'position': joint.get_position(),
                'velocity': joint.get_velocity(),
                'effort': joint.get_effort()
            }
        self.recording_data['joint_states'].append(joint_states)

        # Record sensor data
        sensor_data = {}
        for sensor_name, sensor in self.robot.sensors.items():
            sensor_data[sensor_name] = sensor.get_data()
        self.recording_data['sensor_data'].append(sensor_data)

    def stop_recording(self):
        """Stop recording and save data"""
        self.is_recording = False

        # Save recording data to file
        with open(self.output_path, 'wb') as f:
            pickle.dump(self.recording_data, f)

        print(f"Recording saved to {self.output_path}")

Data Playback​

class NORDReplayPlayer:
    def __init__(self, recording_path):
        self.recording_path = recording_path
        self.recording_data = self.load_recording()
        self.current_frame = 0
        self.playback_speed = 1.0

    def load_recording(self):
        """Load recording data from file"""
        with open(self.recording_path, 'rb') as f:
            return pickle.load(f)

    def play_frame(self, target_robot):
        """Play back a single frame of recording"""
        if self.current_frame >= len(self.recording_data['timestamps']):
            return False  # End of recording

        # Apply joint states
        joint_states = self.recording_data['joint_states'][self.current_frame]
        for joint_name, state in joint_states.items():
            target_robot.set_joint_position(joint_name, state['position'])
            target_robot.set_joint_velocity(joint_name, state['velocity'])

        # Apply sensor data (for visualization/analysis)
        sensor_data = self.recording_data['sensor_data'][self.current_frame]

        self.current_frame += 1
        return True

    def play_sequence(self, target_robot, callback=None):
        """Play back entire sequence"""
        self.current_frame = 0

        while self.current_frame < len(self.recording_data['timestamps']):
            success = self.play_frame(target_robot)
            if not success:
                break

            if callback:
                callback(self.current_frame, len(self.recording_data['timestamps']))

            time.sleep(0.01 / self.playback_speed)  # Simulate real-time playback

Best Practices for NORD Implementation​

Performance Optimization​

  • Use level-of-detail (LOD) models for complex robots
  • Optimize mesh resolution based on use case
  • Implement efficient collision geometries
  • Use instancing for repeated components

Quality Assurance​

  • Validate robot kinematics before simulation
  • Test sensor configurations in isolation
  • Verify physics parameters with real-world data
  • Perform stress testing with extreme inputs

Integration Guidelines​

  • Maintain compatibility with standard ROS/ROS2 interfaces
  • Use standard coordinate frames and units
  • Implement proper error handling and logging
  • Document custom extensions clearly

NORD provides a powerful framework for creating detailed robot definitions that integrate seamlessly with NVIDIA Omniverse, enabling sophisticated simulation and digital twin capabilities for physical AI applications.