Skip to content

README.md

Latest commit

 

History

History
201 lines (142 loc) · 10.1 KB

README.md

File metadata and controls

201 lines (142 loc) · 10.1 KB

Isaac Gym Benchmark Environments

Website | Technical Paper | Videos

About this repository

This repository contains example RL environments for the NVIDIA Isaac Gym high performance environments described in our NeurIPS 2021 Datasets and Benchmarks paper

Installation

Download the Isaac Gym Preview 4 release from the website, then follow the installation instructions in the documentation. We highly recommend using a conda environment to simplify set up. Please ensure that the conda environment is named isaacgym during this setup.

Ensure that Isaac Gym works on your system by running one of the examples from the python/examples directory, like joint_monkey.py. Follow troubleshooting steps described in the Isaac Gym Preview 4 install instructions if you have any trouble running the samples.

Once Isaac Gym is installed and samples work within your current python environment, install this repo:

pip install -e .
pip install -r requirements.txt

Creating an environment

We offer an easy-to-use API for creating preset vectorized environments. For more info on what a vectorized environment is and its usage, please refer to the Gym library documentation.

import isaacgym
import isaacgymenvs
import torch

# number of parallel environments
num_envs = 64

# create a FrankaCabinet environment
envs = isaacgymenvs.make(
    seed=0,
    task="FrankaCabinet",
    num_envs=num_envs,
    sim_device="cuda:0",
    rl_device="cuda:0",
)
print("Observation space is", envs.observation_space)
print("Action space is", envs.action_space)
obs = envs.reset()
for _ in range(20):
    random_actions = 2.0 * torch.rand((num_envs,) + envs.action_space.shape, device='cuda:0') - 1.0
    envs.step(random_actions)

Running the benchmarks

To train your first policy, run this line:

# train a cabinet manipulation policy from scratch
python train.py task=FrankaCabinet

This command trains the Franka arm to open and close a cabinet drawer. Training should converge within a few minutes on a modern GPU.

To avoid the overhead of rendering, you can disable the viewer:

python train.py task=FrankaCabinet headless=True

When running headlessly, you can stop training at any time with Ctrl+C.

To run the inference‑only grasping task, use:

python train.py task=GraspTaskPick test=True headless=True num_envs=64

Loading trained models // Checkpoints

Checkpoints are saved in the folder runs/EXPERIMENT_NAME/nn where EXPERIMENT_NAME defaults to the task name, but can also be overridden via the experiment argument.

To load a trained checkpoint and continue training, use the checkpoint argument:

python train.py task=FrankaCabinet checkpoint=runs/FrankaCabinet/nn/FrankaCabinet.pth

To load a trained checkpoint and only perform inference (no training), pass test=True as an argument, along with the checkpoint name. To avoid rendering overhead, you may also want to run with fewer environments using num_envs=64:

python train.py task=FrankaCabinet checkpoint=runs/FrankaCabinet/nn/FrankaCabinet.pth test=True num_envs=64

Note that If there are special characters such as [ or = in the checkpoint names, you will need to escape them and put quotes around the string. For example, checkpoint="./runs/FrankaCabinet/nn/last_FrankaCabinet_ep\=501rew\[5981.31\].pth"

Configuration and command line arguments

We use Hydra to manage the config. Note that this has some differences from previous incarnations in older versions of Isaac Gym.

Key arguments to the train.py script are:

  • task=TASK – selects which task to use. Supported tasks in this repository are FactoryTaskNutBoltPick, FactoryTaskNutBoltPlace, FactoryTaskNutBoltScrew, FactoryTaskInsertion, FactoryTaskGears, GraspTaskPick, IndustRealTaskPegsInsert, and FrankaCabinet (these correspond to the config files in isaacgymenvs/cfg/task).
  • train=TRAIN - selects which training config to use. Will automatically default to the correct config for the environment (ie. <TASK>PPO).
  • num_envs=NUM_ENVS - selects the number of environments to use (overriding the default number of environments set in the task config).
  • seed=SEED - sets a seed value for randomizations, and overrides the default seed set up in the task config
  • sim_device=SIM_DEVICE_TYPE - Device used for physics simulation. Set to cuda:0 (default) to use GPU and to cpu for CPU. Follows PyTorch-like device syntax.
  • rl_device=RL_DEVICE - Which device / ID to use for the RL algorithm. Defaults to cuda:0, and also follows PyTorch-like device syntax.
  • graphics_device_id=GRAPHICS_DEVICE_ID - Which Vulkan graphics device ID to use for rendering. Defaults to 0. Note - this may be different from CUDA device ID, and does not follow PyTorch-like device syntax.
  • pipeline=PIPELINE - Which API pipeline to use. Defaults to gpu, can also set to cpu. When using the gpu pipeline, all data stays on the GPU and everything runs as fast as possible. When using the cpu pipeline, simulation can run on either CPU or GPU, depending on the sim_device setting, but a copy of the data is always made on the CPU at every step.
  • test=TEST- If set to True, only runs inference on the policy and does not do any training.
  • checkpoint=CHECKPOINT_PATH - Set to path to the checkpoint to load for training or testing.
  • headless=HEADLESS - Whether to run in headless mode.
  • experiment=EXPERIMENT - Sets the name of the experiment.
  • max_iterations=MAX_ITERATIONS - Sets how many iterations to run for. Reasonable defaults are provided for the provided environments.

Hydra also allows setting variables inside config files directly as command line arguments. As an example, to set the discount rate for a rl_games training run, you can use train.params.config.gamma=0.999. Similarly, variables in task configs can also be set. For example, task.env.enableDebugVis=True.

Hydra Notes

Default values for each of these are found in the isaacgymenvs/config/config.yaml file.

The way that the task and train portions of the config works are through the use of config groups. You can learn more about how these work here The actual configs for task are in isaacgymenvs/config/task/<TASK>.yaml and for train in isaacgymenvs/config/train/<TASK>PPO.yaml.

In some places in the config you will find other variables referenced (for example, num_actors: ${....task.env.numEnvs}). Each . represents going one level up in the config hierarchy. This is documented fully here.

Tasks

Source code for tasks can be found in isaacgymenvs/tasks.

This repository includes the following tasks (each of which subclasses VecEnv in isaacgymenvs/tasks/base/vec_task.py):

  • GraspTaskPick – inference‑only grasping task using Contact GraspNet.
  • IndustRealTaskPegsInsert – peg insertion task from IndustRealSim.
  • FrankaCabinet – open and close a cabinet drawer with a Franka arm.

See docs/framework.md for guidance on implementing custom tasks.

WandB support

You can run WandB with Isaac Gym Envs by setting wandb_activate=True flag from the command line. You can set the group, name, entity, and project for the run by setting the wandb_group, wandb_name, wandb_entity and wandb_project set. Make sure you have WandB installed with pip install wandb before activating.

Troubleshooting

Please review the Isaac Gym installation instructions first if you run into any issues.

You can either submit issues through GitHub or through the Isaac Gym forum here.

FAQ

Q: How do I set up this repository for GPUs with compute capability sm_86 or greater (e.g., RTX 30-series, A5000)?
A: The PyTorch version shipped with Isaac Gym’s default environment may not support these newer GPU architectures. Replace python/rlgpu_conda_env.yml with the YAML we provide here: docs/rlgpu_conda_env.yml, then recreate the environment from scratch using exactly the same steps you originally followed to create the Isaac Gym environment. We intentionally do not provide CLI commands for this step.

Citing

Please cite this work as:

@misc{makoviychuk2021isaac,
      title={Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learning}, 
      author={Viktor Makoviychuk and Lukasz Wawrzyniak and Yunrong Guo and Michelle Lu and Kier Storey and Miles Macklin and David Hoeller and Nikita Rudin and Arthur Allshire and Ankur Handa and Gavriel State},
      year={2021},
      journal={arXiv preprint arXiv:2108.10470}
}

Note if you use the Factory simulation methods (e.g., SDF collisions, contact reduction) or Factory learning tools (e.g., assets, environments, or controllers) in your work, please cite the following paper:

@inproceedings{
	narang2022factory,
	author = {Yashraj Narang and Kier Storey and Iretiayo Akinola and Miles Macklin and Philipp Reist and Lukasz Wawrzyniak and Yunrong Guo and Adam Moravanszky and Gavriel State and Michelle Lu and Ankur Handa and Dieter Fox},
	title = {Factory: Fast contact for robotic assembly},
	booktitle = {Robotics: Science and Systems},
	year = {2022}
}

Note if you use the IndustReal training environments or algorithms in your work, please cite the following paper:

@inproceedings{
	tang2023industreal,
	author = {Bingjie Tang and Michael A Lin and Iretiayo Akinola and Ankur Handa and Gaurav S Sukhatme and Fabio Ramos and Dieter Fox and Yashraj Narang},
	title = {IndustReal: Transferring contact-rich assembly tasks from simulation to reality},
	booktitle = {Robotics: Science and Systems},
	year = {2023}
}