diff --git a/README.rst b/README.rst index 3cc5f81293..9a54b1a5f4 100644 --- a/README.rst +++ b/README.rst @@ -13,6 +13,7 @@ Transformer Engine Latest News =========== +* [12/2025] `NVIDIA Nemotron 3: Efficient and Open Intelligence `_ - trained with NVFP4 on Transformer Engine * [11/2025] `NVIDIA Blackwell Architecture Sweeps MLPerf Training v5.1 Benchmarks `_ * [11/2025] `Scale Biology Transformer Models with PyTorch and NVIDIA BioNeMo Recipes `_ * [11/2025] `FP8 Training of Large-Scale RL Models `_ @@ -30,7 +31,8 @@ What is Transformer Engine? Transformer Engine (TE) is a library for accelerating Transformer models on NVIDIA GPUs, including using 8-bit floating point (FP8) precision on Hopper, Ada, and Blackwell GPUs, to provide better -performance with lower memory utilization in both training and inference. TE provides a collection +performance with lower memory utilization in both training and inference. On Blackwell GPUs, TE also +supports MXFP8 (Microscaling FP8) and NVFP4 formats for even greater efficiency. TE provides a collection of highly optimized building blocks for popular Transformer architectures and an automatic mixed precision-like API that can be used seamlessly with your framework-specific code. TE also includes a framework agnostic C++ API that can be integrated with other deep learning libraries to enable FP8 @@ -58,6 +60,7 @@ Highlights * Easy-to-use modules for building Transformer layers with FP8 support * Optimizations (e.g. fused kernels) for Transformer models * Support for FP8 on NVIDIA Hopper, Ada, and Blackwell GPUs +* Support for MXFP8 and NVFP4 on NVIDIA Blackwell GPUs * Support for optimizations across all precisions (FP16, BF16) on NVIDIA Ampere GPU architecture generations and later Examples @@ -190,7 +193,7 @@ We recommend updating to the latest NGC container available here: * https://catalog.ngc.nvidia.com/orgs/nvidia/containers/pytorch * https://catalog.ngc.nvidia.com/orgs/nvidia/containers/jax -If you run any examples, please ensure you are using a matching version of TransformerEngine. TransformerEngine is pre-built and packaged inside the containers with examples available at ``/opt/transformerengine`` or ``/opt/transformer-engine``. If you would like to use examples from TE main branch and are running into import errors, please try the latest pip package or building from source, although NGC containers are recommended for ease-of-use for most users. +If you run any examples, please ensure you are using a matching version of TransformerEngine. TransformerEngine is pre-built and packaged inside the containers with examples available at ``/opt/transformerengine`` or ``/opt/transformer-engine``. **Benefits of using NGC containers:** @@ -381,35 +384,27 @@ FP8 has been tested extensively across different model architectures and configu +------------+------------------+---------------------------------------------------------------------------------------------------------+ | Model | Framework | Source | +============+==================+=========================================================================================================+ -| T5-770M | JAX/T5x | https://github.com/NVIDIA/JAX-Toolbox/tree/main/rosetta/rosetta/projects/t5x#convergence-and-performance| -+------------+------------------+---------------------------------------------------------------------------------------------------------+ | MPT-1.3B | Mosaic Composer | https://www.mosaicml.com/blog/coreweave-nvidia-h100-part-1 | +------------+------------------+---------------------------------------------------------------------------------------------------------+ -| GPT-5B | JAX/Paxml | https://github.com/NVIDIA/JAX-Toolbox/tree/main/rosetta/rosetta/projects/pax#h100-results | -+------------+------------------+---------------------------------------------------------------------------------------------------------+ -| GPT-5B | NeMo Framework | Available on request | -+------------+------------------+---------------------------------------------------------------------------------------------------------+ | LLama2-7B | Alibaba Pai | https://mp.weixin.qq.com/s/NQT0uKXLbXyh5031zBdeBQ | +------------+------------------+---------------------------------------------------------------------------------------------------------+ -| T5-11B | JAX/T5x | Available on request | +| LLM-8B | Megatron Core | https://arxiv.org/abs/2506.08027 | +------------+------------------+---------------------------------------------------------------------------------------------------------+ | MPT-13B | Mosaic Composer | https://www.databricks.com/blog/turbocharged-training-optimizing-databricks-mosaic-ai-stack-fp8 | +------------+------------------+---------------------------------------------------------------------------------------------------------+ -| GPT-22B | NeMo Framework | Available on request | +| MoE-16B | Megatron Core | https://arxiv.org/abs/2506.08027 | +------------+------------------+---------------------------------------------------------------------------------------------------------+ | LLama2-70B | Alibaba Pai | https://mp.weixin.qq.com/s/NQT0uKXLbXyh5031zBdeBQ | +------------+------------------+---------------------------------------------------------------------------------------------------------+ -| GPT-175B | JAX/Paxml | https://github.com/NVIDIA/JAX-Toolbox/tree/main/rosetta/rosetta/projects/pax#h100-results | -+------------+------------------+---------------------------------------------------------------------------------------------------------+ Integrations ============ Transformer Engine has been integrated with popular LLM frameworks such as: -* `DeepSpeed `_ +* `DeepSpeed `_ * `Hugging Face Accelerate `_ -* `Lightning `_ +* `Lightning `_ * `MosaicML Composer `_ * `NVIDIA JAX Toolbox `_ * `NVIDIA Megatron-LM `_ @@ -417,10 +412,7 @@ Transformer Engine has been integrated with popular LLM frameworks such as: * `Amazon SageMaker Model Parallel Library `_ * `Levanter `_ * `GPT-NeoX `_ -* `Hugging Face Nanotron `_ - Coming soon! -* `Colossal-AI `_ - Coming soon! -* `PeriFlow `_ - Coming soon! - +* `Hugging Face Nanotron `_ Contributing ============