From eb6bb884f110a01c2724e503474152df879ffe75 Mon Sep 17 00:00:00 2001 From: Santosh Bhavani Date: Sat, 31 Jan 2026 18:47:46 -0800 Subject: [PATCH 1/3] docs(readme): update FP8 convergence table and add MXFP8/NVFP4 support info - Add MXFP8 and NVFP4 format support to highlights and description - Update FP8 convergence table with MXFP8 results from arxiv paper - Remove outdated JAX-Toolbox links and "available on request" entries - Update Docker container versions to 26.01 - Fix DeepSpeed and Lightning integration links - Add Nemotron 3 paper to Latest News - Add quickstart notebook link after PyTorch example Signed-off-by: Santosh Bhavani --- README.rst | 29 +++++++++++------------------ 1 file changed, 11 insertions(+), 18 deletions(-) diff --git a/README.rst b/README.rst index 3cc5f81293..9a21c5d764 100644 --- a/README.rst +++ b/README.rst @@ -13,6 +13,7 @@ Transformer Engine Latest News =========== +* [12/2025] `NVIDIA Nemotron 3: Efficient and Open Intelligence `_ - trained with NVFP4 on Transformer Engine * [11/2025] `NVIDIA Blackwell Architecture Sweeps MLPerf Training v5.1 Benchmarks `_ * [11/2025] `Scale Biology Transformer Models with PyTorch and NVIDIA BioNeMo Recipes `_ * [11/2025] `FP8 Training of Large-Scale RL Models `_ @@ -30,7 +31,8 @@ What is Transformer Engine? Transformer Engine (TE) is a library for accelerating Transformer models on NVIDIA GPUs, including using 8-bit floating point (FP8) precision on Hopper, Ada, and Blackwell GPUs, to provide better -performance with lower memory utilization in both training and inference. TE provides a collection +performance with lower memory utilization in both training and inference. On Blackwell GPUs, TE also +supports MXFP8 (Microscaling FP8) and NVFP4 formats for even greater efficiency. TE provides a collection of highly optimized building blocks for popular Transformer architectures and an automatic mixed precision-like API that can be used seamlessly with your framework-specific code. TE also includes a framework agnostic C++ API that can be integrated with other deep learning libraries to enable FP8 @@ -58,6 +60,7 @@ Highlights * Easy-to-use modules for building Transformer layers with FP8 support * Optimizations (e.g. fused kernels) for Transformer models * Support for FP8 on NVIDIA Hopper, Ada, and Blackwell GPUs +* Support for MXFP8 and NVFP4 on NVIDIA Blackwell GPUs * Support for optimizations across all precisions (FP16, BF16) on NVIDIA Ampere GPU architecture generations and later Examples @@ -91,6 +94,7 @@ PyTorch loss = out.sum() loss.backward() +For a tutorial with more details, see the `Quickstart Notebook `_. JAX ^^^ @@ -190,7 +194,7 @@ We recommend updating to the latest NGC container available here: * https://catalog.ngc.nvidia.com/orgs/nvidia/containers/pytorch * https://catalog.ngc.nvidia.com/orgs/nvidia/containers/jax -If you run any examples, please ensure you are using a matching version of TransformerEngine. TransformerEngine is pre-built and packaged inside the containers with examples available at ``/opt/transformerengine`` or ``/opt/transformer-engine``. If you would like to use examples from TE main branch and are running into import errors, please try the latest pip package or building from source, although NGC containers are recommended for ease-of-use for most users. +If you run any examples, please ensure you are using a matching version of TransformerEngine. TransformerEngine is pre-built and packaged inside the containers with examples available at ``/opt/transformerengine`` or ``/opt/transformer-engine``. **Benefits of using NGC containers:** @@ -381,35 +385,27 @@ FP8 has been tested extensively across different model architectures and configu +------------+------------------+---------------------------------------------------------------------------------------------------------+ | Model | Framework | Source | +============+==================+=========================================================================================================+ -| T5-770M | JAX/T5x | https://github.com/NVIDIA/JAX-Toolbox/tree/main/rosetta/rosetta/projects/t5x#convergence-and-performance| -+------------+------------------+---------------------------------------------------------------------------------------------------------+ | MPT-1.3B | Mosaic Composer | https://www.mosaicml.com/blog/coreweave-nvidia-h100-part-1 | +------------+------------------+---------------------------------------------------------------------------------------------------------+ -| GPT-5B | JAX/Paxml | https://github.com/NVIDIA/JAX-Toolbox/tree/main/rosetta/rosetta/projects/pax#h100-results | -+------------+------------------+---------------------------------------------------------------------------------------------------------+ -| GPT-5B | NeMo Framework | Available on request | -+------------+------------------+---------------------------------------------------------------------------------------------------------+ | LLama2-7B | Alibaba Pai | https://mp.weixin.qq.com/s/NQT0uKXLbXyh5031zBdeBQ | +------------+------------------+---------------------------------------------------------------------------------------------------------+ -| T5-11B | JAX/T5x | Available on request | +| LLM-8B | Megatron Core | https://arxiv.org/abs/2506.08027 | +------------+------------------+---------------------------------------------------------------------------------------------------------+ | MPT-13B | Mosaic Composer | https://www.databricks.com/blog/turbocharged-training-optimizing-databricks-mosaic-ai-stack-fp8 | +------------+------------------+---------------------------------------------------------------------------------------------------------+ -| GPT-22B | NeMo Framework | Available on request | +| MoE-16B | Megatron Core | https://arxiv.org/abs/2506.08027 | +------------+------------------+---------------------------------------------------------------------------------------------------------+ | LLama2-70B | Alibaba Pai | https://mp.weixin.qq.com/s/NQT0uKXLbXyh5031zBdeBQ | +------------+------------------+---------------------------------------------------------------------------------------------------------+ -| GPT-175B | JAX/Paxml | https://github.com/NVIDIA/JAX-Toolbox/tree/main/rosetta/rosetta/projects/pax#h100-results | -+------------+------------------+---------------------------------------------------------------------------------------------------------+ Integrations ============ Transformer Engine has been integrated with popular LLM frameworks such as: -* `DeepSpeed `_ +* `DeepSpeed `_ * `Hugging Face Accelerate `_ -* `Lightning `_ +* `Lightning `_ * `MosaicML Composer `_ * `NVIDIA JAX Toolbox `_ * `NVIDIA Megatron-LM `_ @@ -417,10 +413,7 @@ Transformer Engine has been integrated with popular LLM frameworks such as: * `Amazon SageMaker Model Parallel Library `_ * `Levanter `_ * `GPT-NeoX `_ -* `Hugging Face Nanotron `_ - Coming soon! -* `Colossal-AI `_ - Coming soon! -* `PeriFlow `_ - Coming soon! - +* `Hugging Face Nanotron `_ Contributing ============ From d5922a3916c5639ce10373cc4e7b9ff443f76d46 Mon Sep 17 00:00:00 2001 From: Santosh Bhavani Date: Mon, 2 Feb 2026 15:16:53 -0800 Subject: [PATCH 2/3] fix(readme): address review feedback - Replace quickstart.ipynb link with fp8_primer.ipynb (file exists) - Fix extra whitespace in Megatron Core table rows Signed-off-by: Santosh Bhavani --- README.rst | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/README.rst b/README.rst index 9a21c5d764..64acaa7a9e 100644 --- a/README.rst +++ b/README.rst @@ -94,7 +94,7 @@ PyTorch loss = out.sum() loss.backward() -For a tutorial with more details, see the `Quickstart Notebook `_. +For a tutorial with more details, see the `FP8 Primer `_. JAX ^^^ @@ -141,7 +141,7 @@ Flax for _ in range(10): loss, (param_grads, other_grads) = fwd_bwd_fn(params, other_variables, inp) -For a more comprehensive tutorial, check out our `Quickstart Notebook `_. +For a more comprehensive tutorial, check out our `FP8 Primer `_. .. overview-end-marker-do-not-remove @@ -389,11 +389,11 @@ FP8 has been tested extensively across different model architectures and configu +------------+------------------+---------------------------------------------------------------------------------------------------------+ | LLama2-7B | Alibaba Pai | https://mp.weixin.qq.com/s/NQT0uKXLbXyh5031zBdeBQ | +------------+------------------+---------------------------------------------------------------------------------------------------------+ -| LLM-8B | Megatron Core | https://arxiv.org/abs/2506.08027 | +| LLM-8B | Megatron Core | https://arxiv.org/abs/2506.08027 | +------------+------------------+---------------------------------------------------------------------------------------------------------+ | MPT-13B | Mosaic Composer | https://www.databricks.com/blog/turbocharged-training-optimizing-databricks-mosaic-ai-stack-fp8 | +------------+------------------+---------------------------------------------------------------------------------------------------------+ -| MoE-16B | Megatron Core | https://arxiv.org/abs/2506.08027 | +| MoE-16B | Megatron Core | https://arxiv.org/abs/2506.08027 | +------------+------------------+---------------------------------------------------------------------------------------------------------+ | LLama2-70B | Alibaba Pai | https://mp.weixin.qq.com/s/NQT0uKXLbXyh5031zBdeBQ | +------------+------------------+---------------------------------------------------------------------------------------------------------+ From 52deeda6960f282725169f732a4b1f8e6b226359 Mon Sep 17 00:00:00 2001 From: Santosh Bhavani Date: Tue, 10 Feb 2026 15:23:31 -0800 Subject: [PATCH 3/3] Revert FP8 Primer link changes, defer to PR #2641 Signed-off-by: Santosh Bhavani --- README.rst | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/README.rst b/README.rst index 64acaa7a9e..9a54b1a5f4 100644 --- a/README.rst +++ b/README.rst @@ -94,7 +94,6 @@ PyTorch loss = out.sum() loss.backward() -For a tutorial with more details, see the `FP8 Primer `_. JAX ^^^ @@ -141,7 +140,7 @@ Flax for _ in range(10): loss, (param_grads, other_grads) = fwd_bwd_fn(params, other_variables, inp) -For a more comprehensive tutorial, check out our `FP8 Primer `_. +For a more comprehensive tutorial, check out our `Quickstart Notebook `_. .. overview-end-marker-do-not-remove