Question about 4K benchmark details on Jetson Orin NX

[lixianbang]

I was reviewing the benchmarks in the README and noticed the Jetson Orin NX result showing 950 ms for 4K images. I have a couple of questions about the specific testing conditions:

Regarding the 4K image resolution: The README states "4K resolution", but SAM3 natively uses a 1008×1008 input resolution. Could you clarify whether the benchmark was run with:

Native 4K images (e.g., 3840×2160) fed directly into the model?

Or were the 4K images first resized/cropped to SAM3's native 1008×1008 resolution before inference?

Regarding the 950ms latency: Does this end-to-end processing time include any pre-processing steps (like resizing, normalization, or padding), or is it purely the TensorRT engine inference time? The README mentions "end to end image processing latency per image" excluding image load/save, but I'd like to confirm what's included in that measurement.

Extra: What TensorRT precision was used for this benchmark (FP16 / INT8)? And what were the exact Jetson Orin NX power modes and JetPack versions?

These details would be super helpful for anyone trying to reproduce the results or evaluate the model for real-time applications on edge devices. Thanks in advance!

[dataplayer12]

Thanks for the question. This project implements efficient pre processing, SAM3 inference as well as post processing. Pre and post processing are performed with fused CUDA kernels. We perform resize, normalization and format conversion to NCHW in one single kernel. Usually people trip up on pre and post processing implementation when they want to create a production grade codebase. This project fills in this gap.

With this in mind, let me respond to your questions. The 950 ms is end to end latency from a 4K RGB buffer in to semantic segmentation result drawn on the input image out.

The power profile is MAXN.

[dataplayer12]

On precision, I use fp16 for all benchmarks

[lixianbang]

Thank you very much for your detailed and prompt response. I really appreciate the explanation about the fused CUDA kernels for pre/post processing and the clarification that the 950 ms is end-to-end latency from a 4K RGB buffer to the final drawn result.

However, I still have one core question regarding the input resolution to the TensorRT engine itself. In your pipeline, when we feed a 4K image (e.g., 3840×2160), does the pre‑processing step resize it to the native SAM3 input size of 1008×1008 before passing it to the TensorRT inference? Or does your export process modify the TensorRT engine to accept the full 4K resolution directly (i.e., the input tensor shape is set to 4K so that inference is performed on the original resolution)?

The reason I ask is that SAM3 was originally designed for 1008×1008 inputs, and I'm trying to understand whether the 950 ms latency includes inference on a 1008×1008 tensor (with resizing as pre‑processing) or on a 4K tensor, which would be a very different scenario. If it's the former, that would clarify a lot for me and anyone else trying to reproduce or evaluate the performance.

Could you please confirm which case applies? Additionally, if the resize to 1008×1008 is performed, is that done by your fused CUDA kernel before the model inference (as part of the pre‑processing step)?

Thank you again for your great work and for taking the time to answer questions. Your project is extremely valuable for production deployment on edge devices.

[dataplayer12]

Thank you for the kind words. I'm gosh you find the project useful.

Resizing is done by the fused preprocessing kernel from 4K to 1008x1008. TensorRT engine accepts the 1008x1008 input. I have not modified the native resolution of the model.

[lixianbang]

Thank you very much for the clear and prompt clarification. I now fully understand that the fused preprocessing kernel resizes the 4K input to 1008×1008 before feeding it into the TensorRT engine, while the native SAM3 resolution remains unchanged. This exactly answers my core question and is very helpful for my evaluation.

I truly appreciate your time and the great work you've put into this project – it's extremely valuable for production deployment on edge devices.