Support for peer-to-peer DMA using the DMA-BUF/P2PDMA framework on non-SoC platforms

[nullbytepl]

I'm currently working on modernizing a Linux driver for a certain data acquisition device that is basically just an FPGA that transfers data to a host PC using PCIe, after which it's processed using CUDA. Part of my work was to restore long-unused support for GPUDirect RDMA, which led me to find that:
a) peer-to-peer DMA is supposed to be handled using the P2PDMA framework in Linux these days;
b) the old method of doing P2P on Nvidia (linking to exported symbols in nvidia.ko) is basically impossible to consistently do across most of the OS/kernel configs I've tested;
c) Nvidia seems to support DMA-BUF with P2P in the open source drivers!

However, I couldn't get it working out of the box, which led me on a fairly long debugging journey, the conclusion of which was a modified driver, which was enough to get working peer-to-peer DMA on an Quadro RTX 5000 (Turing). All it took was to remove a few checks, which basically locked the feature down to SoCs with C2C interconnects (like Grace Superchip) or no framebuffer (GB10B/Thor).

Is there a reason for this limitation being present in the first place? Could Nvidia unlock it for non-SoC setups too, even if behind a flag for example? Even if it had similar limitations as GPUDirect RDMA, like only working with 1:1/disabled IOMMU, it would be better than a vendor-specific interface.

cc @pjarosik