How to setup LinuxNativeWindow with SDL

[DarthCloud64]

Hello. I'm wondering if anyone has experience with grabbing the native X11 window handle from SDL2 and passing that to Diligent's LinuxNativeWindow. I must be doing it wrong because I get a segfault when creating the swap chain using the LinuxNativeWindow I create

This is where I grab the window information from SDL

NativeWindowInformation WindowManager::GetNativeWindowInformation() {
        SDL_SysWMinfo sdlSysWMinfo;
        SDL_GetWindowWMInfo(_window, &sdlSysWMinfo);

        NativeWindowInformation nativeWindowInformation;

        nativeWindowInformation.x11WindowHandle = sdlSysWMinfo.info.x11.display;
        nativeWindowInformation.x11WindowId = sdlSysWMinfo.info.x11.window;

        return nativeWindowInformation;
    }

And this is where I'm creating the LinuxNativeWindow using that information

void RenderingManager::InitializeDiligentEngine(NativeWindowInformation nativeWindowInformation) {
        Diligent::EngineVkCreateInfo engineVkCreateInfo;

        IEngineFactoryVk* factoryVk = GetEngineFactoryVk();
        factoryVk->CreateDeviceAndContextsVk(engineVkCreateInfo, &_renderDevice, &_deviceContext);

        LinuxNativeWindow linuxNativeWindow;
        linuxNativeWindow.WindowId = nativeWindowInformation.x11WindowId;
        linuxNativeWindow.pXCBConnection = (void*)nativeWindowInformation.x11WindowHandle;

        SwapChainDesc swapChainDesc;

        factoryVk->CreateSwapChainVk(_renderDevice, _deviceContext, swapChainDesc, linuxNativeWindow, &_swapChain);
    }

[TheMostDiligent]

You are passing X11 window handle as XCB connection, which does not seem right to me.

[DarthCloud64]

Ok so yeah turns out I just need the XLIB Window and Display. One thing I am stuck on is the "Window" type in the XLIB is of type unsigned long, but the WindowId in the LinuxNativeWindow for Diligent is type uint32_t. Seems like that would cause issues when assigning the XLIB window id to the LinuxNativeWindow

I updated my code to this, but I'm still experiencing issues with creating the swap chain

    void RenderingManager::InitializeDiligentEngine(NativeWindowInformation* nativeWindowInformation) {
        Diligent::EngineVkCreateInfo engineVkCreateInfo;

        IEngineFactoryVk* factoryVk = GetEngineFactoryVk();
        factoryVk->CreateDeviceAndContextsVk(engineVkCreateInfo, &_renderDevice, &_deviceContext);

        LinuxNativeWindow linuxNativeWindow;
        linuxNativeWindow.WindowId = nativeWindowInformation->x11WindowId;
        linuxNativeWindow.pDisplay = nativeWindowInformation->x11Display;

        SwapChainDesc swapChainDesc;

        factoryVk->CreateSwapChainVk(_renderDevice, _deviceContext, swapChainDesc, linuxNativeWindow, &_swapChain);
    }

[DarthCloud64]

Does diligent automatically use XCB for Vulkan? Looks like the segfault is happening when obtaining a XCB connection, but SDL is using xlib

[TheMostDiligent]

Take a look at this tutorial - it shows what native window parameters are needed.

[stanoddly]

It seems you can get XCB connection from a display using XGetXCBConnection from libX11-xcb.so.1. Note I use C# and SDL3 😅

First to define the external function:

[DllImport("libX11-xcb.so.1", EntryPoint="XGetXCBConnection", CallingConvention = CallingConvention.Cdecl)]
public static extern IntPtr XGetXCBConnection(IntPtr display);

Then once you have the X11 display from SDL, you can call XGetXCBConnection. I omitted error handling to simplify the code:

SDL3.SDL_Init(SDL_InitFlags.SDL_INIT_VIDEO);

var window = SDL3.SDL_CreateWindow("name"u8, 640, 480, SDL_WindowFlags.SDL_WINDOW_RESIZABLE);

SDL_PropertiesID propertiesId = SDL3.SDL_GetWindowProperties(window);

fixed (byte* displayStringPointer = "SDL.window.x11.display"u8)
{
    IntPtr display = SDL3.SDL_GetPointerProperty(propertiesId, displayStringPointer, 0);

    IntPtr connection = XGetXCBConnection(display);
    if (connection == IntPtr.Zero) throw new Exception($"XGetXCBConnection failed");
}

[markusbkk]

Any further progress on this?

I used @stanoddly 's suggestion but end up with an ERROR_OUT_OF_HOST_MEMORY.

#322