How do i madvise the mapped_region for managed_shared_memory?

[rahuldutta90]

For performance reasons we want to preallocate memory in the shared memory by calling madvise(MADV_POPULATE_WRITE). This generally has significant benefit in performance during the bootstrap phase as compared to allocate memory when the memory is accessed. However there are 2 reasons why we cannot do this today:

1. First the managed_shared_memory does not expose the public apis of managed_open_or_create_impl so we are unable to access to api get_mapped_region
2. Even we would have access to the mapped_region, the advise api supports only selected options (enum advice_types) and MADV_POPULATE_WRITE is not part of it.

Do you plan to support this?

I guess the only viable alternative is I separately create the shared_memory and madvise it before initializing using boost's shared_memory something like this:

fd= shm_open("BoostShmem", O_CREATE)
ptr = mmap64(
NULL, fileSize, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);
madvise(ptr, fileSize, MADV_POPULATE_WRITE);
boost::interprocess::managed_shared_memory shared_memory(boost::interprocess::create_only,
"BoostShmem", fileSize);

[rahuldutta90]

Hi any input here?

[igaztanaga]

There is no plan to add madvice support for managed types, but this could change if there is support on the community. I guess supporting madvise for the whole memory mapped range would be enough.

On the other hand, the available madvise-like flags are modelled after POSIX flags because Boost.Interprocess hopes for minimally portable abstractions, and MADV_POPULATE_XXX options seem quite Linux-specific. There a lot of different, system-specific flags around (BSD, Linux, etc.), which I don't think are compatible enough, although I might be wrong. Would "advice_willneed" (translated to MADV_WILLNEED for linux) be acceptable for your use-case? Which is the main advantages of those MADV_POPULATE_XXX specific options?

[rahuldutta90]

Thanks @igaztanaga

I guess supporting madvise for the whole memory mapped range would be enough.

Yes, this is what i am trying to do.

Would "advice_willneed" (translated to MADV_WILLNEED for linux) be acceptable for your use-case? Which is the main advantages of those MADV_POPULATE_XXX specific options?

I need to check if that would suffice. MADV_POPULATE_XXX helps pre-allocating the underlying memory and also make it visible to userspace.

However i do see a workaround. Looking through the code, i do see this api "get_address" (exposed here:

interprocess/include/boost/interprocess/detail/managed_memory_impl.hpp

Line 229 in b11c6a1

{ return reinterpret_cast<char*>(mp_header) - Offset; }

). Based on high level look through the code, I think it does return the pointer that we have gotten from mmap of the shared memory. Is that a correct understanding? If yes, then we should be able to call madvise ourselves on the memory pointer that we get from get_address provided that is the memory pointer.

This is my understanding:

We get mmap address here:

interprocess/include/boost/interprocess/mapped_region.hpp

Line 711 in b11c6a1

m_base = static_cast<char*>(base) + page_offset;

and we set the address from mapped_region here:

interprocess/include/boost/interprocess/detail/managed_open_or_create_impl.hpp

Line 367 in b11c6a1

construct_func( static_cast<char*>(region.get_address()) + ManagedOpenOrCreateUserOffset

which finally sets the address here:

interprocess/include/boost/interprocess/detail/managed_memory_impl.hpp

Line 742 in b11c6a1

return m_frontend->create_impl(addr, static_cast<size_type>(size));

which is setting it here:

interprocess/include/boost/interprocess/detail/managed_memory_impl.hpp

Line 180 in b11c6a1

mp_header = ::new(addr, boost_container_new_t()) segment_manager(size);

and being returned by the get_address.

Is my understanding correct ehre? Can we safely assume the pointer we get from get_address is the mmap ed memory pointer?

[igaztanaga]

Yes, the address returned by get_address is the base address of the first mapped memory page.