Enable user-space for DP (part 4 of #10287) by lyakh · Pull Request #10386 · thesofproject/sof

lyakh · 2025-11-21T16:39:57Z

The rest of DP-userspace commits, WiP.

Copilot

Pull request overview

This pull request enables user-space support for the DP (Data Processing) scheduler in SOF (Sound Open Firmware), implementing part 4 of #10287. The changes introduce two implementations: a "native" version with full userspace capabilities and a "proxy" version with limited support, conditional on the CONFIG_SOF_USERSPACE_PROXY flag.

Key changes include:

Splitting DP scheduler implementation into native and proxy versions to support different userspace configurations
Adding syscall wrappers for memory allocation functions (mod_alloc_ext, mod_free, mod_fast_get) to enable userspace modules
Modifying buffer allocation API to accept heap parameters for DP module-specific heaps
Implementing memory domain management for userspace DP modules
Updating sparse annotations removal for stack allocation functions

Reviewed changes

Copilot reviewed 34 out of 34 changed files in this pull request and generated 9 comments.

Show a summary per file

File	Description
src/schedule/zephyr_dp_schedule_native.c	New native implementation with full IPC handling and memory domain support for userspace DP modules
src/schedule/zephyr_dp_schedule_proxy.c	New proxy implementation with simplified userspace support
src/schedule/zephyr_dp_schedule.h	Shared header defining common structures for both DP scheduler implementations
src/schedule/zephyr_dp_schedule.c	Refactored to move implementation-specific code to native/proxy files
src/schedule/CMakeLists.txt	Updated build configuration to conditionally compile native or proxy implementation
src/audio/module_adapter/module/generic.c	Added syscall implementations and verification handlers for userspace module APIs
src/audio/module_adapter/module_adapter.c	Updated to use per-module heaps for DP components with reference counting
src/audio/buffers/comp_buffer.c	Modified buffer allocation to support DP-specific heaps and reference counting
src/include/sof/audio/module_adapter/module/generic.h	Added syscall declarations for module memory APIs
src/include/sof/audio/buffer.h	Updated buffer allocation API signatures to include heap parameter
src/ipc/ipc4/helper.c	Modified component connection to use DP heap for DP domain components
src/library_manager/llext_manager.c	Added `llext_manager_rm_domain` function to remove library partitions from memory domains
zephyr/lib/userspace_helper.c	Removed sparse annotations from stack allocation functions
test/cmocka/src/*/.c	Updated all test code to pass NULL heap parameter to buffer_new calls
app/prj.conf	Added dynamic thread allocation configuration
app/boards/intel_adsp_ace30_ptl.conf	Enabled userspace support

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

src/schedule/zephyr_dp_schedule_proxy.c

src/schedule/zephyr_dp_schedule_application.c

Copilot · 2025-11-26T16:00:17Z

src/audio/module_adapter/module_adapter.c

+	list_item_del(&mod->dev->bsink_list);
 	sof_heap_free(mod_heap, mod->dev);
 	sof_heap_free(mod_heap, mod);
+	if (!mod_heap || !--mod_heap_user->client_count)


Logic error: The condition !mod_heap will always be false at this point because mod_heap_user is derived from mod_heap using container_of. If mod_heap were NULL, the container_of would have already caused undefined behavior. The check should be if (!mod_heap_user || !--mod_heap_user->client_count) or restructured to check mod_heap_user before calling container_of.

Copilot · 2025-11-26T16:00:17Z

src/audio/module_adapter/module_adapter.c

 static void module_adapter_mem_free(struct processing_module *mod)
 {
 	struct k_heap *mod_heap = mod->priv.resources.heap;
+	struct dp_heap_user *mod_heap_user = container_of(mod_heap, struct dp_heap_user, heap);


Incorrect use of container_of: When not in DP domain, mod_heap points to drv->user_heap (not part of a dp_heap_user structure), making container_of() invalid. This will compute an incorrect pointer. Should check if the module used a DP heap before using container_of, similar to how line 112 sets mod_heap_user = NULL for non-DP cases.

Can you @lyakh comment on this? Isn't this a valid point, for LL module, priv.resources.heap will hav a k_heap and container_of() points to random place? I guess this is fine if mod_heap_user is NULL, but is it?

src/audio/buffers/comp_buffer.c

src/audio/module_adapter/module_adapter.c

src/schedule/zephyr_dp_schedule_application.c

lyakh · 2025-12-16T13:40:03Z

SOFCI TEST

kv2019i

Can you @lyakh check the one container_of() case?

Otherwise my concerns addressed. There are some K_FOREVER/k_timout things open, but I guess we can address these in follow-ups.

kv2019i · 2025-12-17T12:30:46Z

src/audio/module_adapter/module/generic.c

 * The allocated memory is automatically freed when the module is unloaded.
 */
-void *mod_alloc_ext(struct processing_module *mod, uint32_t flags, size_t size, size_t alignment)
+void *z_impl_mod_alloc_ext(struct processing_module *mod, uint32_t flags, size_t size,


Otoh, this is not on the hot path so maybe the syscall is a safer bet to start with.

kv2019i · 2025-12-17T12:44:24Z

src/audio/module_adapter/module_adapter.c

 static void module_adapter_mem_free(struct processing_module *mod)
 {
 	struct k_heap *mod_heap = mod->priv.resources.heap;
+	struct dp_heap_user *mod_heap_user = container_of(mod_heap, struct dp_heap_user, heap);


Can you @lyakh comment on this? Isn't this a valid point, for LL module, priv.resources.heap will hav a k_heap and container_of() points to random place? I guess this is fine if mod_heap_user is NULL, but is it?

kv2019i · 2025-12-17T12:50:37Z

app/os_linux_overlay.conf

 # SOF Linux driver does not require FW to retain its
 # state, so context save can be disabled
 CONFIG_ADSP_IMR_CONTEXT_SAVE=n
+CONFIG_SOF_USERSPACE_PROXY=n


Minor: an inline comment in os_linux_overlay.conf would be nice and serve as an example for future addition.

kv2019i · 2025-12-17T12:51:40Z

app/os_linux_overlay.conf

@@ -6,3 +6,4 @@
 # SOF Linux driver does not require FW to retain its


Another nit: this is not a "Linux build", but a build of SOF that is targetted to be used with Linux SOF driver. You can also build SOF for LInux (the posix library build), so better to not to add to the confusion.

rrrright, but this isn't from this PR, so we can improve it independently

@lyakh Uhm? I was commenting on the git commit message "switch to "application" mode for Linux builds " -- Surely that is part of this PR?

@lyakh Uhm? I was commenting on the git commit message "switch to "application" mode for Linux builds " -- Surely that is part of this PR?

@kv2019i aah, ok, sorry, I thought you meant the comment in that file where your comment happened to land. Yes, GH's inability to comment on commit messages is unfortunate.

lyakh · 2025-12-17T14:26:39Z

Can you @lyakh comment on this? Isn't this a valid point, for LL module, priv.resources.heap will hav a k_heap and container_of() points to random place? I guess this is fine if mod_heap_user is NULL, but is it?

@kv2019i I should, yes, here's my comment: "I'm sorry" :-) I had a fix for this but forgot to apply when pushing. Fixed now.

kv2019i · 2025-12-18T07:44:49Z

app/os_linux_overlay.conf

@@ -6,3 +6,4 @@
 # SOF Linux driver does not require FW to retain its


@lyakh Uhm? I was commenting on the git commit message "switch to "application" mode for Linux builds " -- Surely that is part of this PR?

abonislawski

Main open: is there any data for performance impact? We agreed to monitor it and this PR looks like significant step forward (enabling all DP threads in userspace).

Do Linux CI tests include DP modules?

abonislawski · 2025-12-19T08:46:59Z

app/os_linux_overlay.conf

 # SOF Linux driver does not require FW to retain its
 # state, so context save can be disabled
 CONFIG_ADSP_IMR_CONTEXT_SAVE=n
+CONFIG_SOF_USERSPACE_PROXY=n


So many !CONFIG_SOF_USERSPACE_PROXY in the code and surprisingly CONFIG_SOF_USERSPACE_PROXY=n is enabling all DP threads in userspace. This will cause a lot of confusion in the future - why does disabling some strange proxy make so many changes?

It looks like all of this would look a lot clearer if we had CONFIG_SOF_USERSPACE_APP/LICATION.

Especially for places where we have a part of the code only for one type:

#if CONFIG_USERSPACE && !CONFIG_SOF_USERSPACE_PROXY
...
#endif

@abonislawski yes, maybe you're right. We can consider adding such an option. I just wasn't very keen on adding a redundant option - if it's always perfectly derivable from what we have now. Maybe just

#define SOF_USERSPACE_APPLICATION (CONFIG_USERSPACE && !CONFIG_SOF_USERSPACE_PROXY)

or similar... We could also make a choice in Kconfig - but a choice with just 2 options is also rather redundant. A=n immediately forces B=y

So this will become clearer as the code evolves, Kconfig has a nice way to manage the dependencies and enforce mutual exclusion (we just have to spell it out clearly in the help text), probably as a top level userspace menu (since this would also contain Kconfigs for various stack sizes and shared memory heaps etc).

lyakh · 2025-12-22T08:52:57Z

Main open: is there any data for performance impact? We agreed to monitor it and this PR looks like significant step forward (enabling all DP threads in userspace).

Do Linux CI tests include DP modules?

@abonislawski no, currently Jenkins isn't running any DP tests, so we cannot compare performance. And yes, the goal is to run all DP processing in userspace

module_is_ready_to_process() can call the .is_ready_to_process() module method, therefore it can only be called from the DP thread context itself, not from the scheduler. Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>

With the application DP version IPCs should be processed on the thread itself. Prepare the scheduler for the transition. Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>

Instead of calling module hooks inline signal the DP thread to call them in the thread context in DP-scheduled module case. Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>

The only functions that have to be converted to syscalls are mod_alloc_ext() and mod_free(), the rest of the API is implemented using inline functions. Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>

Run DP threads in user-space. Move all the respective memory and kobjects to a dedicated memory domain. Work around Zephyr inability to remove memory domains on Xtensa. Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>

DP scheduler's .cancel() method is so far only used with the system agent and with proxy, make it panic in other configurations to avoid accidental use. Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>

Add userspace mapping for cold module code and data. Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>

While waiting for zephyrproject-rtos/zephyr#101073 to be merged, need to drop const attribute. Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>

If CONFIG_SOF_USERSPACE_PROXY isn't used all DP threads should run in userspace mode. Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>

With userspace enabled we easily use up the currently configured for ACE 3.0 by Zephyr 64 L2 page tables when running tests with multiple pipelines on multiple cores with userspace enabled. Double the number to cover current test cases. Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>

Linux SOF builds should use the "application" userspace implementation for DP. Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>

lgirdwood

LGTM, but are Kconfig opens now all resolved ?

lgirdwood · 2025-12-22T21:10:03Z

src/audio/module_adapter/module/generic.c

+
 	if (ops->reset) {
-		ret = ops->reset(mod);
+		if (mod->dev->ipc_config.proc_domain == COMP_PROCESSING_DOMAIN_DP) {


@lyakh now resolved ?

lgirdwood · 2025-12-23T13:58:33Z

MTL CI errors showing up as Soundwire link errors and unrelated to PTL MMU logic. Lets get this more tested in CI now.

lyakh force-pushed the user-p3 branch 6 times, most recently from a72e1f5 to f123bd6 Compare November 26, 2025 15:15

lyakh changed the title ~~[WiP][SKIP SOF-TEST] Finally enable user-space for DP (part 4 of #10287)~~ [WiP] Finally enable user-space for DP (part 4 of #10287) Nov 26, 2025

lyakh force-pushed the user-p3 branch 2 times, most recently from 1d75c53 to 364e1e1 Compare November 26, 2025 15:52

lyakh marked this pull request as ready for review November 26, 2025 15:53

Copilot AI review requested due to automatic review settings November 26, 2025 15:53

lyakh requested review from LaurentiuM1234, RanderWang, abonislawski, bardliao, dbaluta, iuliana-prodan, jxstelter, kv2019i, lbetlej, lgirdwood, marcinszkudlinski, mmaka1, pblaszko, plbossart and ranj063 as code owners November 26, 2025 15:53

Copilot started reviewing on behalf of lyakh November 26, 2025 15:53 View session

Copilot finished reviewing on behalf of lyakh November 26, 2025 15:56

Copilot AI reviewed Nov 26, 2025

View reviewed changes

lgirdwood changed the title ~~[WiP] Finally enable user-space for DP (part 4 of #10287)~~ Enable user-space for DP (part 4 of #10287) Nov 27, 2025

lyakh mentioned this pull request Dec 16, 2025

DP: more preparation for application-level user-space (part 4.2 of #10287) #10446

Merged

lyakh force-pushed the user-p3 branch from 910ba02 to 72d289a Compare December 17, 2025 11:59

kv2019i reviewed Dec 17, 2025

View reviewed changes

lyakh force-pushed the user-p3 branch 2 times, most recently from 5f9d42b to 2993b96 Compare December 17, 2025 14:01

kv2019i approved these changes Dec 18, 2025

View reviewed changes

abonislawski reviewed Dec 19, 2025

View reviewed changes

lyakh force-pushed the user-p3 branch from 2993b96 to 193dc18 Compare December 19, 2025 11:09

lyakh added 11 commits December 22, 2025 15:41

dp: application: move IPC execution to the thread

ceca79c

With the application DP version IPCs should be processed on the thread itself. Prepare the scheduler for the transition. Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>

module-adapter: switch to DP signaling

5950bf8

Instead of calling module hooks inline signal the DP thread to call them in the thread context in DP-scheduled module case. Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>

mod_alloc: convert to syscalls

cf7fed9

The only functions that have to be converted to syscalls are mod_alloc_ext() and mod_free(), the rest of the API is implemented using inline functions. Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>

dp: convert to user-space

bd10353

Run DP threads in user-space. Move all the respective memory and kobjects to a dedicated memory domain. Work around Zephyr inability to remove memory domains on Xtensa. Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>

dp: disable the .cancel() method when unused

f56a11c

DP scheduler's .cancel() method is so far only used with the system agent and with proxy, make it panic in other configurations to avoid accidental use. Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>

userspace: llext: add support for cold code and data

f1cd7f6

Add userspace mapping for cold module code and data. Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>

llext: temporary compilation fix waiting for Zephyr PR

d30457f

While waiting for zephyrproject-rtos/zephyr#101073 to be merged, need to drop const attribute. Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>

dp: switch all threads to userspace unless PROXY is used

16212df

If CONFIG_SOF_USERSPACE_PROXY isn't used all DP threads should run in userspace mode. Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>

DP: userspace: switch to "application" mode for Linux builds

9d9102c

Linux SOF builds should use the "application" userspace implementation for DP. Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>

lyakh force-pushed the user-p3 branch from 193dc18 to 9d9102c Compare December 22, 2025 14:43

lgirdwood reviewed Dec 22, 2025

View reviewed changes

lgirdwood approved these changes Dec 23, 2025

View reviewed changes

lgirdwood merged commit ee15b8c into thesofproject:main Dec 23, 2025
37 of 43 checks passed

lyakh deleted the user-p3 branch December 23, 2025 14:26

lyakh mentioned this pull request Dec 23, 2025

Add a simple object pool allocator #10456

Merged

softwarecki mentioned this pull request Feb 2, 2026

module-adapter: fix memory leaks caused by DP dask #10512

Closed

		@@ -6,3 +6,4 @@
		# SOF Linux driver does not require FW to retain its

Conversation

lyakh commented Nov 21, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI Nov 26, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 26, 2025

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

lyakh commented Dec 16, 2025

Uh oh!

kv2019i left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lyakh commented Dec 17, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

abonislawski left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lyakh commented Dec 22, 2025

Uh oh!

lgirdwood left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lgirdwood commented Dec 23, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants