Skip to content

Commit f43c18b

Browse files
ruanjinjie-engakpm00
authored andcommitted
arm64: avoid memcpy() for syscall_get_arguments()
Do not use memcpy() to extract syscall arguments from struct pt_regs but rather just perform direct assignments. Update syscall_set_arguments() too to keep syscall_get_arguments() and syscall_set_arguments() in sync. With Generic Entry patch[1] and turn on audit, the performance benchmarks from perf bench basic syscall on kunpeng920 gives roughly a 1% performance uplift. | Metric | W/O this patch | With this patch | Change | | ---------- | -------------- | --------------- | --------- | | Total time | 2.241 [sec] | 2.211 [sec] | ↓1.36% | | usecs/op | 0.224157 | 0.221146 | ↓1.36% | | ops/sec | 4,461,157 | 4,501,409 | ↑0.9% | Disassembly shows that using direct assignment causes syscall_set_arguments() to be inlined and cuts the instruction count by five or six compared to memcpy(). Because __audit_syscall_entry() only uses four syscall arguments, the compiler has also elided the copy of regs->regs[4] and regs->regs[5]. Before: <syscall_get_arguments.constprop.0>: aa0103e2 mov x2, x1 91002003 add x3, x0, #0x8 f9408804 ldr x4, [x0, #272] f8008444 str x4, [x2], #8 a9409404 ldp x4, x5, [x0, #8] a9009424 stp x4, x5, [x1, #8] a9418400 ldp x0, x1, [x0, #24] a9010440 stp x0, x1, [x2, #16] f9401060 ldr x0, [x3, #32] f9001040 str x0, [x2, #32] d65f03c0 ret d503201f nop After: a9408e82 ldp x2, x3, [x20, #8] 2a1603e0 mov w0, w22 f9400e84 ldr x4, [x20, #24] f9408a81 ldr x1, [x20, #272] 9401c4ba bl ffff800080215ca8 <__audit_syscall_entry> This also aligns the implementation with x86 and RISC-V. Link: https://lkml.kernel.org/r/20251201120633.1193122-3-ruanjinjie@huawei.com Link: https://lore.kernel.org/all/20251126071446.3234218-1-ruanjinjie@huawei.com/ [1] Signed-off-by: Jinjie Ruan <ruanjinjie@huawei.com> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Charlie Jenkins <charlie@rivosinc.com> Cc: Christian Zankel <chris@zankel.net> Cc: "Dmitry V. Levin" <ldv@strace.io> Cc: Helge Deller <deller@gmx.de> Cc: Maciej W. Rozycki <macro@orcam.me.uk> Cc: Marc Rutland <mark.rutland@arm.com> Cc: Max Filippov <jcmvbkbc@gmail.com> Cc: Russell King (Oracle) <rmk+kernel@armlinux.org.uk> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Thomas Gleinxer <tglx@linutronix.de> Cc: Will Deacon <will@kernel.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
1 parent 99630b9 commit f43c18b

1 file changed

Lines changed: 12 additions & 4 deletions

File tree

arch/arm64/include/asm/syscall.h

Lines changed: 12 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -82,16 +82,24 @@ static inline void syscall_get_arguments(struct task_struct *task,
8282
unsigned long *args)
8383
{
8484
args[0] = regs->orig_x0;
85-
args++;
86-
87-
memcpy(args, &regs->regs[1], 5 * sizeof(args[0]));
85+
args[1] = regs->regs[1];
86+
args[2] = regs->regs[2];
87+
args[3] = regs->regs[3];
88+
args[4] = regs->regs[4];
89+
args[5] = regs->regs[5];
8890
}
8991

9092
static inline void syscall_set_arguments(struct task_struct *task,
9193
struct pt_regs *regs,
9294
const unsigned long *args)
9395
{
94-
memcpy(&regs->regs[0], args, 6 * sizeof(args[0]));
96+
regs->regs[0] = args[0];
97+
regs->regs[1] = args[1];
98+
regs->regs[2] = args[2];
99+
regs->regs[3] = args[3];
100+
regs->regs[4] = args[4];
101+
regs->regs[5] = args[5];
102+
95103
/*
96104
* Also copy the first argument into orig_x0
97105
* so that syscall_get_arguments() would return it

0 commit comments

Comments
 (0)