Missed optimization opportunity when trivially moving tuple of slices

[recatek]

Example code:

#[repr(C)]
pub struct ThreeSlices<'a>(&'a [u32], &'a [u32], &'a [u32]);

#[inline(always)]
pub fn should_be_no_op(val: ThreeSlices) -> ThreeSlices {
    val
}

pub fn sum_slices_1(val: ThreeSlices) -> u32 {
    sum(&val)
}

pub fn sum_slices_2(val: ThreeSlices) -> u32 {
    let val = should_be_no_op(val);
    sum(&val)
}

#[inline(never)]
pub fn sum(val: &ThreeSlices) -> u32 {
    val.0.iter().sum::<u32>() + val.1.iter().sum::<u32>() + val.2.iter().sum::<u32>()
}

In rustc 1.67 stable this generates a number of moves (I suppose for calling convention?) that I don't think need to be there, especially when inlining:

example::sum_slices_1:
        jmp     qword ptr [rip + example::sum@GOTPCREL]

example::sum_slices_2:
        sub     rsp, 56
        movups  xmm0, xmmword ptr [rdi]
        movups  xmm1, xmmword ptr [rdi + 16]
        movups  xmm2, xmmword ptr [rdi + 32]
        movaps  xmmword ptr [rsp + 32], xmm2
        movaps  xmmword ptr [rsp + 16], xmm1
        movaps  xmmword ptr [rsp], xmm0
        mov     rdi, rsp
        call    qword ptr [rip + example::sum@GOTPCREL]
        add     rsp, 56
        ret

See https://rust.godbolt.org/z/azs11edK8

While this is a pretty pointless example, this comes up in situations where you might want to convert a tuple of slices into a struct of slices in order to assign names to the tuple members.

[clubby789]

LLVM should probably be able to elide the below memcpy and pass the original pointer.

  %val1 = alloca %"ThreeSlices<'_>", align 8
  call void @llvm.lifetime.start.p0(i64 48, ptr nonnull %val1)
  call void @llvm.memcpy.p0.p0.i64(ptr noundef nonnull align 8 dereferenceable(48) %val1, ptr noundef nonnull align 8 dereferenceable(48) %val, i64 48, i1 false)
  %0 = call noundef i32 @example::sum(ptr noalias noundef nonnull readonly align 8 dereferenceable(48) %val1)
  call void @llvm.lifetime.end.p0(i64 48, ptr nonnull %val1)
  ret i32 %0

@rustbot label +A-llvm +I-slow

[khei4]

This seems to be resolved on current nightly by #112157 & https://reviews.llvm.org/D150970 Thanks @erikdesjardins @nikic 🎉 (I'm not so sure about an incoming inherent problem on alignment attribute)

[nikic]

Fixed as part of the LLVM 17 update (#114048), needs codegen test.

[khei4]

Making the sum a little more vague than the above code prevents the removal of memcpy.

#[repr(C)]
pub struct ThreeSlices<'a>(&'a [u32], &'a [u32], &'a [u32]);

#[no_mangle]
pub fn sum_slices_2(val: ThreeSlices, f: fn(_: &ThreeSlices))  {
    let val = val;
    f(&val)
}

https://rust.godbolt.org/z/1hK4f1zGs
And I now think that preventing memcpy removal is the right, because the function may capture the argument. But rustc seems to annotate the original argument with nocapture.

define void @sum_slices_2(ptr noalias nocapture noundef readonly align 8 dereferenceable(48) %val, ptr nocapture noundef nonnull readonly %f) unnamed_addr #0 {
  %val1 = alloca %"ThreeSlices<'_>", align 8
  call void @llvm.lifetime.start.p0(i64 48, ptr nonnull %val1)
  call void @llvm.memcpy.p0.p0.i64(ptr noundef nonnull align 8 dereferenceable(48) %val1, ptr noundef nonnull align 8 dereferenceable(48) %val, i64 48, i1 false)
  call void %f(ptr noalias noundef nonnull readonly align 8 dereferenceable(48) %val1)
  call void @llvm.lifetime.end.p0(i64 48, ptr nonnull %val1)
  ret void
}

Is this parameter nocapture attribution broken? Shared reference could be captured by closure generally, and nocapture attributes with argument guarantee that no captures happen during the function execution, right?

Edit: It seems like not so, precisely, the above argument is a function pointer, not a closure, and I now think function pointers never capture any variables (because of no-capture closure only coerced to function pointers?, fn doc). So is this another missing opportunity?

[nikic]

And I now think that preventing memcpy removal is the right, because the function may capture the argument.

Yes, the removal is only allowed in the original example, because the called function is known. It does not work with an unknown function due to missing nocapture.

But rustc seems to annotate the original argument with nocapture

Note that the nocapture is on the by-value argument of sum_slices_2, not the by-reference argument of the function pointer f.

[khei4]

Note that the nocapture is on the by-value argument of sum_slices_2, not the by-reference argument of the function pointer f.

That sounds reasonable. Thanks! So, in this case, if the assumption that the function pointer never captures the argument was correct, missing attribution opportunities for the callbase?

[nikic]

I don't think there is any missing attribution opportunity here. In your example, if %val is nocapture, then that does not imply anything about %val1 being nocapture (it's a completely different object).

Even if you directly passed through the argument to the call (without intermediate memcpy to alloca), inferring nocapture is still not straightforward, because nocapture allows interior captures (i.e. it's possible to capture the pointer within the function, as long as it does not stay captured after the function returns).

[khei4]

Even if you directly passed through the argument to the call (without intermediate memcpy to alloca), inferring nocapture is still not straightforward, because nocapture allows interior captures (i.e. it's possible to capture the pointer within the function, as long as it does not stay captured after the function returns).

Oh, I missed that fact. I think for general pointers, it sounds correct, but I'm still wondering whether rust's safe function pointer fn semantics can deduce the nocapture for all callee arguments.

(Edit: Now I think that on this case, capturing in LLVM might happen. Thanks!