Performance regression in nightly with target-cpu=native on znver4

[nsarlin-zama]

Code

Hi, I have a regression when I try to run benchmarks using the nightly compiler on a crate that does elliptic curve operations:
In this repo: https://github.com/zama-ai/tfhe-rs, on branch v1.0 , in the crate tfhe-zk-pok, running the bench with RUSTFLAGS="-C target-cpu=znver4" cargo +nightly-2025-02-18 bench -p tfhe-zk-pok --bench pke_v2 -- pke_zk_verify_v2::PKEV2.

The machine running the bench is an aws hpc7 with an AMD EPYC 9R14 CPU.
Depending on the bench I get around 30% slower timings.

A bisect shows that the regression was introduced with nightly 2025-02-18. Stable toolchains do not seem to be impacted. The slowdown is only visible with -C target-cpu=znver4.

Version it worked on

All stable toolchains, and nightly before 2025-02-18.

Version with regression

nightly 2025-02-18 and after. The latest one as of today (2025-04-04) also has the regression.

Thank you for your help.

[lqd]

2025-02-18 sounds recent enough to have a more precise bisection, to a given PR with https://github.com/rust-lang/cargo-bisect-rustc/ -- could you try to do it?

[nikic]

2025-02-18 is probably the LLVM 20 upgrade.

[lqd]

Ah indeed, #135763 landed on the 17th.

[nsarlin-zama]

I can confirm from cargo-bisect-rustc (great tool btw!) that the regressing commit is the one introducing llvm-20: ce36a96

[apiraino]

@nsarlin-zama do you think you can manage to provide a minimal/reduced reproducible for the perf. regression? Thanks.

@rustbot label E-needs-mcve

[nsarlin-zama]

This is not yet an MRE, but I can pinpoint the slowdown to calls of RustCrypto's sha3.
By running their sha3 benchmark on the target server:

$ cargo +nightly-2025-04-04 bench -- shake256_10000
test shake256_10000 ... bench:      24,946.81 ns/iter (+/- 144.52) = 400 MB/s

$ RUSTFLAGS="-C target-cpu=znver4" cargo +nightly-2025-04-04 bench -- shake256_10000
test shake256_10000 ... bench:      44,230.92 ns/iter (+/- 121.21) = 226 MB/s

[nsarlin-zama]

From the keccak implementation in RustCrypto I can produce a smaller reproducing example:

#![feature(test)]
extern crate test;

const RHO: [u32; 24] = [
    1, 3, 6, 10, 15, 21, 28, 36, 45, 55, 2, 14, 27, 41, 56, 8, 25, 43, 62, 18, 39, 61, 20, 44,
];

const PI: [usize; 24] = [
    10, 7, 11, 17, 18, 3, 5, 16, 8, 21, 24, 4, 15, 23, 19, 13, 12, 2, 20, 14, 22, 9, 6, 1,
];

const RC: [u64; 24] = [
    0x0000000000000001,
    0x0000000000008082,
    0x800000000000808a,
    0x8000000080008000,
    0x000000000000808b,
    0x0000000080000001,
    0x8000000080008081,
    0x8000000000008009,
    0x000000000000008a,
    0x0000000000000088,
    0x0000000080008009,
    0x000000008000000a,
    0x000000008000808b,
    0x800000000000008b,
    0x8000000000008089,
    0x8000000000008003,
    0x8000000000008002,
    0x8000000000000080,
    0x000000000000800a,
    0x800000008000000a,
    0x8000000080008081,
    0x8000000000008080,
    0x0000000080000001,
    0x8000000080008008,
];

pub fn keccak_p(state: &mut [u64; 25]) {
    for rc in RC {
        let mut array = [0u64; 5];

        // Theta
        for x in 0..5 {
            for y in 0..5 {
                array[x] ^= state[5 * y + x];
            }
        }

        for x in 0..5 {
            let t1 = array[(x + 4) % 5];
            let t2 = array[(x + 1) % 5].rotate_left(1);
            for y in 0..5 {
                state[5 * y + x] ^= t1 ^ t2;
            }
        }

        // // Rho and pi
        let mut last: u64 = state[1];
        for x in 0..24 {
            array[0] = state[PI[x]];
            state[PI[x]] = last.rotate_left(RHO[x]);
            last = array[0];
        }

        // Chi
        for y_step in 0..5 {
            let y = 5 * y_step;

            array.copy_from_slice(&state[y..][..5]);

            for x in 0..5 {
                let t1 = !array[(x + 1) % 5];
                let t2 = array[(x + 2) % 5];
                state[y + x] = array[x] ^ (t1 & t2);
            }
        }

        // Iota
        state[0] ^= rc;
    }
}

#[cfg(test)]
mod tests {
    use std::hint::black_box;

    use super::*;
    use test::Bencher;

    #[bench]
    fn bench_keccak(b: &mut Bencher) {
        let mut state = [0; 25];
        b.iter(|| keccak_p(black_box(&mut state)));
    }
}

$ cargo +nightly-2025-04-04 bench
test tests::bench_keccak ... bench:         325.82 ns/iter (+/- 1.83)

$ RUSTFLAGS="-C target-cpu=znver4" cargo +nightly-2025-04-04 bench
test tests::bench_keccak ... bench:         543.16 ns/iter (+/- 2.74)

[nikic]

Godbolt: https://rust.godbolt.org/z/9PjcjEK9q

Looks like it's being vectorized now, presumably in some non-profitable way.

[apiraino]

Assigning priority (discussion on Zulip).

@rustbot label -I-prioritize +P-high -regression-untriaged -E-needs-mcve +regression-from-stable-to-nightly