Add dist step for enzyme, including aarch64-apple-darwin

[sgasho]

• almost identical to Add dist step for Enzyme #150071

I cherry-picked commits from #150071 then added a commit below to enable the production of an enzyme artifact for aarch64-apple-darwin.

set llvm.enzyme=true on dist-aarch64-apple

background: #145899 (comment)

I've been looking into the dlopen problem and I need an artifact for aarch64 for me, not only for x86_64.

@Kobzol @ZuseZ4
Could you...

1. check the commit set llvm.enzyme=true on dist-aarch64-apple if that changes are appropriate?
2. delegate try rights for me on this PR?

I'll try to run "try-job dist-aarch64-apple" after delegation.

[ZuseZ4]

@bors delegate try

[rust-bors]

Unknown command "delegate". Run @bors help to see available commands.

[ZuseZ4]

Hmm, not sure about the exact syntax, but delegate itself should have been recognized, maybe not ported to the new bots yet? I'll let Jakub handle it.
For now

@bors try jobs=dist-aarch64-apple

[Kobzol]

@bors delegate=try

[rust-bors]

✌️ @sgasho, you can now perform try builds on this pull request!

You can now post @bors try to start a try build.

[rust-bors]

☀️ Try build successful (CI)
Build commit: a658070 (a65807068ca7fef2605b24756206b9e5dfa49bbd, parent: db1484bdee1cb065b322a5bd512b39c49dadc9e5)

[rust-bors]

☔ The latest upstream changes (presumably #150541) made this pull request unmergeable. Please resolve the merge conflicts.

[sgasho]

@bors try jobs=dist-aarch64-apple

[sgasho]

@bors try jobs=dist-aarch64-apple

[sgasho]

@bors try jobs=dist-aarch64-apple

[rust-bors]

💔 Test for 2c9be7e failed: CI. Failed job:

• try - dist-aarch64-apple (web logs, enhanced plaintext logs)

[ZuseZ4]

@bors retry

[rust-bors]

❗ You can only retry pull requests that are approved and have a previously failed auto build.

[sgasho]

I realized that we cannot use export-symbols immediately; we have to expand it to use it for dylib.
I'll make some extra commits, then create CI artifacts.

[sgasho]

I have updated this branch because #150992 has been merged into main. I'll expand it so that it can be used for dylib.

[sgasho]

@bors try jobs=dist-aarch64-apple

[rust-bors]

💔 Test for 094255a failed: CI

[ZuseZ4]

@bors try jobs=dist-aarch64-apple

[rust-bors]

☀️ Try build successful (CI)
Build commit: 6d0c1c7 (6d0c1c7f8aeee86ea677552367208f666f622c41, parent: bb8b30a5fce59f5d3d17a8f010601c59f0f19d79)

[sgasho]

I verified that the simple autodiff code below works with llvm.link-shared=true.
Next, I'm trying to expand the export-symbols flag so that we can use autodiff on macOS with statically linked LLVM.

#![feature(autodiff)]
use std::autodiff::*;

// f(x) = x * x, f'(x) = 2.0 * x
// bar therefore returns (x * x, 2.0 * x)
#[autodiff_reverse(bar, Active, Active)]
fn foo(x: f32) -> f32 {
    x * x
}

fn main() {
    assert_eq!(bar(3.0, 1.0), (9.0, 6.0));
    assert_eq!(bar(4.0, 1.0), (16.0, 8.0));
}

[sgasho]

Current stable rustc exports 436 llvm symbols.

nm -gU /Users/sgasho/.rustup/toolchains/stable-aarch64-apple-darwin/lib/librustc_driver-015fbde41a4573ae.dylib | grep -i "LLVM" | wc -l
     436

If we implement #151243, its result would be 40k

nm -gU ~/Downloads/rust-nightly-aarch64-apple-darwin\ 2/rustc/lib/librustc_driver-87e8ef082ced32c9.dylib | grep -i "LLVM" | wc -l
   43404

All symbols in rustc_driver would be 60k

nm -gU ~/Downloads/rust-nightly-aarch64-apple-darwin\ 2/rustc/lib/librustc_driver-87e8ef082ced32c9.dylib | wc -l 
   64141

Undefined symbols for libEnzyme

nm -u ~/Desktop/enz-ci/lib/libEnzyme-22.dylib | grep -i "LLVM" | wc -l
     860

I have not found the good solution to export only what libEnzyme needs in a reasonable way.
I'm going to explore it step by step, narrowing the symbols to export.
On the other hand, I don't have a strong opinion of whether we should limit what symbols to export, I mean, I don't know the critical problem of exporting 60k~ symbols in rustc_driver. I know it's not a clean way though.

So, for the first step, I implemented export-all-symbols and I'll see if it works fine. I think I could expand export-symbols but I did not have enough time to implement it today...

@bors try jobs=dist-aarch64-apple

[rust-bors]

☀️ Try build successful (CI)
Build commit: c2aef13 (c2aef135c60223aee3244fd33b97639eae6fca97, parent: 71e00273c0921e1bc850ae8cc4161fbb44cfa848)