x86: Correctly pass large integers and structs in registers with -Zregparm (v2) - #161027
x86: Correctly pass large integers and structs in registers with -Zregparm (v2)#161027sulix wants to merge 1 commit into
Conversation
…gparm The -Zregparm option for 32-bit x86 modifies the calling convention to pass a number of arguments in registers instead of on the stack. (This is primarily used by the Linux kernel.) Currently, rustc will only pass integers of size <= 32 bits in registers, but gcc (and clang) will pass larger integers (e.g., 64-bit integers) across two registers if possible. This is not the case with fastcall/vectorcall. Equally, struct arguments will be split into registers and passed where possible. Supporting this in rustc required moving the 'inreg' determination into compute_abi_info, as this is the stage where we determine if structs are passed directly or indirectly. This involves updating the x86_win32 ABI implementation as well, as it previously shared the inreg handling, but has a different compute_abi_info implementation. Also add some assembly-llvm and codegen-llvm tests. Note that the LLVM IR generated doesn't exactly match clang, as rustc treats struct arguments as one argument in three registers, whereas clang seems to decompose it into three registers. It seems to work regardless...
rustbot
added
S-waiting-on-review
|
Thanks for the pull request, and welcome! The Rust Project is excited to review your changes, and you should hear from @mu001999 (or someone else) some time within the next two weeks. Please see the contribution instructions for more information. Namely, in order to ensure the minimum review times lag, PR authors and assigned reviewers should ensure that the review label (
Why was this reviewer chosen?The reviewer was selected based on:
|
|
@rustbot reroll |
|
I believe @folkertdev and @beetrees have been working on this code recently |
|
I haven't looked into the details, what are you basing the implementation on? I think I've found two issues with the My interpretation is that we have 3 32-bit registers available. Here we use up 2 registers for the first https://godbolt.org/z/z7zeGG3P1 define dso_local noundef i32 @callee(i64 inreg noundef %a, i64 noundef %b, i32 noundef returned %tail) local_unnamed_addr {
entry:
ret i32 %tail
}versus this PR (with define noundef i32 @callee(i64 inreg noundef %_a, i64 noundef %_b, i32 inreg noundef returned %tail) unnamed_addr #0 !guid !4 {
start:
ret i32 %tail
}The Another one, less relevant for linux but this flag should work in general: structs containing a single float are coerced to integer registers: clang does not do that. https://godbolt.org/z/56Pne7dEE define dso_local noundef i32 @callee_f1(float %s.0, i32 inreg noundef returned %tail) local_unnamed_addr {
entry:
ret i32 %tail
}
define dso_local noundef i32 @callee_d1(double %s.0, i32 inreg noundef returned %tail) local_unnamed_addr {
entry:
ret i32 %tail
}
define dso_local noundef i32 @callee_f2(i32 inreg %s.coerce0, i32 inreg %s.coerce1, i32 inreg noundef returned %tail) local_unnamed_addr {
entry:
ret i32 %tail
}versus this PR ; Function Attrs: mustprogress nofree norecurse nosync nounwind willreturn memory(none) uwtable
define noundef i32 @callee_d1([2 x i32] inreg %0, i32 inreg noundef returned %tail) unnamed_addr #0 !guid !4 {
start:
ret i32 %tail
}
; Function Attrs: mustprogress nofree norecurse nosync nounwind willreturn memory(none) uwtable
define noundef i32 @callee_f1(i32 inreg %0, i32 inreg noundef returned %tail) unnamed_addr #0 !guid !5 {
start:
ret i32 %tail
}
; Function Attrs: mustprogress nofree norecurse nosync nounwind willreturn memory(none) uwtable
define noundef i32 @callee_f2([2 x i32] inreg %0, i32 inreg noundef returned %tail) unnamed_addr #0 !guid !6 {
start:
ret i32 %tail
} |
There was a problem hiding this comment.
not sure where this should go but we should add a c-variadic example too
There was a problem hiding this comment.
Per the GCC regparm docs C-variadic functions have all arguments passed on the stack (this is missing from the current implementation).
There was a problem hiding this comment.
It might be worth test your implementation against Clang and GCC with abi-cafe if you can - it's quite good and finding places where ABI implementations don't match.
| } | ||
|
|
||
| if arg.layout.pass_indirectly_in_non_rustic_abis(cx) { | ||
| arg.make_indirect(); |
There was a problem hiding this comment.
This indirect pointer also needs to have inreg set on it as appropriate.
|
|
||
| #[derive(PartialEq)] | ||
| pub(crate) enum Flavor { | ||
| General, |
There was a problem hiding this comment.
| General { regparm: u32 }, |
Not added in the PR, but since only Flavor::General uses regparm, I think it would be better to store it directly in the enum variant.
There was a problem hiding this comment.
Per the GCC regparm docs C-variadic functions have all arguments passed on the stack (this is missing from the current implementation).
| } | ||
|
|
||
| for arg in fn_abi.args.iter_mut() { | ||
| if arg.is_ignore() || !arg.layout.is_sized() { |
There was a problem hiding this comment.
As the body of this for loop is growing, I think it would make sense to split the loop body into a classify_arg, similar to other files in this module.
That's how I found those two examples. I guess on linux it might work to cross-compile for i686, but I also have a fork (I do hope to get this upstreamed eventually) with a version that can cross-compile https://github.com/folkertdev/abi-cafe#cross-compilation That's been finding all sorts of ABI inconsistencies. |
|
I think probably r? @folkertdev |
7 participants
[This is a new version of #147628.]
Fixes#145694. I think this should be the last compiler fix needed to unblock 32-bit x86 support for Rust-for-linux (Rust-for-Linux/linux#78)
Tracking issue for -Zregparm: #131749
The -Zregparm option for 32-bit x86 modifies the calling convention to pass a number of arguments in registers instead of on the stack. (This is primarily used by the Linux kernel.)
Currently, rustc will only pass integers of size <= 32 bits in registers, but gcc (and clang) will pass larger integers (e.g., 64-bit integers) across two registers if possible. This is not the case with fastcall/vectorcall.
Equally, struct arguments will be split into registers and passed where possible.
Supporting this in rustc required moving the 'inreg' determination into compute_abi_info, as this is the stage where we determine if structs are passed directly or indirectly. This involves updating the x86_win32 ABI implementation as well, as it previously shared the inreg handling, but has a different compute_abi_info implementation.
Also add some assembly-llvm and codegen-llvm tests. Note that the LLVM IR generated doesn't exactly match clang, as rustc treats struct arguments as one argument in three registers, whereas clang seems to decompose it into three registers. It seems to work regardless...
CC: @tgross35