ci: switch to ui-test instead of trybuild#5863
Open
davidhewitt wants to merge 36 commits into
Open
Conversation
davidhewitt
changed the title
ui-test instead of trybuildui-test instead of trybuild
Member
|
This looks interesting and does indeed seem to be quite a bit faster than While playing around with it locally I ended up adjusting things a bit. I attached the diff below, in case that is useful. Detailsdiff --git a/tests/test_compile_error.rs b/tests/test_compile_error.rs
index 22784c9df0..de83f6a7e1 100644
--- a/tests/test_compile_error.rs
+++ b/tests/test_compile_error.rs
@@ -82,6 +82,13 @@
// generic pyclasses only supported on 3.9+, doesn't fail gracefully on older versions
#[cfg(not(Py_3_9))]
"invalid_pyclass_generic.rs".into(),
+ // an extra "note" is emitted on abi3
+ #[cfg(any(not(Py_LIMITED_API), not(Py_3_12)))]
+ "invalid_base_class.rs".into(),
+ #[cfg(all(Py_LIMITED_API, not(Py_3_10)))]
+ "invalid_pyfunction_argument.rs".into(),
+ #[cfg(all(Py_LIMITED_API, not(Py_3_10)))]
+ "invalid_pyclass_args.rs".into(),
]);
match std::env::var("UI_TEST").as_deref() {
diff --git a/tests/ui/ambiguous_associated_items.rs b/tests/ui/ambiguous_associated_items.rs
index 3c9f4b0039..bd905fab99 100644
--- a/tests/ui/ambiguous_associated_items.rs
+++ b/tests/ui/ambiguous_associated_items.rs
@@ -1,4 +1,5 @@
//@check-pass
+#![allow(dead_code)]
use pyo3::prelude::*;
#[pyclass(eq)]
diff --git a/tests/ui/ambiguous_associated_items.stderr b/tests/ui/ambiguous_associated_items.stderr
deleted file mode 100644
index a8b30ca7fb..0000000000
--- a/tests/ui/ambiguous_associated_items.stderr
+++ /dev/null
@@ -1,15 +0,0 @@
-warning: enum `DeriveItems` is never used
- --> tests/ui/ambiguous_associated_items.rs:20:6
- |
-20 | enum DeriveItems {
- | ^^^^^^^^^^^
- |
- = note: `#[warn(dead_code)]` (part of `#[warn(unused)]`) on by default
-
-warning: enum `DeriveItemsFromPyObject` is never used
- --> tests/ui/ambiguous_associated_items.rs:43:6
- |
-43 | enum DeriveItemsFromPyObject {
- | ^^^^^^^^^^^^^^^^^^^^^^^
-
-warning: 2 warnings emitted
diff --git a/tests/ui/invalid_base_class.stderr b/tests/ui/invalid_base_class.stderr
index 5ee05a405a..7f6a85b2d6 100644
--- a/tests/ui/invalid_base_class.stderr
+++ b/tests/ui/invalid_base_class.stderr
@@ -6,6 +6,7 @@
|
= help: the trait `PyClassBaseType` is not implemented for `PyBool`
= note: `PyBool` must have `#[pyclass(subclass)]` to be eligible for subclassing
+ = note: with the `abi3` feature enabled, PyO3 does not support subclassing native types
= help: the following other types implement trait `PyClassBaseType`:
PyAny
PyArithmeticError
@@ -15,7 +16,7 @@
PyBaseExceptionGroup
PyBlockingIOError
PyBrokenPipeError
- and 69 others
+ and 63 others
= note: this error originates in the attribute macro `pyclass` (in Nightly builds, run with -Z macro-backtrace for more info)
error[E0277]: pyclass `PyBool` cannot be subclassed
@@ -26,6 +27,7 @@
|
= help: the trait `PyClassBaseType` is not implemented for `PyBool`
= note: `PyBool` must have `#[pyclass(subclass)]` to be eligible for subclassing
+ = note: with the `abi3` feature enabled, PyO3 does not support subclassing native types
= help: the following other types implement trait `PyClassBaseType`:
PyAny
PyArithmeticError
@@ -35,7 +37,7 @@
PyBaseExceptionGroup
PyBlockingIOError
PyBrokenPipeError
- and 69 others
+ and 63 others
note: required by a bound in `PyClassImpl::BaseType`
--> src/impl_/pyclass.rs:184:33
|
diff --git a/tests/ui/invalid_property_args.stderr b/tests/ui/invalid_property_args.stderr
index e2c719babd..f819a76271 100644
--- a/tests/ui/invalid_property_args.stderr
+++ b/tests/ui/invalid_property_args.stderr
@@ -66,12 +66,12 @@
and 151 others
= note: required for `PhantomData<i32>` to implement `for<'py> PyO3GetField<'py>`
note: required by a bound in `PyClassGetterGenerator::<ClassT, FieldT, OFFSET, false, false, IMPLEMENTS_INTOPYOBJECT>::generate`
- --> src/impl_/pyclass.rs:1328:26
+ --> src/impl_/pyclass.rs:1297:26
|
-1324 | pub const fn generate(&self, name: &'static CStr, doc: Option<&'static CStr>) -> PyMethodDefType
+1293 | pub const fn generate(&self, name: &'static CStr, doc: Option<&'static CStr>) -> PyMethodDefType
| -------- required by a bound in this associated function
...
-1328 | for<'py> FieldT: PyO3GetField<'py>,
+1297 | for<'py> FieldT: PyO3GetField<'py>,
| ^^^^^^^^^^^^^^^^^ required by this bound in `PyClassGetterGenerator::<ClassT, FieldT, OFFSET, false, false, IMPLEMENTS_INTOPYOBJECT>::generate`
error: aborting due to 9 previous errors
diff --git a/tests/ui/not_send.rs b/tests/ui/not_send.rs
index d93d605327..6ff785d463 100644
--- a/tests/ui/not_send.rs
+++ b/tests/ui/not_send.rs
@@ -1,8 +1,9 @@
+//@normalize-stderr-test: ".*/src/rust/(.*)" -> "../src/$1"
use pyo3::prelude::*;
fn test_not_send_detach(py: Python<'_>) {
- py.detach(|| { drop(py); });
-//~^ ERROR: `*mut pyo3::Python<'static>` cannot be shared between threads safely
+ py.detach(|| drop(py));
+ //~^ ERROR: `*mut pyo3::Python<'static>` cannot be shared between threads safely
}
fn main() {
diff --git a/tests/ui/not_send.stderr b/tests/ui/not_send.stderr
index 837f0a6963..81ae40120a 100644
--- a/tests/ui/not_send.stderr
+++ b/tests/ui/not_send.stderr
@@ -1,14 +1,14 @@
error[E0277]: `*mut pyo3::Python<'static>` cannot be shared between threads safely
- --> tests/ui/not_send.rs:4:15
+ --> tests/ui/not_send.rs:5:15
|
- 4 | py.detach(|| { drop(py); });
- | ------ ^^^^^^^^^^^^^^^^ `*mut pyo3::Python<'static>` cannot be shared between threads safely
+ 5 | py.detach(|| drop(py));
+ | ------ ^^^^^^^^^^^ `*mut pyo3::Python<'static>` cannot be shared between threads safely
| |
| required by a bound introduced by this call
|
= help: within `pyo3::Python<'_>`, the trait `Sync` is not implemented for `*mut pyo3::Python<'static>`
note: required because it appears within the type `PhantomData<*mut pyo3::Python<'static>>`
- --> /Users/david/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/src/rust/library/core/src/marker.rs:819:12
+../src/library/core/src/marker.rs:819:12
|
819 | pub struct PhantomData<T: PointeeSized>;
| ^^^^^^^^^^^
@@ -18,7 +18,7 @@
357 | struct NotSend(PhantomData<*mut Python<'static>>);
| ^^^^^^^
note: required because it appears within the type `PhantomData<pyo3::marker::NotSend>`
- --> /Users/david/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/src/rust/library/core/src/marker.rs:819:12
+../src/library/core/src/marker.rs:819:12
|
819 | pub struct PhantomData<T: PointeeSized>;
| ^^^^^^^^^^^
@@ -29,11 +29,11 @@
| ^^^^^^
= note: required for `&pyo3::Python<'_>` to implement `Send`
note: required because it's used within this closure
- --> tests/ui/not_send.rs:4:15
+ --> tests/ui/not_send.rs:5:15
|
- 4 | py.detach(|| { drop(py); });
+ 5 | py.detach(|| drop(py));
| ^^
- = note: required for `{closure@tests/ui/not_send.rs:4:15: 4:17}` to implement `Ungil`
+ = note: required for `{closure@tests/ui/not_send.rs:5:15: 5:17}` to implement `Ungil`
note: required by a bound in `pyo3::Python::<'py>::detach`
--> src/marker.rs:560:12
|
diff --git a/tests/ui/not_send2.rs b/tests/ui/not_send2.rs
index b1d3262ead..58382a7c20 100644
--- a/tests/ui/not_send2.rs
+++ b/tests/ui/not_send2.rs
@@ -1,3 +1,4 @@
+//@normalize-stderr-test: ".*/src/rust/(.*)" -> "../src/$1"
use pyo3::prelude::*;
use pyo3::types::PyString;
@@ -6,7 +7,7 @@
let string = PyString::new(py, "foo");
py.detach(|| {
-//~^ ERROR: `*mut pyo3::Python<'static>` cannot be shared between threads safely
+ //~^ ERROR: `*mut pyo3::Python<'static>` cannot be shared between threads safely
println!("{:?}", string);
});
});
diff --git a/tests/ui/not_send2.stderr b/tests/ui/not_send2.stderr
index 6921905be4..c69132e46b 100644
--- a/tests/ui/not_send2.stderr
+++ b/tests/ui/not_send2.stderr
@@ -1,18 +1,18 @@
error[E0277]: `*mut pyo3::Python<'static>` cannot be shared between threads safely
- --> tests/ui/not_send2.rs:8:19
+ --> tests/ui/not_send2.rs:9:19
|
- 8 | py.detach(|| {
+ 9 | py.detach(|| {
| ____________------_^
| | |
| | required by a bound introduced by this call
- 9 | |
- 10 | | println!("{:?}", string);
- 11 | | });
+ 10 | |
+ 11 | | println!("{:?}", string);
+ 12 | | });
| |_________^ `*mut pyo3::Python<'static>` cannot be shared between threads safely
|
= help: within `pyo3::Bound<'_, PyString>`, the trait `Sync` is not implemented for `*mut pyo3::Python<'static>`
note: required because it appears within the type `PhantomData<*mut pyo3::Python<'static>>`
- --> /Users/david/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/src/rust/library/core/src/marker.rs:819:12
+../src/library/core/src/marker.rs:819:12
|
819 | pub struct PhantomData<T: PointeeSized>;
| ^^^^^^^^^^^
@@ -22,7 +22,7 @@
357 | struct NotSend(PhantomData<*mut Python<'static>>);
| ^^^^^^^
note: required because it appears within the type `PhantomData<pyo3::marker::NotSend>`
- --> /Users/david/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/src/rust/library/core/src/marker.rs:819:12
+../src/library/core/src/marker.rs:819:12
|
819 | pub struct PhantomData<T: PointeeSized>;
| ^^^^^^^^^^^
@@ -38,11 +38,11 @@
| ^^^^^
= note: required for `&pyo3::Bound<'_, PyString>` to implement `Send`
note: required because it's used within this closure
- --> tests/ui/not_send2.rs:8:19
+ --> tests/ui/not_send2.rs:9:19
|
- 8 | py.detach(|| {
+ 9 | py.detach(|| {
| ^^
- = note: required for `{closure@tests/ui/not_send2.rs:8:19: 8:21}` to implement `Ungil`
+ = note: required for `{closure@tests/ui/not_send2.rs:9:19: 9:21}` to implement `Ungil`
note: required by a bound in `pyo3::Python::<'py>::detach`
--> src/marker.rs:560:12
|
|
Co-authored-by: Icxolu <10486322+Icxolu@users.noreply.github.com>
Member
Author
|
Thanks! Yes agreed, RE coverage: I might be wrong, but because the ui tests get compiled as part of the workspace under |
davidhewitt
added
the
CI-no-fail-fast
Merged
Member
Author
|
Note to self: #5807 (comment) |
Closed
Icxolu
reviewed
May 25, 2026
Comment on lines
+143
to
+152
| #[cfg(feature = "experimental-inspect")] | ||
| config.skip_files.extend([ | ||
| // extra error messages appear due to additional macro processing | ||
| // would be nice to somehow make this not a problem | ||
| "duplicate_pymodule_submodule.rs".into(), | ||
| "missing_intopy.rs".into(), | ||
| "invalid_pyclass_args.rs".into(), | ||
| "invalid_property_args.rs".into(), | ||
| "invalid_pyfunction_argument.rs".into(), | ||
| ]); |
Member
There was a problem hiding this comment.
I think these are still never run, as we currently always enable experimental-inspect. Given that this makes the test faster, maybe it would be worth to introduce a dedicated job which just runs the ui tests for each feature (only one platform with latest Python or something), so that we minimize missing any?
Member
There was a problem hiding this comment.
I haven't looked at ui test but I bet there is some kind of rule or config to make this problem go away.
Member
Author
|
I think this is now working nicely! Please excuse the huge diff; most of the lines are due to changes in normalization of the test output (plus adding the diagnostics inline in the It looks like despite adding additional UI test runs for Remaining failure is unrelated and fixed by #6077 |
mejrs
approved these changes
May 28, 2026
Member
mejrs
left a comment
mejrs
left a comment
There was a problem hiding this comment.
Thanks, just some minor comments.
Comment on lines
+5
to
+6
| //~^ error: `PyException` cannot be subclassed | ||
| //~| error: `PyException` cannot be subclassed |
Member
There was a problem hiding this comment.
Suggested change
| //~^ error: `PyException` cannot be subclassed | |
| //~| error: `PyException` cannot be subclassed | |
| //~^ ERROR: `PyException` cannot be subclassed | |
| //~| ERROR: `PyException` cannot be subclassed |
Comment on lines
+6
to
+7
| //~^ error: `PyDict` cannot be subclassed | ||
| //~| error: `PyDict` cannot be subclassed |
Member
There was a problem hiding this comment.
Suggested change
| //~^ error: `PyDict` cannot be subclassed | |
| //~| error: `PyDict` cannot be subclassed | |
| //~^ ERROR: `PyDict` cannot be subclassed | |
| //~| ERROR: `PyDict` cannot be subclassed |
| | / impl<'a, 'holder, T: PyClass<Frozen = False>> PyFunctionArgument<'a, 'holder, '_, false> | ||
| | | for &'holder mut T | ||
| | |______________________^ `&'holder mut T` implements `pyo3::impl_::extract_argument::PyFunctionArgument<'a, 'holder, '_, false>` | ||
|
|
Member
There was a problem hiding this comment.
This message is kinda unfortunate, can we use #[diagnostic::do_not_recommend] on those impls?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
I had some conversations at RustNation which inspired me to look into
ui-testinstead oftrybuildfor our UI testing. It seems pretty good on first glance:check-pass)trybuildin my testing this morning.Pushing to get feedback in CI, I am optimistic this might be a nice improvement.