urtest: Convert AuTest tests to pytest

.github/copilot-instructions.md

@@ -313,7 +313,7 @@ trafficserver/
 **Pattern discovery:**
 - Search for similar functionality in existing code
 - Check `include/ts/ts.h` for plugin API patterns
-- Look at tests in `tests/gold_tests/` for usage examples
+- Look at tests in `tests/uranium_tests/` for usage examples
 
 ### Code Organization
 
@@ -362,7 +362,7 @@ plugins/my_plugin/
 
 **When adding new functionality:**
 1. Check if unit tests exist in same directory (Catch2)
-2. Add integration tests in `tests/gold_tests/` (autest)
+2. Add integration tests in `tests/uranium_tests/` (autest)
 3. Prefer `Test.ATSReplayTest()` with `replay.yaml` format (Proxy Verifier). If
    `ATSReplayTest` doesn't fit, prefer organizing the test around a test class with
    separate functions for configuring the servers, ATS, client, etc.

.gitignore

@@ -202,10 +202,10 @@ cmake-build-*
 tests/env-test/
 tests/proxy-verifier
 tests/Pipfile.lock
-tests/gold_tests/bigobj/check_ramp
-tests/gold_tests/bigobj/push_request
-tests/gold_tests/chunked_encoding/smuggle-client
-tests/gold_tests/tls/ssl-post
+tests/uranium_tests/bigobj/check_ramp
+tests/uranium_tests/bigobj/push_request
+tests/uranium_tests/chunked_encoding/smuggle-client
+tests/uranium_tests/tls/ssl-post
 
 src/iocore/cache/test_*
 src/iocore/cache/unit_tests/var/trafficserver/cache.db

AGENTS.md

@@ -11,15 +11,15 @@ with a sophisticated plugin system.
 **Key Technologies:**
 - Language: C++20
 - Build System: CMake (migrated from autotools in v10)
-- Testing: Catch2 (unit tests) + AuTest Python framework (end-to-end tests)
+- Testing: Catch2 (unit tests) + pytest (Uranium tests, with an AuTest compatibility backend)
 - Protocols: TLS, HTTP/1.1, HTTP/2, HTTP/3 (via Quiche)
 
 ## Project Structure
 
 Core sources live in `src/` (for example `src/proxy`, `src/iocore`,
 `src/traffic_server`). Public headers are in `include/`. Built-in plugins are
-in `plugins/` and `plugins/experimental/`. End-to-end tests are in `tests/`,
-especially `tests/gold_tests/`. Build system files are in `cmake/` plus the
+in `plugins/` and `plugins/experimental/`. Uranium tests are in `tests/`,
+especially `tests/uranium_tests/`. Build system files are in `cmake/` plus the
 top-level `CMakeLists.txt`, and docs are in `doc/`. Third party libraries that
 we include locally are in `lib/`.
 
@@ -80,61 +80,63 @@ Unit tests are built into executables. Find the test binary and run it directly:
 ./build/src/tscore/test_tscore
 ```
 
-### End-to-End Tests (AuTest)
+### Uranium Tests (pytest)
 
-**Enable autests during configuration:**
+**Enable Uranium tests during configuration:**
 ```bash
-cmake -B build -DENABLE_AUTEST=ON
+cmake -B build -DENABLE_URTEST=ON
 cmake --build build
 cmake --install build
 ```
 
-**Run all autests:**
+**Run all Uranium tests:**
 ```bash
-cmake --build build -t autest
+cmake --build build -t urtest
 ```
 
 **Run specific test(s):**
 ```bash
 cd build/tests
-./autest.sh --sandbox /tmp/sbcodex --clean=none -f <test_name_without_test_py>
+./urtest.sh -f <test_name_without_extension>
 ```
 
 For example, to run `cache-auth.test.py`:
 ```bash
-./autest.sh --sandbox /tmp/sbcursor --clean=none -f cache-auth
+./urtest.sh -f cache-auth
 ```
 
 To run multiple tests efficiently, pass the -j option.
 
 ```bash
 cd build/tests
-./autest.sh -j4 --sandbox /tmp/sbcodex --clean=none -f 'header_rewrite*'
+./urtest.sh -j4 -f 'header_rewrite*'
 ```
 
-Most end-to-end test coverage is in `tests/gold_tests/`. The CI system uses the
-Docker image `ci.trafficserver.apache.org/ats/fedora:43` (Fedora version updated
-regularly).
+Most Uranium test coverage is in `tests/uranium_tests/`. The CI system uses the
+Docker image `ci.trafficserver.apache.org/ats/fedora:44` (Fedora version updated
+regularly). The source-tree `tests/urtest.sh` defaults to this image. It runs
+directly instead when it detects that it is already inside a Fedora 44
+container. Use `--run-in-docker` or `--no-run-in-docker` to override.
 
-### Writing Autests
+### Writing Uranium Tests
 
-**New tests should use the `Test.ATSReplayTest()` approach**, which references a
-`replay.yaml` file that describes the test configuration and traffic patterns
-using the Proxy Verifier format. This is simpler, more maintainable, and
-parseable by tools.
+**New tests should normally be direct pytest replay tests.** Name the Proxy
+Verifier replay `<scenario>.test.yaml`; the file's `urtest` section describes
+DNS, server, client, and ATS setup, and pytest collects it without a companion
+`.test.py` wrapper. Run these with `cmake --build build -t urtest-replay`.
 
-If `ATSReplayTest` is not a good fit (say, the test needs a custom client), then
+If a direct replay test is not a good fit (say, the test needs a custom client), then
 organize the test around a test class with member functions that configure any
 servers, the ATS process, and the client. See
-`tests/gold_tests/ats_probe/ats_probe.test.py` for an example of a test organized
+`tests/uranium_tests/ats_probe/ats_probe.test.py` for an example of a test organized
 around a test class.
 
-In autests, launch Python helpers with `{sys.executable}` rather than a
+In compatibility tests, launch Python helpers with `{sys.executable}` rather than a
 hardcoded `python3`, so the test runs under the same interpreter the harness
 uses.
 
-**For complete details on writing autests, see:**
-- `doc/developer-guide/testing/autests.en.rst` - Comprehensive guide to autest
+**For complete details on writing Uranium tests, see:**
+- `doc/developer-guide/testing/uranium-tests.en.rst` - Comprehensive Uranium test guide
 - Proxy Verifier format: https://github.com/yahoo/proxy-verifier
 - AuTest framework: https://autestsuite.bitbucket.io/
 

CMakeLists.txt

@@ -161,8 +161,8 @@ option(ENABLE_DISK_FAILURE_TESTS "Build disk failure tests (enables AIO fault in
 if(ENABLE_DISK_FAILURE_TESTS)
   add_compile_definitions("AIO_FAULT_INJECTION")
 endif()
-option(ENABLE_AUTEST "Setup autest (default OFF)")
-option(ENABLE_AUTEST_UDS "Setup autest with curl using UDS (default OFF)")
+option(ENABLE_URTEST "Set up Uranium tests (default OFF)")
+option(ENABLE_URTEST_UDS "Set up Uranium tests with curl using UDS (default OFF)")
 option(ENABLE_BENCHMARKS "Build benchmarks (default OFF)")
 option(EXTERNAL_YAML_CPP "Use external yaml-cpp (default OFF)")
 option(EXTERNAL_LIBSWOC "Use external libswoc (default OFF)")
@@ -772,7 +772,7 @@ check_struct_has_member("struct mptcp_info" mptcpi_subflows "linux/mptcp.h" HAVE
 # find resolv library if available
 find_package(resolv)
 
-if(ENABLE_DOCS OR ENABLE_AUTEST)
+if(ENABLE_DOCS OR ENABLE_URTEST)
   find_package(Python3 REQUIRED)
   find_program(UV uv REQUIRED)
   find_program(NETCAT_PROGRAM nc REQUIRED)
@@ -788,27 +788,34 @@ if(ENABLE_DOCS)
   find_program(GRAPHVIZ_DOT dot REQUIRED)
 endif()
 
-if(ENABLE_AUTEST)
+if(ENABLE_URTEST)
   # Default the sandbox to /tmp to keep paths short. Unix domain socket paths
   # are limited to 108 characters and deep build directories (e.g. in home
   # directories) can exceed this limit, causing confusing test failures. A hash
   # of CMAKE_BINARY_DIR provides per-build isolation while keeping the path
   # deterministic across runs.
   string(MD5 _build_dir_hash "${CMAKE_BINARY_DIR}")
   string(SUBSTRING "${_build_dir_hash}" 0 8 _build_dir_hash)
-  set(AUTEST_SANDBOX
-      "/tmp/sb_${_build_dir_hash}"
-      CACHE STRING "Location for autest output (default /tmp/sb_<hash>)"
+  set(URTEST_SANDBOX
+      "/tmp/urtest_${_build_dir_hash}"
+      CACHE STRING "Location for Uranium test output (default /tmp/urtest_<hash>)"
   )
-  set(AUTEST_OPTIONS
+  if(AUTEST_SANDBOX)
+    message(DEPRECATION "AUTEST_SANDBOX is deprecated; use URTEST_SANDBOX")
+    set(URTEST_SANDBOX "${AUTEST_SANDBOX}")
+  endif()
+  set(URTEST_OPTIONS
       ""
-      CACHE STRING "Additional options for autest (default \"\")"
+      CACHE STRING "Additional options for Uranium tests (default \"\")"
   )
-  # Also create AUTEST_OPTIONS_LIST from the string for use in cmake targets.
-  # This prevents cmake from escaping spaces in the arguments, which confuses
-  # the autest command. The original AUTEST_OPTIONS string is used in the
-  # autest.sh script.
-  separate_arguments(AUTEST_OPTIONS_LIST UNIX_COMMAND "${AUTEST_OPTIONS}")
+  if(AUTEST_OPTIONS)
+    message(DEPRECATION "AUTEST_OPTIONS is deprecated; use URTEST_OPTIONS")
+    string(PREPEND URTEST_OPTIONS "${AUTEST_OPTIONS} ")
+  endif()
+  if(PYTEST_OPTIONS)
+    message(DEPRECATION "PYTEST_OPTIONS is deprecated; use URTEST_OPTIONS")
+    string(APPEND URTEST_OPTIONS " ${PYTEST_OPTIONS}")
+  endif()
   file(READ "${CMAKE_SOURCE_DIR}/tests/proxy-verifier-version.txt" PROXY_VERIFIER_VERSION)
   string(STRIP "${PROXY_VERIFIER_VERSION}" PROXY_VERIFIER_VERSION)
   file(READ "${CMAKE_SOURCE_DIR}/tests/proxy-verifier-checksum.txt" PROXY_VERIFIER_SHA1)
@@ -928,7 +935,7 @@ add_subdirectory(src/traffic_via)
 if(ENABLE_CRIPTS)
   add_subdirectory(src/cripts)
 endif()
-if(ENABLE_AUTEST)
+if(ENABLE_URTEST)
   add_subdirectory(tests)
 endif()
 if(ENABLE_FUZZING)

CMakePresets.json

@@ -52,25 +52,29 @@
       }
     },
     {
-      "name": "autest",
+      "name": "urtest",
+      "displayName": "Uranium tests",
+      "description": "Build and run the ATS Uranium test suite",
       "inherits": ["default"],
-      "binaryDir": "${sourceDir}/build-autest",
+      "binaryDir": "${sourceDir}/build-urtest",
       "cacheVariables": {
-        "ENABLE_AUTEST": "ON",
-        "CMAKE_INSTALL_PREFIX": "/tmp/ts-autest",
+        "ENABLE_URTEST": "ON",
+        "CMAKE_INSTALL_PREFIX": "/tmp/ts-urtest",
         "BUILD_EXPERIMENTAL_PLUGINS": "ON",
         "ENABLE_WASM_WAMR": "OFF",
         "ENABLE_EXAMPLE": "ON"
       }
     },
     {
-      "name": "autest-uds",
+      "name": "urtest-uds",
+      "displayName": "Uranium tests over Unix sockets",
+      "description": "Build and run the ATS Uranium test suite with curl over Unix sockets",
       "inherits": ["default"],
-      "binaryDir": "${sourceDir}/build-autest",
+      "binaryDir": "${sourceDir}/build-urtest",
       "cacheVariables": {
-        "ENABLE_AUTEST": "ON",
-        "ENABLE_AUTEST_UDS": "ON",
-        "CMAKE_INSTALL_PREFIX": "/tmp/ts-autest",
+        "ENABLE_URTEST": "ON",
+        "ENABLE_URTEST_UDS": "ON",
+        "CMAKE_INSTALL_PREFIX": "/tmp/ts-urtest",
         "BUILD_EXPERIMENTAL_PLUGINS": "ON",
         "ENABLE_EXAMPLE": "ON"
       }
@@ -225,10 +229,10 @@
       }
     },
     {
-      "name": "ci-fedora-autest",
-      "displayName": "CI Fedora Autest",
-      "description": "CI Pipeline config for Fedora Linux (autest build)",
-      "inherits": ["ci-fedora", "autest"]
+      "name": "ci-fedora-urtest",
+      "displayName": "CI Fedora Uranium tests",
+      "description": "CI pipeline configuration for the ATS Uranium test suite",
+      "inherits": ["ci-fedora", "urtest"]
     },
     {
       "name": "ci-freebsd",
@@ -337,7 +341,7 @@
       "displayName": "CI branch Quiche",
       "inherits": ["branch"],
       "cacheVariables": {
-        "ENABLE_AUTEST": "ON",
+        "ENABLE_URTEST": "ON",
         "nuraft_ROOT": "/opt/nuraft-boringssl",
         "OPENSSL_ROOT_DIR": "/opt/h3-tools-boringssl/boringssl",
         "quiche_ROOT": "/opt/h3-tools-boringssl/quiche",
@@ -349,7 +353,7 @@
       "displayName": "CI branch Quiche",
       "inherits": ["branch"],
       "cacheVariables": {
-        "ENABLE_AUTEST": "ON",
+        "ENABLE_URTEST": "ON",
         "nuraft_ROOT": "/opt",
         "OPENSSL_ROOT_DIR": "/opt/openssl-quic/",
         "quiche_ROOT": "/opt/quiche",
@@ -365,22 +369,22 @@
       }
     },
     {
-      "name": "branch-autest",
-      "displayName": "CI branch autest",
+      "name": "branch-urtest",
+      "displayName": "CI branch Uranium tests",
       "inherits": ["branch"],
       "cacheVariables": {
-        "ENABLE_AUTEST": "ON",
+        "ENABLE_URTEST": "ON",
         "ENABLE_EXAMPLE": "ON",
         "ENABLE_CRIPTS": "ON"
       }
     },
     {
-      "name": "branch-autest-uds",
-      "displayName": "CI branch autest",
+      "name": "branch-urtest-uds",
+      "displayName": "CI branch Uranium tests over Unix sockets",
       "inherits": ["branch"],
       "cacheVariables": {
-        "ENABLE_AUTEST": "ON",
-        "ENABLE_AUTEST_UDS": "ON",
+        "ENABLE_URTEST": "ON",
+        "ENABLE_URTEST_UDS": "ON",
         "ENABLE_EXAMPLE": "ON",
         "ENABLE_CRIPTS": "ON"
       }
@@ -441,7 +445,7 @@
       "name": "branch-coverage",
       "displayName": "CI branch coverage",
       "description": "Defaults for branch coverage builds",
-      "inherits": ["branch-autest"],
+      "inherits": ["branch-urtest"],
       "cacheVariables": {
         "CMAKE_CXX_FLAGS_DEBUG": "--coverage",
         "CMAKE_C_FLAGS_DEBUG": "--coverage"
@@ -450,9 +454,9 @@
   ],
   "buildPresets": [
     {
-      "name": "autest",
-      "configurePreset": "autest",
-      "targets": ["autest"]
+      "name": "urtest",
+      "configurePreset": "urtest",
+      "targets": ["urtest"]
     }
   ]
 }