Skip to content

Add AuTest for connect_attempts rr_retries and max_retries#12932

Draft
masaori335 wants to merge 4 commits intoapache:masterfrom
masaori335:asf-master-hostdb-autest
Draft

Add AuTest for connect_attempts rr_retries and max_retries#12932
masaori335 wants to merge 4 commits intoapache:masterfrom
masaori335:asf-master-hostdb-autest

Conversation

@masaori335
Copy link
Contributor

Covers combination of proxy.config.http.connect_attempts_rr_retries and proxy.config.http.connect_attempts_max_retries.

The connect_attempts_max_retries_down_server will be covered by #12922

@masaori335 masaori335 added this to the 11.0.0 milestone Mar 1, 2026
@masaori335 masaori335 self-assigned this Mar 1, 2026
@masaori335 masaori335 force-pushed the asf-master-hostdb-autest branch from ab39cfa to 3b069bc Compare March 2, 2026 01:41
@masaori335 masaori335 requested a review from Copilot March 2, 2026 01:41
@masaori335 masaori335 changed the title Add AuTest for connect_attempts Add AuTest for connect_attempts rr_retries and max_retries Mar 2, 2026
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds new AuTest coverage for origin connect-attempt retry behaviors (round-robin retries vs max retries) and extends the ATS replay test harness to support DNS record injection and error.log validation via replay YAML.

Changes:

  • Added three new Proxy Verifier replay scenarios covering proxy.config.http.connect_attempts_rr_retries and proxy.config.http.connect_attempts_max_retries.
  • Added new dns/connect_attempts.test.py to run the new replay tests and gold files for error.log expectations.
  • Extended tests/gold_tests/autest-site/ats_replay.test.ext to support DNS records injection and error_log validation (including gold-file based validation).

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 7 comments.

Show a summary per file
File Description
tests/gold_tests/dns/replay/connect_attempts_rr_retries.replay.yaml New replay scenario for round-robin retry behavior and down-server cache behavior.
tests/gold_tests/dns/replay/connect_attempts_rr_no_retry.replay.yaml New replay scenario covering baseline behavior with no retries.
tests/gold_tests/dns/replay/connect_attempts_rr_max_retries.replay.yaml New replay scenario covering connect_attempts_max_retries behavior.
tests/gold_tests/dns/gold/connect_attempts_rr_retries_error_log.gold Expected error.log output for rr-retries scenario.
tests/gold_tests/dns/gold/connect_attempts_rr_no_error_log.gold Expected error.log output for no-retry scenario.
tests/gold_tests/dns/gold/connect_attempts_rr_max_retries_error_log.gold Expected error.log output for max-retries scenario.
tests/gold_tests/dns/connect_attempts.test.py Test driver that runs the three new replay tests.
tests/gold_tests/autest-site/ats_replay.test.ext Adds error_log validation support and DNS records wiring for replay-based tests.

@bryancall bryancall added the DNS label Mar 2, 2026
@bryancall bryancall self-requested a review March 2, 2026 22:39
@masaori335
Copy link
Contributor Author

Somehow, the connect_attempts_rr_max_retries test is keep trying 0.0.0.1 even after it's marked "down" instead of round robin to 0.0.0.2 in this env.

20260302.01h50m26s CONNECT: attempt fail [CONNECTION_ERROR] to 0.0.0.1:62299 for host='example.com' connection_result=ETIMEDOUT [110] error=ETIMEDOUT [110] retry_attempts=0 url='http://backend.example.com:62299/path/'
20260302.01h50m28s CONNECT: attempt fail [CONNECTION_ERROR] to 0.0.0.1:62299 for host='example.com' connection_result=ETIMEDOUT [110] error=ETIMEDOUT [110] retry_attempts=1 url='http://backend.example.com:62299/path/'
20260302.01h50m30s CONNECT: attempt fail [CONNECTION_ERROR] to 0.0.0.1:62299 for host='example.com' connection_result=ETIMEDOUT [110] error=ETIMEDOUT [110] retry_attempts=2 url='http://backend.example.com:62299/path/'
20260302.01h50m30s CONNECT : ETIMEDOUT [110] connecting to 0.0.0.1:62299 for host='example.com' url='http://backend.example.com:62299/path/' fail_count='1' marking down
20260302.01h50m31s CONNECT: attempt fail [CONNECTION_ERROR] to 0.0.0.1:62299 for host='example.com' connection_result=ETIMEDOUT [110] error=ETIMEDOUT [110] retry_attempts=0 url='http://backend.example.com:62299/path/'
20260302.01h50m33s CONNECT: attempt fail [CONNECTION_ERROR] to 0.0.0.1:62299 for host='example.com' connection_result=ETIMEDOUT [110] error=ETIMEDOUT [110] retry_attempts=1 url='http://backend.example.com:62299/path/'
20260302.01h50m34s CONNECT: attempt fail [CONNECTION_ERROR] to 0.0.0.1:62299 for host='example.com' connection_result=ETIMEDOUT [110] error=ETIMEDOUT [110] retry_attempts=2 url='http://backend.example.com:62299/path/'

@masaori335 masaori335 marked this pull request as draft March 3, 2026 00:31
@masaori335
Copy link
Contributor Author

I found difference on my mac and docker env. We're hitting ETIMEDOUT [110]. This takes time and down_server.cache_time: 5 was too short. I extended it to 10 seconds.

An idea of speed up this test is adding some features to proxy-verifer to refuse connection immediately.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

3 participants