Skip to content

Commit f269e43

Browse files
authored
Merge pull request #438 from Integration-Automation/feat/adaptive-timeout-batch
Add adaptive_timeout: derive a wait timeout from observed durations
2 parents 59ac066 + 7c0af83 commit f269e43

11 files changed

Lines changed: 380 additions & 0 deletions

File tree

WHATS_NEW.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,12 @@
22

33
## What's new (2026-06-26)
44

5+
### Adaptive Timeout from Observed Durations
6+
7+
Stop guessing wait timeouts — learn them from how long the step actually takes. Full reference: [`docs/source/Eng/doc/new_features/v211_features_doc.rst`](docs/source/Eng/doc/new_features/v211_features_doc.rst).
8+
9+
- **`recommend_timeout` / `timeout_stats`** (`AC_adaptive_timeout`, `AC_timeout_stats`): hard-coded waits are a perennial flakiness source — too short races a slow machine, too long makes every failure pay the full timeout. This learns the timeout from observed step durations: a high percentile (the slow-but-real case) scaled by a safety `factor`, clamped to a sane `[min_s, max_s]` band. `recommend_timeout` is the single number to feed a `wait_for_*` / actionability `GateConfig`; `timeout_stats` also exposes the percentiles and `floored`/`capped` flags for tuning. Both are pure and reuse `stats.percentile`; with no samples they fall back to `default_s`. Third feature of the ROUND-15 input-fidelity lane. No `PySide6`.
10+
511
### Verify a Field After Typing
612

713
Read the field back and confirm the value actually landed — don't type and hope. Full reference: [`docs/source/Eng/doc/new_features/v210_features_doc.rst`](docs/source/Eng/doc/new_features/v210_features_doc.rst).
Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
Adaptive Timeout from Observed Durations
2+
========================================
3+
4+
Hard-coded waits are a perennial source of flakiness: too short and a slow
5+
machine races the UI; too long and every failure pays the full timeout. The
6+
durable fix is to *learn* the timeout from how long a step has actually taken.
7+
``adaptive_timeout`` turns a sample of observed durations into a robust timeout
8+
— a high percentile (the slow-but-real case) scaled by a safety ``factor``,
9+
then clamped to a sane ``[min_s, max_s]`` band.
10+
11+
* :func:`recommend_timeout` — the single number to feed a wait or ``GateConfig``.
12+
* :func:`timeout_stats` — the same with the percentiles and clamp flags exposed
13+
for logging / tuning.
14+
15+
Both are pure and reuse :func:`stats.percentile`; with no samples they fall back
16+
to ``default_s`` (or ``min_s``). Imports no ``PySide6``.
17+
18+
Headless API
19+
------------
20+
21+
.. code-block:: python
22+
23+
from je_auto_control import recommend_timeout, timeout_stats
24+
25+
# The dialog has historically taken these seconds to appear:
26+
seen = [0.8, 1.1, 0.9, 3.2, 1.0, 1.3]
27+
28+
recommend_timeout(seen) # ~ p95 * 1.5, clamped to [1, 60]
29+
recommend_timeout(seen, percentile_q=99.0, factor=2.0, max_s=30.0)
30+
31+
timeout_stats(seen)
32+
# {'n': 6, 'p50': 1.05, 'p_high': 2.7..., 'percentile_q': 95.0,
33+
# 'recommended': 4.1..., 'floored': False, 'capped': False}
34+
35+
Use the recommendation as the ``timeout_s`` for the next ``wait_for_*`` /
36+
actionability gate, recomputing it as the duration sample grows. With no samples
37+
yet, pass ``default_s`` for the cold-start value.
38+
39+
Executor commands
40+
-----------------
41+
42+
``AC_adaptive_timeout`` (``durations`` + ``percentile_q`` / ``factor`` /
43+
``min_s`` / ``max_s`` → ``{timeout_s}``) and ``AC_timeout_stats`` (same inputs →
44+
``{n, p50, p_high, percentile_q, recommended, floored, capped}``). ``durations``
45+
accepts a JSON list. They are the matching read-only ``ac_*`` MCP tools and
46+
Script Builder commands under **Flow**.
Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
依觀測時長自適應逾時
2+
====================
3+
4+
寫死的等待是長年的不穩定來源:太短則慢機器與 UI 競速;太長則每次失敗都得付滿整個逾時。可長久的修法是
5+
從某步驟*實際*花了多久來*學習*逾時。``adaptive_timeout`` 把一組觀測時長轉為穩健的逾時——取高百分位
6+
(慢但真實的情況)乘上安全 ``factor``,再夾到合理的 ``[min_s, max_s]`` 區間。
7+
8+
* :func:`recommend_timeout` ——餵給等待或 ``GateConfig`` 的單一數值。
9+
* :func:`timeout_stats` ——同上,但額外暴露百分位與夾值旗標以利記錄 / 調校。
10+
11+
兩者皆為純函式並重用 :func:`stats.percentile`;沒有樣本時退回 ``default_s``(或 ``min_s``)。
12+
不匯入 ``PySide6``。
13+
14+
無頭 API
15+
--------
16+
17+
.. code-block:: python
18+
19+
from je_auto_control import recommend_timeout, timeout_stats
20+
21+
# 對話框歷來出現所花的秒數:
22+
seen = [0.8, 1.1, 0.9, 3.2, 1.0, 1.3]
23+
24+
recommend_timeout(seen) # 約 p95 * 1.5,夾到 [1, 60]
25+
recommend_timeout(seen, percentile_q=99.0, factor=2.0, max_s=30.0)
26+
27+
timeout_stats(seen)
28+
# {'n': 6, 'p50': 1.05, 'p_high': 2.7..., 'percentile_q': 95.0,
29+
# 'recommended': 4.1..., 'floored': False, 'capped': False}
30+
31+
把建議值當作下一個 ``wait_for_*`` / actionability 閘的 ``timeout_s``,並隨樣本增長重新計算。
32+
尚無樣本時,以 ``default_s`` 作為冷啟動值。
33+
34+
執行器指令
35+
----------
36+
37+
``AC_adaptive_timeout``(``durations`` 加上 ``percentile_q`` / ``factor`` /
38+
``min_s`` / ``max_s`` → ``{timeout_s}``)與 ``AC_timeout_stats``(同樣輸入 →
39+
``{n, p50, p_high, percentile_q, recommended, floored, capped}``)。``durations``
40+
接受 JSON 清單。皆以對應的唯讀 ``ac_*`` MCP 工具及 Script Builder 指令(位於 **Flow** 分類下)形式提供。

je_auto_control/__init__.py

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -123,6 +123,10 @@
123123
from je_auto_control.utils.verify_field import (
124124
compare_field_value, fill_and_verify, verify_field_value,
125125
)
126+
# Derive a wait timeout from observed step durations
127+
from je_auto_control.utils.adaptive_timeout import (
128+
recommend_timeout, timeout_stats,
129+
)
126130
# Rich clipboard formats — RTF + CSV/TSV codecs and Windows get / set
127131
from je_auto_control.utils.clipboard_rich_formats import (
128132
build_rtf, csv_to_rows, get_clipboard_csv, get_clipboard_rtf, rows_to_csv,
@@ -1744,6 +1748,7 @@ def start_autocontrol_gui(*args, **kwargs):
17441748
"decode_conversion_mode",
17451749
"RetryBudget", "run_with_budget", "backoff_delay", "jittered_delay",
17461750
"compare_field_value", "verify_field_value", "fill_and_verify",
1751+
"recommend_timeout", "timeout_stats",
17471752
"build_rtf", "rtf_to_text", "rows_to_csv", "csv_to_rows",
17481753
"set_clipboard_rtf", "get_clipboard_rtf",
17491754
"set_clipboard_csv", "get_clipboard_csv",

je_auto_control/gui/script_builder/command_schema.py

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4458,6 +4458,32 @@ def _add_work_queue_specs(specs: List[CommandSpec]) -> None:
44584458
),
44594459
description="Read a control's value back and confirm it equals expected.",
44604460
))
4461+
specs.append(CommandSpec(
4462+
"AC_adaptive_timeout", "Flow", "Adaptive Timeout",
4463+
fields=(
4464+
FieldSpec("durations", FieldType.STRING,
4465+
placeholder="JSON list of durations (seconds)"),
4466+
FieldSpec("percentile_q", FieldType.FLOAT, optional=True,
4467+
default=95.0),
4468+
FieldSpec("factor", FieldType.FLOAT, optional=True, default=1.5),
4469+
FieldSpec("min_s", FieldType.FLOAT, optional=True, default=1.0),
4470+
FieldSpec("max_s", FieldType.FLOAT, optional=True, default=60.0),
4471+
),
4472+
description="Recommend a wait timeout from observed step durations.",
4473+
))
4474+
specs.append(CommandSpec(
4475+
"AC_timeout_stats", "Flow", "Timeout Stats",
4476+
fields=(
4477+
FieldSpec("durations", FieldType.STRING,
4478+
placeholder="JSON list of durations (seconds)"),
4479+
FieldSpec("percentile_q", FieldType.FLOAT, optional=True,
4480+
default=95.0),
4481+
FieldSpec("factor", FieldType.FLOAT, optional=True, default=1.5),
4482+
FieldSpec("min_s", FieldType.FLOAT, optional=True, default=1.0),
4483+
FieldSpec("max_s", FieldType.FLOAT, optional=True, default=60.0),
4484+
),
4485+
description="Timeout recommendation plus percentiles and clamp flags.",
4486+
))
44614487
specs.append(CommandSpec(
44624488
"AC_normalize_ext", "Shell", "Normalize Extension",
44634489
fields=(
Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
"""Derive a wait timeout from observed step durations instead of guessing."""
2+
from je_auto_control.utils.adaptive_timeout.adaptive_timeout import (
3+
recommend_timeout, timeout_stats,
4+
)
5+
6+
__all__ = ["recommend_timeout", "timeout_stats"]
Lines changed: 78 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,78 @@
1+
"""Derive a wait timeout from observed step durations instead of guessing.
2+
3+
Hard-coded waits are a perennial source of flakiness: too short and a slow
4+
machine races the UI; too long and every failure pays the full timeout. The
5+
durable fix is to *learn* the timeout from how long the step has actually taken.
6+
``adaptive_timeout`` turns a sample of observed durations into a robust timeout:
7+
a high percentile (the slow-but-real case) scaled by a safety ``factor``, then
8+
clamped to a sane ``[min_s, max_s]`` band.
9+
10+
* :func:`recommend_timeout` — the single number to feed a wait / ``GateConfig``.
11+
* :func:`timeout_stats` — the same with the percentiles and clamp flags exposed
12+
for logging / tuning.
13+
14+
Both are pure and reuse :func:`stats.percentile`; with no samples they fall back
15+
to ``default_s`` (or ``min_s``). Imports no ``PySide6``.
16+
"""
17+
from typing import Any, Dict, List, Optional, Sequence
18+
19+
from je_auto_control.utils.stats.stats import percentile
20+
21+
22+
def _clamp(value: float, min_s: float, max_s: Optional[float]) -> float:
23+
"""Clamp ``value`` to ``[min_s, max_s]`` (``max_s`` None = no upper cap)."""
24+
bounded = max(float(min_s), float(value))
25+
if max_s is not None:
26+
bounded = min(float(max_s), bounded)
27+
return bounded
28+
29+
30+
def _fallback(default_s: Optional[float], min_s: float) -> float:
31+
"""The timeout to use when there are no duration samples."""
32+
return float(default_s) if default_s is not None else float(min_s)
33+
34+
35+
def recommend_timeout(durations: Sequence[float], *, percentile_q: float = 95.0,
36+
factor: float = 1.5, min_s: float = 1.0,
37+
max_s: Optional[float] = 60.0,
38+
default_s: Optional[float] = None) -> float:
39+
"""Recommend a wait timeout (seconds) from observed ``durations``.
40+
41+
Takes the ``percentile_q``-th percentile of the samples, scales it by
42+
``factor``, and clamps to ``[min_s, max_s]``. With no samples returns
43+
``default_s`` (or ``min_s``).
44+
"""
45+
samples = [float(d) for d in durations if d is not None]
46+
if not samples:
47+
return _fallback(default_s, min_s)
48+
scaled = percentile(samples, float(percentile_q)) * float(factor)
49+
return _clamp(scaled, min_s, max_s)
50+
51+
52+
def timeout_stats(durations: Sequence[float], *, percentile_q: float = 95.0,
53+
factor: float = 1.5, min_s: float = 1.0,
54+
max_s: Optional[float] = 60.0,
55+
default_s: Optional[float] = None) -> Dict[str, Any]:
56+
"""Recommend a timeout and expose the percentiles and clamp decisions.
57+
58+
Returns ``{n, p50, p_high, percentile_q, recommended, floored, capped}``.
59+
"""
60+
samples: List[float] = [float(d) for d in durations if d is not None]
61+
recommended = recommend_timeout(
62+
samples, percentile_q=percentile_q, factor=factor, min_s=min_s,
63+
max_s=max_s, default_s=default_s)
64+
if not samples:
65+
return {"n": 0, "p50": None, "p_high": None,
66+
"percentile_q": float(percentile_q),
67+
"recommended": recommended, "floored": False, "capped": False}
68+
p_high = percentile(samples, float(percentile_q))
69+
scaled = p_high * float(factor)
70+
return {
71+
"n": len(samples),
72+
"p50": percentile(samples, 50.0),
73+
"p_high": p_high,
74+
"percentile_q": float(percentile_q),
75+
"recommended": recommended,
76+
"floored": scaled < float(min_s),
77+
"capped": max_s is not None and scaled > float(max_s),
78+
}

je_auto_control/utils/executor/action_executor.py

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2783,6 +2783,28 @@ def _verify_field_value(expected: Any, name: Optional[str] = None,
27832783
mode=str(mode))
27842784

27852785

2786+
def _adaptive_timeout(durations: Any, percentile_q: Any = 95.0,
2787+
factor: Any = 1.5, min_s: Any = 1.0,
2788+
max_s: Any = 60.0) -> Dict[str, Any]:
2789+
"""Adapter: recommend a wait timeout from observed durations (pure)."""
2790+
from je_auto_control.utils.adaptive_timeout import recommend_timeout
2791+
samples = [float(d) for d in _coerce_list(durations)] if durations else []
2792+
timeout = recommend_timeout(samples, percentile_q=float(percentile_q),
2793+
factor=float(factor), min_s=float(min_s),
2794+
max_s=float(max_s))
2795+
return {"timeout_s": float(timeout)}
2796+
2797+
2798+
def _timeout_stats(durations: Any, percentile_q: Any = 95.0, factor: Any = 1.5,
2799+
min_s: Any = 1.0, max_s: Any = 60.0) -> Dict[str, Any]:
2800+
"""Adapter: timeout recommendation plus percentiles / clamp flags (pure)."""
2801+
from je_auto_control.utils.adaptive_timeout import timeout_stats
2802+
samples = [float(d) for d in _coerce_list(durations)] if durations else []
2803+
return timeout_stats(samples, percentile_q=float(percentile_q),
2804+
factor=float(factor), min_s=float(min_s),
2805+
max_s=float(max_s))
2806+
2807+
27862808
def _normalize_ext(target: str) -> Dict[str, Any]:
27872809
"""Adapter: the lowercased extension of a path / bare ext (pure)."""
27882810
from je_auto_control.utils.file_assoc import normalize_ext
@@ -6809,6 +6831,8 @@ def __init__(self):
68096831
"AC_plan_retry_delays": _plan_retry_delays,
68106832
"AC_compare_field_value": _compare_field_value,
68116833
"AC_verify_field_value": _verify_field_value,
6834+
"AC_adaptive_timeout": _adaptive_timeout,
6835+
"AC_timeout_stats": _timeout_stats,
68126836
"AC_normalize_ext": _normalize_ext,
68136837
"AC_file_association": _file_association,
68146838
"AC_get_control_text": _get_control_text,

je_auto_control/utils/mcp_server/tools/_factories.py

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1826,6 +1826,37 @@ def smart_wait_tools() -> List[MCPTool]:
18261826
handler=h.verify_field_value,
18271827
annotations=READ_ONLY,
18281828
),
1829+
MCPTool(
1830+
name="ac_adaptive_timeout",
1831+
description=("Recommend a wait timeout (seconds) from observed step "
1832+
"'durations': the 'percentile_q'-th percentile scaled "
1833+
"by 'factor', clamped to [min_s, max_s]. Returns "
1834+
"{timeout_s}."),
1835+
input_schema=schema({"durations": {"type": "array",
1836+
"items": {"type": "number"}},
1837+
"percentile_q": {"type": "number"},
1838+
"factor": {"type": "number"},
1839+
"min_s": {"type": "number"},
1840+
"max_s": {"type": "number"}},
1841+
required=["durations"]),
1842+
handler=h.adaptive_timeout,
1843+
annotations=READ_ONLY,
1844+
),
1845+
MCPTool(
1846+
name="ac_timeout_stats",
1847+
description=("Recommend a timeout and expose the percentiles and "
1848+
"clamp decisions. Returns {n, p50, p_high, "
1849+
"percentile_q, recommended, floored, capped}."),
1850+
input_schema=schema({"durations": {"type": "array",
1851+
"items": {"type": "number"}},
1852+
"percentile_q": {"type": "number"},
1853+
"factor": {"type": "number"},
1854+
"min_s": {"type": "number"},
1855+
"max_s": {"type": "number"}},
1856+
required=["durations"]),
1857+
handler=h.timeout_stats,
1858+
annotations=READ_ONLY,
1859+
),
18291860
]
18301861

18311862

je_auto_control/utils/mcp_server/tools/_handlers.py

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -698,6 +698,20 @@ def verify_field_value(expected, name=None, role=None, app_name=None,
698698
mode)
699699

700700

701+
def adaptive_timeout(durations, percentile_q=95.0, factor=1.5, min_s=1.0,
702+
max_s=60.0):
703+
from je_auto_control.utils.executor.action_executor import (
704+
_adaptive_timeout,
705+
)
706+
return _adaptive_timeout(durations, percentile_q, factor, min_s, max_s)
707+
708+
709+
def timeout_stats(durations, percentile_q=95.0, factor=1.5, min_s=1.0,
710+
max_s=60.0):
711+
from je_auto_control.utils.executor.action_executor import _timeout_stats
712+
return _timeout_stats(durations, percentile_q, factor, min_s, max_s)
713+
714+
701715
def normalize_ext(target):
702716
from je_auto_control.utils.executor.action_executor import _normalize_ext
703717
return _normalize_ext(target)

0 commit comments

Comments
 (0)