diff --git a/skills/csharp/debug-performance/SKILL.md b/skills/csharp/debug-performance/SKILL.md new file mode 100644 index 0000000000..d5b1fec2b6 --- /dev/null +++ b/skills/csharp/debug-performance/SKILL.md @@ -0,0 +1,92 @@ +--- +name: debug-performance +description: > + Diagnoses slow QuantConnect Python (.py) and C# (.cs) backtests using the + Performance Chart first and Python cProfile only when the chart does not + pinpoint the hot function. Trigger phrases: "slow backtest", "high CPU", "algorithm slow", "CPU usage", "RAM usage", "memory usage", "performance chart", "profiling", "bottleneck", "debug performance", "taking too long", "optimize algorithm". +--- + +# /debug-performance - QuantConnect Performance Debugging +Invoked on a slow backtest. Work top to bottom. Find the dominant cost before +changing code. Same Performance Chart rules for `main.py` and `main.cs`; Python +can add cProfile only when Section 4 says to. + +## 0. Never add error-hiding patterns +Never introduce py`try`/`except`cs`try`/`catch` or +py`hasattr`/`getattr`/`setattr`/`isinstance`cs`reflection`. They hide the exact +failure or slow path. A profiler wrapper measures work; it is not a catch. If +existing code wraps the slow region in one, remove it, re-run, read the real +behavior, then continue. + +## 1. Enable Performance Chart first +Record the original py`set_start_date`/`set_end_date`cs`SetStartDate`/`SetEndDate` +and universe, then shrink both: usually 1 to 3 months and the smallest universe +that still reproduces the slowdown. Set +`Settings.PerformanceSamplePeriod = TimeSpan.FromDays(7)` +in `Initialize`, then run the short backtest. +Open the results and inspect the Performance Chart. Record the dominant series: +the tallest spike, or the sustained plateau if there is no single spike. Do not +optimize before naming the series. + +## 2. Route by dominant series +Read which series dominates: +- Selection, Consolidators, OnData, Schedule, Subscriptions, Securities, + Transactions, SplitsDividendsDelisting, Slice, HistoryDataPoints, or + ActiveSecurities -> Section 3. +- CPU, ManagedRAM, or TotalRAM high with no single time-series dominant -> + Section 4 for Python, or keep using the chart for C#. +- DataPoints low -> Section 3, Subscriptions row. +If multiple series spike together, start with the earliest causal source: +Subscriptions before Consolidators, Consolidators before OnData, Selection +before Securities or Transactions. Treat CPU and RAM as symptoms until the chart +or profiler names the code path. + +## 3. Fix by bottleneck +Change only the code path that corresponds to the dominant series. Re-run the +same short backtest after each change and compare the same chart region. +For rolling calculations, use this order: built-in indicator first, indicator +extension second, `Security.session` for cached OHLCV/session data third, and +manual `RollingWindow`/`deque` only when LEAN has no built-in equivalent. Example: +a trailing mean of closes is an SMA, so use py`self.sma(security, period)` instead +of storing closes and averaging them manually. +| Series | Meaning | Fix | +| --- | --- | --- | +| Selection | Universe add/remove and selection functions. | Reduce coarse universe size; move expensive fundamental queries out of the filter; cache results across calls. | +| Consolidators | Consolidation, indicator updates, consolidator handlers. | Reduce universe size; consolidate at a coarser resolution; share one consolidator where logic allows. | +| OnData | `OnData` and alpha update time. | Move heavy logic to a scheduled event; replace manual series calculations with a built-in indicator or indicator extension before considering a manual window. | +| Schedule | Scheduled event handler time. | Reduce event frequency; cache intermediate results computed in the handler. | +| Subscriptions | Time reading subscribed data. | Reduce data resolution or universe size; confirm only needed resolutions are subscribed. | +| Securities | Security updates, security changes, symbol changes. | Reduce `ActiveSecurities` count, usually fewer holdings or open orders. | +| Transactions | Order event processing. | Batch orders via portfolio targets; reduce rebalance frequency. | +| SplitsDividendsDelisting | Corporate action events. | Reduce universe size or move corporate-action logic out of the handler. | +| Slice | Time creating the `Slice` object. | Reduce subscription count, resolution, and custom data fields before optimizing handlers. | +| HistoryDataPoints | History provider data points. | Reduce `History` calls; replace rolling calculations with indicators/extensions; use `Security.Session` for cached OHLCV/session data; request fewer symbols, fields, or bars. | +| ActiveSecurities | Selected securities plus holdings and open orders. | Prune stale symbols; liquidate or cancel old positions/orders before broadening the universe. | +Logging rules: measure with `Log(...)` in the narrow handler +being debugged. Never use the Object Store for profiler output. Never log inside +`OnData` or an often-firing scheduled event without a counter limit. +Remove diagnostic logs after the fix. + +## 4. Python profiler with logs +Use cProfile only when CPU, ManagedRAM, or TotalRAM is high and the Performance +Chart does not isolate the expensive function. This is Python only. C# does not +have a built-in cProfile equivalent; use the chart series exclusively. +1. At the top of `main.py`, before the class: +2. In `OnEndOfAlgorithm`, disable the profiler and log + the top cumulative-time lines. Do not save the report to the Object Store: + +3. Read the cumulative time column in the backtest logs. The top function is the + bottleneck; map it back to a Section 3 series and apply the matching fix. +Remove the profiler after measurement unless the user explicitly wants another +profiling run. + +## 5. Checklist +1. No banned error-hiding pattern (Section 0). +2. Original dates + universe recorded before shrinking. +3. Performance Chart enabled; short backtest run to expose the dominant series. +4. Dominant series identified before applying any fix (Section 2). +5. Fix targets the real bottleneck series, not a guess (Section 3). +6. Built-in indicators/extensions considered before adding py`RollingWindow`/`deque`cs`RollingWindow`. +7. Profiler used only for Python when chart alone does not pinpoint the function. +8. Profiler output logged with `Log`; Object Store not used. +9. Original period + universe restored; final backtest confirms the spike is gone. diff --git a/skills/python/debug-performance/SKILL.md b/skills/python/debug-performance/SKILL.md new file mode 100644 index 0000000000..2c0c40c611 --- /dev/null +++ b/skills/python/debug-performance/SKILL.md @@ -0,0 +1,110 @@ +--- +name: debug-performance +description: > + Diagnoses slow QuantConnect Python (.py) and C# (.cs) backtests using the + Performance Chart first and Python cProfile only when the chart does not + pinpoint the hot function. Trigger phrases: "slow backtest", "high CPU", "algorithm slow", "CPU usage", "RAM usage", "memory usage", "performance chart", "profiling", "bottleneck", "debug performance", "taking too long", "optimize algorithm". +--- + +# /debug-performance - QuantConnect Performance Debugging +Invoked on a slow backtest. Work top to bottom. Find the dominant cost before +changing code. Same Performance Chart rules for `main.py` and `main.cs`; Python +can add cProfile only when Section 4 says to. + +## 0. Never add error-hiding patterns +Never introduce py`try`/`except`cs`try`/`catch` or +py`hasattr`/`getattr`/`setattr`/`isinstance`cs`reflection`. They hide the exact +failure or slow path. A profiler wrapper measures work; it is not a catch. If +existing code wraps the slow region in one, remove it, re-run, read the real +behavior, then continue. + +## 1. Enable Performance Chart first +Record the original py`set_start_date`/`set_end_date`cs`SetStartDate`/`SetEndDate` +and universe, then shrink both: usually 1 to 3 months and the smallest universe +that still reproduces the slowdown. Set +`self.settings.performance_sample_period = timedelta(7)` +in `initialize`, then run the short backtest. +Open the results and inspect the Performance Chart. Record the dominant series: +the tallest spike, or the sustained plateau if there is no single spike. Do not +optimize before naming the series. + +## 2. Route by dominant series +Read which series dominates: +- Selection, Consolidators, OnData, Schedule, Subscriptions, Securities, + Transactions, SplitsDividendsDelisting, Slice, HistoryDataPoints, or + ActiveSecurities -> Section 3. +- CPU, ManagedRAM, or TotalRAM high with no single time-series dominant -> + Section 4 for Python, or keep using the chart for C#. +- DataPoints low -> Section 3, Subscriptions row. +If multiple series spike together, start with the earliest causal source: +Subscriptions before Consolidators, Consolidators before OnData, Selection +before Securities or Transactions. Treat CPU and RAM as symptoms until the chart +or profiler names the code path. + +## 3. Fix by bottleneck +Change only the code path that corresponds to the dominant series. Re-run the +same short backtest after each change and compare the same chart region. +For rolling calculations, use this order: built-in indicator first, indicator +extension second, `Security.session` for cached OHLCV/session data third, and +manual `RollingWindow`/`deque` only when LEAN has no built-in equivalent. Example: +a trailing mean of closes is an SMA, so use py`self.sma(security, period)` instead +of storing closes and averaging them manually. +| Series | Meaning | Fix | +| --- | --- | --- | +| Selection | Universe add/remove and selection functions. | Reduce coarse universe size; move expensive fundamental queries out of the filter; cache results across calls. | +| Consolidators | Consolidation, indicator updates, consolidator handlers. | Reduce universe size; consolidate at a coarser resolution; share one consolidator where logic allows. | +| OnData | `on_data` and alpha update time. | Move heavy logic to a scheduled event; replace manual series calculations with a built-in indicator or indicator extension before considering a manual window. | +| Schedule | Scheduled event handler time. | Reduce event frequency; cache intermediate results computed in the handler. | +| Subscriptions | Time reading subscribed data. | Reduce data resolution or universe size; confirm only needed resolutions are subscribed. | +| Securities | Security updates, security changes, symbol changes. | Reduce `active_securities` count, usually fewer holdings or open orders. | +| Transactions | Order event processing. | Batch orders via portfolio targets; reduce rebalance frequency. | +| SplitsDividendsDelisting | Corporate action events. | Reduce universe size or move corporate-action logic out of the handler. | +| Slice | Time creating the `slice` object. | Reduce subscription count, resolution, and custom data fields before optimizing handlers. | +| HistoryDataPoints | History provider data points. | Reduce `history` calls; replace rolling calculations with indicators/extensions; use `Security.session` for cached OHLCV/session data; request fewer symbols, fields, or bars. | +| ActiveSecurities | Selected securities plus holdings and open orders. | Prune stale symbols; liquidate or cancel old positions/orders before broadening the universe. | +Logging rules: measure with `self.log(...)` in the narrow handler +being debugged. Never use the Object Store for profiler output. Never log inside +`on_data` or an often-firing scheduled event without a counter limit. +Remove diagnostic logs after the fix. + +## 4. Python profiler with logs +Use cProfile only when CPU, ManagedRAM, or TotalRAM is high and the Performance +Chart does not isolate the expensive function. This is Python only. C# does not +have a built-in cProfile equivalent; use the chart series exclusively. +1. At the top of `main.py`, before the class: +```python +from cProfile import Profile +from pstats import Stats +from io import StringIO + +profile = Profile() +profile.enable() +``` +2. In `on_end_of_algorithm`, disable the profiler and log + the top cumulative-time lines. Do not save the report to the Object Store: + +```python +def on_end_of_algorithm(self): + profile.disable() + stream = StringIO() + Stats(profile, stream=stream).sort_stats('cumulative').print_stats(20) + lines = stream.getvalue().splitlines() + for line in lines[:40]: + self.log(f"PROFILE {line}") +``` + +3. Read the cumulative time column in the backtest logs. The top function is the + bottleneck; map it back to a Section 3 series and apply the matching fix. +Remove the profiler after measurement unless the user explicitly wants another +profiling run. + +## 5. Checklist +1. No banned error-hiding pattern (Section 0). +2. Original dates + universe recorded before shrinking. +3. Performance Chart enabled; short backtest run to expose the dominant series. +4. Dominant series identified before applying any fix (Section 2). +5. Fix targets the real bottleneck series, not a guess (Section 3). +6. Built-in indicators/extensions considered before adding py`RollingWindow`/`deque`cs`RollingWindow`. +7. Profiler used only for Python when chart alone does not pinpoint the function. +8. Profiler output logged with `self.log`; Object Store not used. +9. Original period + universe restored; final backtest confirms the spike is gone.