Skip to content

Load ML Improvements, Put back GivTCP Load #3392

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
springfall2008 merged 10 commits into from
Feb 19, 2026
Merged

Load ML Improvements, Put back GivTCP Load #3392

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
e69f775
New example ML chart
springfall2008 Feb 18, 2026
04e5925
Put back Load Power
springfall2008 Feb 18, 2026
d1e7ed7
[pre-commit.ci lite] apply automatic fixes
pre-commit-ci-lite[bot] Feb 19, 2026
e770310
Weight decay (AdamW-style) and/or dropout would also help regularise …
springfall2008 Feb 19, 2026
8f6b706
Merge branch 'fixes20' of github.com:springfall2008/batpred into fixes20
springfall2008 Feb 19, 2026
76dd55a
[pre-commit.ci lite] apply automatic fixes
pre-commit-ci-lite[bot] Feb 19, 2026
cdbb3b3
Fix failing tests
springfall2008 Feb 19, 2026
61d8b3b
Merge branch 'fixes20' of github.com:springfall2008/batpred into fixes20
springfall2008 Feb 19, 2026
de534a0
Add MAE to docs
springfall2008 Feb 19, 2026
c5b5408
Catch stop in octopus retry loop
springfall2008 Feb 19, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 2 additions & 3 deletions apps/predbat/inverter.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1105,12 +1105,11 @@ def update_status(self, minutes_now, quiet=False):
pdetails = self.rest_data["Power"]
if "Power" in pdetails:
ppdetails = pdetails["Power"]
# self.log("DEBUG: Power details from REST: {}".format(ppdetails))
self.battery_power = float(ppdetails.get("Battery_Power", 0.0))
self.pv_power = float(ppdetails.get("PV_Power", 0.0))
self.grid_power = float(ppdetails.get("Grid_Power", 0.0))
# Calculate load from energy balance instead of using inverter register (which is incorrect during grid charging)
# Load = PV + Grid + Battery (battery negative when charging, positive when discharging)
self.load_power = self.pv_power + self.grid_power + self.battery_power
self.load_power = float(ppdetails.get("Load_Power", 0.0))
Copy link

Copilot AI Feb 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reading REST "Load_Power" directly changes semantics and breaks the existing assumption (and fixtures) that GivTCP load is unreliable during some modes (e.g., grid charging). The repo’s REST fixtures show Load_Power != PV_Power + Grid_Power + Battery_Power (e.g., coverage/cases/rest_v2.json has Load_Power=624 but balance=590; rest_v3.json has 197 vs balance=233), and tests assert the energy-balance value. Consider reverting to the balance calculation, or make using Load_Power an opt-in config with a safe fallback when it disagrees significantly with the energy balance.

Suggested change
self.load_power = float(ppdetails.get("Load_Power", 0.0))
# Derive load power from energy balance rather than trusting REST Load_Power directly
self.load_power = self.pv_power + self.grid_power + self.battery_power

Copilot uses AI. Check for mistakes.
if self.rest_v3:
self.battery_voltage = float(ppdetails.get("Battery_Voltage", 0.0))
else:
Expand Down
2 changes: 1 addition & 1 deletion apps/predbat/predbat.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@
import requests
import asyncio

THIS_VERSION = "v8.33.3"
THIS_VERSION = "v8.33.4"

# fmt: off
PREDBAT_FILES = ["predbat.py", "const.py", "hass.py", "config.py", "prediction.py", "gecloud.py", "utils.py", "inverter.py", "ha.py", "download.py", "web.py", "web_helper.py", "predheat.py", "futurerate.py", "octopus.py", "solcast.py", "execute.py", "plan.py", "fetch.py", "output.py", "userinterface.py", "energydataservice.py", "alertfeed.py", "compare.py", "db_manager.py", "db_engine.py", "plugin_system.py", "ohme.py", "components.py", "fox.py", "carbon.py", "temperature.py", "web_mcp.py", "component_base.py", "axle.py", "solax.py", "solis.py", "unit_test.py", "load_ml_component.py", "load_predictor.py"]
Expand Down
349 changes: 349 additions & 0 deletions templates/example_chart.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1083,3 +1083,352 @@ series:
let res = []; for (const item of entity.attributes.detailedForecast) {
res.push([new Date(item.period_start).getTime(), item.pv_estimateCL]); }
return res.sort((a, b) => a[0] - b[0]);


###############################################
#------------------------ML Home Load Forecast (excl EV)-------------------------------
###############################################

type: custom:apexcharts-card
header:
show: true
title: ML Home Load Forecast (excl EV)
show_states: true
colorize_states: true
graph_span: 4d
update_interval: 15min
span:
start: day
offset: "-2d"
apex_config:
legend:
show: true
now:
show: true
yaxis:
- id: forecasts
show: true
opposite: true
decimals: 0
max: 60
min: 0
apex_config:
tickAmount: 6
- id: mae
show: false
opposite: true
decimals: 2
max: auto
min: -0.1
apex_config:
tickAmount: 6
series:
- entity: sensor.predbat_load_ml_stats
attribute: load_today
stroke_width: 2
extend_to: now
curve: smooth
name: Load
opacity: 0.2
type: area
yaxis_id: forecasts
color: orange
- entity: sensor.predbat_load_ml_stats
attribute: load_today
stroke_width: 3
extend_to: now
curve: smooth
name: Load
opacity: 1
type: line
yaxis_id: forecasts
color: orange
show:
in_header: false
in_legend: false
- entity: sensor.predbat_load_ml_forecast
stroke_width: 3
curve: smooth
name: ML Load
color: red
opacity: 0.35
yaxis_id: forecasts
show:
in_header: false
in_legend: true
data_generator: >
let res = []; for (const [key, value] of
Object.entries(entity.attributes.results)) { res.push([new
Date(key).getTime(), value]); } return res.sort((a, b) => { return a[0] -
b[0] })
Comment on lines +1160 to +1163
Copy link

Copilot AI Feb 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This data_generator assumes entity.attributes.results is always present; if the entity is unavailable or the attribute is missing, Object.entries(undefined) will throw and the chart will break. Consider using the same guard used later in this file (e.g., default to {} / return [] when results is falsy).

Suggested change
let res = []; for (const [key, value] of
Object.entries(entity.attributes.results)) { res.push([new
Date(key).getTime(), value]); } return res.sort((a, b) => { return a[0] -
b[0] })
if (!entity || !entity.attributes || !entity.attributes.results) return [];
const res = [];
for (const [key, value] of Object.entries(entity.attributes.results || {})) {
res.push([new Date(key).getTime(), value]);
}
return res.sort((a, b) => a[0] - b[0]);

Copilot uses AI. Check for mistakes.
- entity: sensor.predbat_load_ml_stats
attribute: load_today_h1
stroke_width: 2
color: green
curve: smooth
name: FC
opacity: 0.6
type: line
offset: "-1h"
extend_to: now
yaxis_id: forecasts
- entity: sensor.predbat_load_ml_stats
attribute: load_today_h8
stroke_width: 2
color: purple
curve: smooth
name: FC
opacity: 0.35
type: line
offset: "-8h"
extend_to: now
yaxis_id: forecasts
Comment on lines +1164 to +1185
Copy link

Copilot AI Feb 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Legend/series names are duplicated (both the -1h and -8h series are named "FC"), which makes them indistinguishable in the legend/tooltip. Use distinct names (e.g., "FC -1h" and "FC -8h").

Copilot uses AI. Check for mistakes.
- entity: sensor.predbat_load_ml_stats
attribute: mae_kwh
float_precision: 4
stroke_width: 2
color: black
curve: smooth
name: MAE
opacity: 1
type: line
extend_to: now
yaxis_id: mae
transform: |
if (x === null || x === undefined) return null;
return Number(parseFloat(x).toFixed(3));

###############################################
#------------------------ML Home Power Forecast (excl EV)-------------------------------
###############################################

type: custom:apexcharts-card
header:
show: true
title: ML Home Power Forecast (excl EV)
show_states: true
colorize_states: true
graph_span: 4d
update_interval: 15min
span:
start: day
offset: "-2d"
apex_config:
legend:
show: true
tooltip:
shared: true
now:
show: true
yaxis:
- id: forecasts
show: true
opposite: false
decimals: 0
max: 4
min: 0
apex_config:
tickAmount: 12
- id: temp
show: true
opposite: true
decimals: 0
max: auto
min: -10
apex_config:
tickAmount: 6
series:
- entity: sensor.predbat_temperature
stroke_width: 2
extend_to: now
curve: smooth
name: Temp
opacity: 1
type: line
yaxis_id: temp
color: blue
- entity: sensor.predbat_temperature
name: Temp FC
type: line
yaxis_id: temp
stroke_width: 2
curve: smooth
color: blue
opacity: 0.35
extend_to: false
show:
in_header: false
in_legend: false
data_generator: |
const results = entity.attributes.results;
if (!results) return [];

const now = Date.now();

return Object.entries(results)
.map(([dt, val]) => [new Date(dt).getTime(), val])
.filter(([ts, _]) => ts >= now);
- entity: sensor.predbat_load_ml_stats
attribute: power_today_now
stroke_width: 2
extend_to: now
curve: smooth
name: Power
opacity: 0.2
type: area
yaxis_id: forecasts
color: orange
- entity: sensor.predbat_load_ml_stats
attribute: power_today_now
stroke_width: 2
extend_to: now
curve: smooth
name: Power
opacity: 1
type: line
yaxis_id: forecasts
color: orange
show:
in_header: false
in_legend: false
- entity: sensor.predbat_load_ml_stats
attribute: power_today_h1
stroke_width: 2
color: green
curve: smooth
name: FC
opacity: 0.6
type: line
offset: "-1h"
extend_to: now
yaxis_id: forecasts
- entity: sensor.predbat_load_ml_stats
attribute: power_today_h8
stroke_width: 2
color: purple
curve: smooth
name: FC
opacity: 0.35
type: line
offset: "-8h"
extend_to: now
yaxis_id: forecasts
Comment on lines +1294 to +1315
Copy link

Copilot AI Feb 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Legend/series names are duplicated here as well (both forecast-offset series are named "FC"), which makes it hard to tell them apart in the legend/tooltip. Use distinct names for the -1h vs -8h series.

Copilot uses AI. Check for mistakes.
- entity: sensor.predbat_load_ml_forecast
name: ML FC
type: line
curve: smooth
stroke_width: 3
color: red
opacity: 0.35
yaxis_id: forecasts
show:
in_header: false
data_generator: |
const raw = Object.entries(entity.attributes.results || {})
.map(([ts, v]) => [new Date(ts).getTime(), Number(v)])
.filter(p => !isNaN(p[0]) && !isNaN(p[1]))
.sort((a,b) => a[0]-b[0]);

const res = [];
for (let i = 1; i < raw.length; i++) {
const [t1, e1] = raw[i-1]; // previous cumulative kWh
const [t2, e2] = raw[i]; // current cumulative kWh

const dt_hours = (t2 - t1) / 3600000; // ms → hours
if (dt_hours <= 0) continue;

const dE = e2 - e1; // kWh used during interval

const power_kw = dE / dt_hours;

res.push([t2, power_kw]);
}
return res;
Comment on lines +1332 to +1346
Copy link

Copilot AI Feb 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This cumulative-kWh → power conversion doesn’t guard against negative deltas (e2 < e1), which can happen if the cumulative series resets/backsteps and will produce misleading negative power spikes. Consider skipping negative dE values (similar to the later rolling-average generator) or otherwise handling resets explicitly.

Copilot uses AI. Check for mistakes.
- entity: sensor.predbat_load_ml_stats
attribute: power_today_now
name: Home Power (Avg 1h)
type: line
yaxis_id: forecasts
color: black
stroke_width: 3
curve: smooth
opacity: 1
extend_to: now
show:
in_header: false
in_legend: false
group_by:
func: avg
duration: 6h
Comment on lines +1348 to +1362
Copy link

Copilot AI Feb 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The label says "Home Power (Avg 1h)", but group_by.duration is set to 6h, so the displayed series won’t match the name. Either change the name or adjust the group_by duration to align with the label.

Copilot uses AI. Check for mistakes.
- entity: sensor.predbat_load_ml_forecast
name: ML Forecast Power (Avg 6h)
type: line
curve: smooth
stroke_width: 3
color: black
opacity: 1
yaxis_id: forecasts
show:
in_header: false
in_legend: false
data_generator: |
// Build raw cumulative kWh points
const raw = Object.entries(entity.attributes.results || {})
.map(([ts, v]) => [new Date(ts).getTime(), Number(v)])
.filter(([t, v]) => !isNaN(t) && !isNaN(v))
.sort((a, b) => a[0] - b[0]);

if (raw.length < 2) return [];

// Convert cumulative kWh -> interval energy (kWh) + duration (ms)
const intervals = [];
for (let i = 1; i < raw.length; i++) {
const [t1, e1] = raw[i - 1];
const [t2, e2] = raw[i];

const dt = t2 - t1; // ms
if (dt <= 0) continue;

const dE = e2 - e1; // kWh used during interval
if (!isFinite(dE)) continue;

// Skip resets/backsteps that create negative "power" spikes/dropouts
if (dE < 0) continue;

intervals.push([t2, dE, dt]); // store at interval end time
}

if (!intervals.length) return [];

// Rolling window: 6 hours
const windowMs = 6 * 60 * 60 * 1000;

// Optional: require at least 1 hour of data in the window before plotting
const minCoverageMs = 60 * 60 * 1000;

// Two-pointer, time-weighted rolling mean: sum(kWh)/sum(hours)
let sumE = 0; // kWh
let sumMs = 0; // ms
let j = 0;
const out = [];

for (let i = 0; i < intervals.length; i++) {
const [ti, dEi, dti] = intervals[i];
sumE += dEi;
sumMs += dti;

while (intervals[j][0] < ti - windowMs) {
sumE -= intervals[j][1];
sumMs -= intervals[j][2];
j++;
}

if (sumMs <= 0 || sumMs < minCoverageMs) {
out.push([ti, null]); // plot gaps until enough window coverage
} else {
const avgKw = sumE / (sumMs / 3600000); // kWh / hours = kW
out.push([ti, avgKw]);
}
}

return out;
Loading