feat: zlib preset dictionary decompressor for S7CommPlus blobs#776
feat: zlib preset dictionary decompressor for S7CommPlus blobs#776gijzelaerr wants to merge 6 commits into
Conversation
) S7-1200/1500 PLCs compress XML metadata (tag definitions, interface descriptions, network comments, etc.) using zlib with Siemens-specific preset dictionaries. Python's zlib.decompress() returns Z_NEED_DICT for these blobs. This adds the 19 known dictionaries and a decompressor that auto-selects the correct one by Adler-32 checksum. The dictionaries were extracted from thomas-v2/S7CommPlusDriver (LGPL-3.0, BlobDecompressor.cs). Key dictionary for #757: IntfDescTag (0xce9b821b) — used for I/Q/M area tag definitions on V3 PLCs (S7-1200 FW V4.5+). New modules: - s7commplus/zlib_dicts.py — 19 preset dictionaries (24KB total) - s7commplus/blob_decompressor.py — decompress_blob() with auto dictionary lookup, find_and_decompress() for scanning raw payloads Replaces the partial oracle-based dictionary reconstruction in examples/browse_tags.py with complete, verified dictionaries. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The dictionaries are 100% printable XML template fragments (with the occasional German ü). Storing them as hex blobs made them unreadable. Now stored as Python string literals, encoded to bytes at import time. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace the 385-line Python module of string-chunked XML with 19
individual .xml files in s7commplus/zlib_dicts/. Each file is named
{adler32}_{name}.xml and can be read directly in any editor or IDE.
The __init__.py loader scans the directory at import time and builds
the same ZLIB_DICTIONARIES/ZLIB_DICT_NAMES dicts as before.
Also adds *.xml to pyproject.toml package-data so the files ship
in the wheel.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The end-of-file-fixer hook requires a trailing newline. The loader strips it with rstrip(b"\n") so the Adler-32 checksums stay correct. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The .xml dictionary files contain raw zlib preset dictionary bytes, not valid standalone XML. Mark them as binary in .gitattributes to prevent CRLF conversion on Windows (which broke Adler-32 checksums), and exclude them from the check-xml pre-commit hook. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add explore_xml() to both sync and async clients — sends an EXPLORE request and decompresses the zlib-compressed XML metadata from the response. Also export decompress_blob and find_and_decompress from the s7commplus package for direct use with raw EXPLORE payloads. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
Independently validated I no longer have the G1 FW V4.5 unit from #757, so all testing here is on the G2 only — but it turns out the G2 declares the exact same preset dictionary. All three EXPLORE areas carry a <?xml version="1.0" encoding="utf-8"?><IdentContainer>
<Ident Name="mtag_word" Scope="Global" LID="13"><SimpleType>Word</SimpleType><Access SubClass="SimpleAccess"><SimpleAccess ByteNumber="102" Width="Word" Range="Memory" /></Access></Ident>
...
</IdentContainer>Follow-up wiring (the part you flagged as TODO)Since the XML is clean, the tag layer is tiny. Mapping import xml.etree.ElementTree as ET
from s7commplus.blob_decompressor import find_and_decompress
_RANGE = {"Input": "I", "Output": "Q", "Memory": "M"}
_WIDTH = {"Byte": "B", "Word": "W", "DWord": "D", "LWord": "L"}
def _address(rng, width, byte_no, bit_no):
letter = _RANGE.get(rng, "?")
if width == "Bit":
return f"%{letter}{byte_no}.{bit_no or 0}"
return f"%{letter}{_WIDTH.get(width, '')}{byte_no}"
def parse_ident_container(xml_text):
tags = []
for ident in ET.fromstring(xml_text).findall("Ident"):
acc = ident.find("./Access/SimpleAccess")
if acc is None:
continue
byte_no = int(acc.get("ByteNumber", "0"))
bit_raw = acc.get("BitNumber")
tags.append({
"name": ident.get("Name"),
"type": ident.findtext("SimpleType"),
"lid": int(ident.get("LID", "0")),
"address": _address(acc.get("Range"), acc.get("Width"),
byte_no, int(bit_raw) if bit_raw else None),
})
return tagsResult against the running G2 (verified against TIA Portal):
This makes the whole oracle pipeline from #757 ( I'm opening a small stacked PR on top of this branch that adds a non-invasive |
…RE XML Follow-up to gijzelaerr#776: wires find_and_decompress() into a small non-invasive tag layer. Parses the clean <IdentContainer> XML into Tag records with TIA addresses (%I0.0, %Q0.1, %MB100, %MW102, %MD104). Includes a regression test built from a real S7-1200 G2 (FW V4.1) M-area EXPLORE blob. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Summary
blob_decompressor.pywithdecompress_blob()that auto-selects the correct dictionary by Adler-32 checksumThe key dictionary for #757 is
IntfDescTag(Adler-320xce9b821b, 1178 bytes) — used for I/Q/M area tag definitions on V3 PLCs (S7-1200 FW V4.5+). With the complete dictionary, decompression produces clean XML that can be parsed with standard tools instead of regex extraction from garbled output.Addresses #757.
Test plan
find_and_decompress()scans raw payloads for zlib streams🤖 Generated with Claude Code