Skip to content

feat: zlib preset dictionary decompressor for S7CommPlus blobs#776

Open
gijzelaerr wants to merge 6 commits into
masterfrom
feat/zlib-preset-dictionaries
Open

feat: zlib preset dictionary decompressor for S7CommPlus blobs#776
gijzelaerr wants to merge 6 commits into
masterfrom
feat/zlib-preset-dictionaries

Conversation

@gijzelaerr

Copy link
Copy Markdown
Owner

Summary

The key dictionary for #757 is IntfDescTag (Adler-32 0xce9b821b, 1178 bytes) — used for I/Q/M area tag definitions on V3 PLCs (S7-1200 FW V4.5+). With the complete dictionary, decompression produces clean XML that can be parsed with standard tools instead of regex extraction from garbled output.

Addresses #757.

Test plan

  • Verify all 19 dictionary Adler-32 checksums match
  • Decompress standard (no-dict) and preset-dict blobs
  • Unknown dictionary raises clear error
  • find_and_decompress() scans raw payloads for zlib streams
  • @tommasofaedo: test against S7-1200 FW V4.5 I/Q/M EXPLORE responses

🤖 Generated with Claude Code

gijzelaerr and others added 3 commits July 16, 2026 21:25
)

S7-1200/1500 PLCs compress XML metadata (tag definitions, interface
descriptions, network comments, etc.) using zlib with Siemens-specific
preset dictionaries. Python's zlib.decompress() returns Z_NEED_DICT for
these blobs. This adds the 19 known dictionaries and a decompressor
that auto-selects the correct one by Adler-32 checksum.

The dictionaries were extracted from thomas-v2/S7CommPlusDriver
(LGPL-3.0, BlobDecompressor.cs). Key dictionary for #757:
IntfDescTag (0xce9b821b) — used for I/Q/M area tag definitions on
V3 PLCs (S7-1200 FW V4.5+).

New modules:
- s7commplus/zlib_dicts.py — 19 preset dictionaries (24KB total)
- s7commplus/blob_decompressor.py — decompress_blob() with auto
  dictionary lookup, find_and_decompress() for scanning raw payloads

Replaces the partial oracle-based dictionary reconstruction in
examples/browse_tags.py with complete, verified dictionaries.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The dictionaries are 100% printable XML template fragments (with the
occasional German ü). Storing them as hex blobs made them unreadable.
Now stored as Python string literals, encoded to bytes at import time.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace the 385-line Python module of string-chunked XML with 19
individual .xml files in s7commplus/zlib_dicts/. Each file is named
{adler32}_{name}.xml and can be read directly in any editor or IDE.

The __init__.py loader scans the directory at import time and builds
the same ZLIB_DICTIONARIES/ZLIB_DICT_NAMES dicts as before.

Also adds *.xml to pyproject.toml package-data so the files ship
in the wheel.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
gijzelaerr and others added 3 commits July 16, 2026 21:55
The end-of-file-fixer hook requires a trailing newline. The loader
strips it with rstrip(b"\n") so the Adler-32 checksums stay correct.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The .xml dictionary files contain raw zlib preset dictionary bytes, not
valid standalone XML. Mark them as binary in .gitattributes to prevent
CRLF conversion on Windows (which broke Adler-32 checksums), and exclude
them from the check-xml pre-commit hook.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add explore_xml() to both sync and async clients — sends an EXPLORE
request and decompresses the zlib-compressed XML metadata from the
response. Also export decompress_blob and find_and_decompress from the
s7commplus package for direct use with raw EXPLORE payloads.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@tommasofaedo

Copy link
Copy Markdown
Contributor

Independently validated find_and_decompress() from this branch against a live S7-1200 G2 (FW V4.1) — it works out of the box. 🎉

I no longer have the G1 FW V4.5 unit from #757, so all testing here is on the G2 only — but it turns out the G2 declares the exact same preset dictionary. All three EXPLORE areas carry a CMF=0x78 FLG=0x7d stream with dict Adler-32 0xce9b821b (IntfDescTag), and find_and_decompress(full_explore_payload) returns clean, zero-null XML:

<?xml version="1.0" encoding="utf-8"?><IdentContainer>
<Ident Name="mtag_word" Scope="Global" LID="13"><SimpleType>Word</SimpleType><Access SubClass="SimpleAccess"><SimpleAccess ByteNumber="102" Width="Word" Range="Memory" /></Access></Ident>
...
</IdentContainer>

Follow-up wiring (the part you flagged as TODO)

Since the XML is clean, the tag layer is tiny. Mapping Range+Width+ByteNumber+BitNumber to TIA syntax:

import xml.etree.ElementTree as ET
from s7commplus.blob_decompressor import find_and_decompress

_RANGE = {"Input": "I", "Output": "Q", "Memory": "M"}
_WIDTH = {"Byte": "B", "Word": "W", "DWord": "D", "LWord": "L"}

def _address(rng, width, byte_no, bit_no):
    letter = _RANGE.get(rng, "?")
    if width == "Bit":
        return f"%{letter}{byte_no}.{bit_no or 0}"
    return f"%{letter}{_WIDTH.get(width, '')}{byte_no}"

def parse_ident_container(xml_text):
    tags = []
    for ident in ET.fromstring(xml_text).findall("Ident"):
        acc = ident.find("./Access/SimpleAccess")
        if acc is None:
            continue
        byte_no = int(acc.get("ByteNumber", "0"))
        bit_raw = acc.get("BitNumber")
        tags.append({
            "name": ident.get("Name"),
            "type": ident.findtext("SimpleType"),
            "lid": int(ident.get("LID", "0")),
            "address": _address(acc.get("Range"), acc.get("Width"),
                                byte_no, int(bit_raw) if bit_raw else None),
        })
    return tags

Result against the running G2 (verified against TIA Portal):

Name Type Address LID
input_1 Bool %I0.0 10
output_1 Bool %Q0.1 9
merker_1 Bool %M0.2 9
mtag_byte Byte %MB100 11
mtag_word Word %MW102 13
mtag_dword DWord %MD104 14
mtag_bool Bool %M108.0 16

This makes the whole oracle pipeline from #757 (_build_fdict / regex _extract_tags / symbolic-READ width workaround) obsolete — the <SimpleType> now carries the real M widths directly.

I'm opening a small stacked PR on top of this branch that adds a non-invasive tag_browser.py (the parser above + an EXPLORE-per-area helper), so you can wire it into browse() however you prefer — or just fold it into this PR. Whatever's least noisy for you. Only G2-validated on my end, flagging that honestly. Thanks again for the complete dictionaries.

tommasofaedo added a commit to tommasofaedo/python-snap7 that referenced this pull request Jul 20, 2026
…RE XML

Follow-up to gijzelaerr#776: wires find_and_decompress() into a small non-invasive
tag layer. Parses the clean <IdentContainer> XML into Tag records with TIA
addresses (%I0.0, %Q0.1, %MB100, %MW102, %MD104). Includes a regression
test built from a real S7-1200 G2 (FW V4.1) M-area EXPLORE blob.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants