[DPE-10791] feat(config): own the patroni template and its render target (3/6) - #183
Conversation
cba92af to
fdf2a44
Compare
a7ee622 to
786d9bd
Compare
fdf2a44 to
ef76d9a
Compare
786d9bd to
ddf2305
Compare
ef76d9a to
d994db8
Compare
…tion Internalizes PostgreSQL parameter calculation (worker-process auto/cap rules, wal_compression, memory-limit conversion) into the library so ConfigManager owns the full config-build flow instead of stopping at a TODO. Resource discovery (cpu_cores, memory_bytes) moves onto the workload rather than a substrate-specific manager: get_available_memory already lives there, ConfigManager already holds self.workload, and K8sManager doesn't exist yet when ConfigManager is constructed in abstract_charm - so the workload is the only place both substrates can share a single no-branch call site. K8sWorkload gains required unit_name/namespace constructor args so its lightkube Pod/Node lookups can resolve "my own pod" without depending on a charm object reference, mirroring what the K8s charm does today via self.unit.name/self.model.name. lightkube becomes a real (not merely transitive) dependency via a new k8s pyproject extra, matching the version pins the K8s test charm already uses. Signed-off-by: Marcelo Henrique Neppel <marcelo.neppel@canonical.com>
ddf2305 to
f9a1111
Compare
d994db8 to
f5d55f2
Compare
The parameter-building port placed lightkube node/pod resource lookups on K8sWorkload and injected unit_name/namespace into its constructor. Both duplicate identity the state already holds and put K8s-API code on the workload abstraction, which reviewers flagged. Move the lookups onto K8sManager, which is already K8s-only-loaded and holds the state, sourcing pod/namespace from state.peer.unit_name and state.model_name. ConfigManager reaches the value through a new state.available_resources property backed by a per-substrate provider (the VM workload, or the K8s manager), so the call site stays substrate-agnostic and lightkube never enters the VM import graph. Behavior is unchanged: node allocatable clamped by the container limit, 403 -> DeployedWithoutTrustError. The Node lookups drop the namespace argument, which lightkube ignores for cluster-scoped resources. Signed-off-by: Marcelo Henrique Neppel <marcelo.neppel@canonical.com>
…ditional template The two charms each carried their own templates/patroni.yml.j2 and the config manager loaded it with a CWD-relative open(), so the lib could not own the render. Merge both copies into a single template the lib ships and loads as package data, so the charms delete their copies in the adoption PR and the template travels with the code that renders it rather than with each charm's working directory. The VM and K8s templates diverge in section ordering, whole blocks (raft, bypass_api_service, pod_ip), path variables, and pg_hba rules, so the merged template branches at the top level on a substrate context var with each substrate's body kept verbatim. That guarantees byte-for-byte identical output per substrate and avoids interleaving conditionals that could silently shift a byte. render_patroni_yml_file now loads the template via importlib.resources and passes the substrate the template branches on. Golden tests render the merged template against byte copies of each charm's original template across the full matrix of conditional dimensions (tls, connectivity, ldap, restore/pitr, slots, peers, watcher, extra replication endpoints, tags) and assert equality, so any divergent byte fails the suite. Signed-off-by: Marcelo Henrique Neppel <marcelo.neppel@canonical.com>
f5d55f2 to
79f2a4f
Compare
The renderer wrote patroni.yaml on both substrates while the K8s Pebble layer starts Patroni with patroni.yml, so on K8s the config subsystem would drive a file Patroni never reads: a fresh pod misses its config, an upgraded one keeps running against a stale one, and every later TLS, HBA or restore change silently never reaches the cluster. The two names are not interchangeable — each is what its substrate has always used (snap: patroni.yaml, K8s charm: patroni.yml) — so the file name belongs to the paths object, where render and launch both read it from one place and cannot drift apart again. Signed-off-by: Marcelo Henrique Neppel <marcelo.neppel@canonical.com>
The template gates the watcher's health-check rule on watcher_addr, but only watcher — the ip:port Raft address — was ever supplied, so Jinja resolved watcher_addr to undefined and the rule was dropped: a related watcher joins Raft yet is refused by PostgreSQL when it probes. The pristine VM charm has the same omission, so this is not a migration regression; the port is where it becomes cheap to fix, since the address is already in hand and only needs its port trimmed. Signed-off-by: Marcelo Henrique Neppel <marcelo.neppel@canonical.com>
The `'hostssl' if enable_tls else 'host'` ternary was spelled out at all 41 pg_hba rule sites, 40 characters of ceremony ahead of the part a reader is actually there for, pushing every rule off the left margin. Binding it once says the same thing in the name and lets the rules read as pg_hba again. Signed-off-by: Marcelo Henrique Neppel <marcelo.neppel@canonical.com>
Each substrate spelled out replication, rewind and superuser as three copies of the same stanza, so the TLS-guarded certificate lines existed six times across the file and any change to them had to be made six times. Driving the three roles from a list keeps the stanza in one place, and the substrates keep their own copy because their certificate handling genuinely differs: VM emits sslrootcert and always emits the client pair, K8s emits neither unless TLS is on. Signed-off-by: Marcelo Henrique Neppel <marcelo.neppel@canonical.com>
The seven charmed_* roles were seven rule lines per substrate that differed only in the role name, so adding a role meant editing two places and the grant itself was buried in fourteen near-identical lines. Listing the roles once puts the set of roles where a reader looks for it and leaves one rule to read. Signed-off-by: Marcelo Henrique Neppel <marcelo.neppel@canonical.com>
The template was the two charm templates concatenated under a substrate conditional, so roughly three quarters of it existed twice: the same 43 PostgreSQL parameters, the same restore command, the same pg_hba rules, each needing every edit applied in two places. The two copies had already drifted apart in ways nobody intended, down to a different indent for the same list. Binding the values that genuinely vary per substrate once at the top collapses that to a single document with conditionals only where the substrates really differ - the raft section, the watcher rule, the Kubernetes-only keys and the asymmetric authentication block. Field order now follows one sequence rather than each charm's, which changes the rendered layout; the parsed configuration is unchanged, verified across the full render matrix. Signed-off-by: Marcelo Henrique Neppel <marcelo.neppel@canonical.com>
The rule this fed is unreachable. The user_databases_map loop emits `postgres watcher 0.0.0.0/0` earlier in the same list whenever a watcher is related - which is exactly when the watcher host is non-null - and pg_hba takes the first matching record, so the narrower /32 rule below it never authorises a connection the broader one has not already allowed. The charm omits the same variable, so restoring the omission keeps this a move rather than a behaviour change, and leaves the dead rule visible for whoever decides what it was meant to do. Signed-off-by: Marcelo Henrique Neppel <marcelo.neppel@canonical.com>
K8s omits the username for the replication and superuser roles, which the merged template expressed as a truthiness test. An empty username would then be dropped on VM too, where the charm template always emitted the key. Testing for none keeps the omission to the roles that mean it. The values are non-empty constants today, so nothing rendered differently - this closes a gap the golden matrix cannot see, since it pins both usernames at one value. Signed-off-by: Marcelo Henrique Neppel <marcelo.neppel@canonical.com>
bin_dir already followed the running major version while the WAL, PostgreSQL log and Patroni log paths spelled 16 out, so a 17 workload would have kept three directories pointing at a 16 tree. The version is already in the render context, so this only reaches for what was there. Nothing renders differently on 16, which is also why the golden matrix cannot see the difference - it pins the version at 16. Signed-off-by: Marcelo Henrique Neppel <marcelo.neppel@canonical.com>
…6/edge Shipped 16/edge launches patroni.yaml on both substrates (K8s at charm.py:2345, VM via the lib 16/edge already consumes). The .yml split matched main, a different track, and would rename the config file on a 16/edge K8s refresh — risking a window where Patroni restarts before the first render and finds no config. Unify patroni_config on patroni.yaml for both substrates so the on-disk filename is unchanged across refresh. Signed-off-by: Marcelo Henrique Neppel <marcelo.neppel@canonical.com>
Both substrates returned the same value (patroni_conf / "patroni.yaml") through identical overrides, differing only in docstring. Move it to one concrete implementation on the base class and drop the per-substrate overrides, so the file name lives in one place and cannot drift apart. Signed-off-by: Marcelo Henrique Neppel <marcelo.neppel@canonical.com>
akram09
left a comment
There was a problem hiding this comment.
Seems good for me!
One thing maybe regarding how we are getting the available resources, I am not sure if the best solution would be to pass through the state. Do you think maybe we can have the K8sManager as K8sClient and use the client in the Managers ?
Yes, @akram09. The lightkube lookups moved onto @dragomirp‚ this reverses the state route from #182, so say if you'd rather keep it. |
Issue
Each charm ships its own
templates/patroni.yml.j2, loaded from the current working directory. The library must own the template and load it as package data sorender_patroni_yml_fileworks regardless of CWD.Owning the render surface exposed a mismatch between what gets written and what reads it: the renderer wrote
patroni.yamlon both substrates while K8s starts Patroni frompatroni.yml, so on K8s the config subsystem drove a file Patroni never reads — a fresh pod comes up without its config, an upgraded one keeps running against a stale copy, and every later TLS, HBA or restore change silently never reaches the cluster.Solution
patroni.yml.j2into one lib-ownedsingle_kernel_postgresql/templates/patroni.yml.j2that renders either substrate from a single document. The values that genuinely vary — paths, addresses, the pgBackRest config flag — are bound once at the top; conditionals remain only where the substrates really differ: the Raft section, the watcher rule, the Kubernetes-only keys and the asymmetric authentication block.importlib.resourcesinstead of a CWD-relativeopen(); add"substrate"to the render context.patroni_config—patroni.yamlon both substrates, matching shipped16/edgeso the on-disk filename is unchanged across a refresh — and read it from the renderer, the K8s Pebble layer andconfigure_patroni_on_unit, so what is written and what is started cannot drift apart again.16out.bin_diralready followed the version, so on a 17 workload the WAL, PostgreSQL-log and Patroni-log directories would have kept pointing at a 16 tree. Nothing renders differently on 16.usernameon absence rather than falsiness, so collapsing the three role stanzas into one loop keeps the omission to the roles that mean it — K8s drops the key for replication and superuser, VM never does.Rendered layout changes: on VM the output loses only blank lines, with every non-whitespace line identical and in the same order; on K8s the top-level keys are emitted in one sequence rather than that charm's own. The configuration itself is unchanged, and nothing downstream reads the file as text — the config hash is taken over the Juju config options, not the rendered output — so this carries no restart. The golden matrix in #188 compares the parsed documents against the preserved pre-merge charm templates, so any changed value, added key or reordered
pg_hbarule fails the suite.Two references the charms leave undefined are preserved rather than fixed, so this stays a move:
conf_pathon K8s andwatcher_addron VM both render empty, exactly as they do today. The watcher's health-check rule thatwatcher_addrgates is unreachable in any case — theuser_databases_maploop emits a broader grant for the same user and database earlier in the list, andpg_hbatakes the first match.Third (code-only) PR in the update_config stack. The golden fixtures and the per-substrate render matrix live in the stack's test PR (#188), along with the minimal update to the existing render test carried here to keep the suite green.
Checklist