docs: refactor service registration brief to HTTP traffic metering

[JoseSzycho]

Warning

In pause till #187 is on place
#181 (comment)

This PR refactors and expands the initial service catalog registration design brief into a structured, standard enhancement proposal focusing on HTTP Traffic Metering for Network Services.

It addresses and resolves all open questions (OD-1 through OD-8) carried over from the previous design brief.

Related to:

• docs: design brief for service catalog registration #156 (comment)

Working implementation using OTLP Transport:

• Feat/envoy logs #188

[scotwells]

Directionally the architecture looks good. Let's just clean up some of the implementation details. We want to keep this document product / consumer focused.

[scotwells]

Following on from the pipeline shape — the piece worth nailing down is how the log line gets the metadata embedded in the first place, since the project isn't something Envoy knows on its own. Project identity lives on the namespace (resourcemanager.miloapis.com/project-name, e.g. p-abc), so it has to be put onto the route before the access log can emit it.

The natural place to do that is the extension server proposed in #187. It already injects per-route filter_metadata for the WAF and Connector, already watches namespaces in a warm cache, and already maps the downstream ns-<uid> back to the upstream namespace — so adding the project name to route metadata is a small, additive change that stays out of the request path. Once it's on the route, the access log format references it with %METADATA(ROUTE:...)% and the line comes out with project (and whatever else we want) already embedded, ready for Vector to map to subject: projects/{name}.

So rather than enrich anywhere downstream, we piggyback on #187 to stamp the metadata at build time and let it fall through into the log line.

[JoseSzycho]

@scotwells I just updated the enhancement, and I have a working implementation here:

• Feat/envoy logs #188

Could you please review the Design Details I introduced in the previous commit?

I propose to different alternatives.

Will wait to hear from you in order to continue with the implementation.

[scotwells]

While this approach works, I believe the approach in #181 (comment) would be better since it doesn't require exposing billing metadata through headers seen by end users.

With the header approach, we would have to prevent the user from ever specifying this header in their request since we use it for billing attribution. Otherwise we could potentially bill the wrong project for an HTTP route.

[JoseSzycho]

thanks for the fast review. Will set this PR on draft and wait till the extension server to be on place for doing the final tweaks into the documentation.

[JoseSzycho]

@scotwells the extension servers works correctly. I followed the approach you recommended in #181 (comment), and was able to correctly inject the project_name in the metadata.

Now the project_name is available in the per request access_logs.

This PR #188 illustrates this. I crated the manifest for deploying the extension server in a dev environment, and was able to perform the e2e integration test for verifying the the updated Design Details described in this commit 320d46a

Will wait for your review so I can proceed with the complete implementation of metering the HTTP Usage.