Implemented OTLP for the metrics crate to standardize metrics collection

[LorenzoTettamanti]

Changes

• Implemented OTLP protocol for the metrics service
• metrics.yaml and otel-agent.yaml
• Introduced otel_init.rs and otel_metrics.rs to handle the otel initialization and the collection of metrics in modo efficiente

Metrics:

Metrics	Type	Description
events_total	Counter<u64>	Total number of eBPF events processed across all perf buffers
packets_total	Counter<u64>	Total number of network-related events produced by the net_metrics
sk_drops	Gauge<i64>	Observed socket drop count (sk_drops) from the kernel sock struct.
sk_err	Gauge<i64>	Observed socket error count (sk_err) from the kernel sock struct.
delta_us	Histogram<u64>	Histogram of delta_us values supplied by the time_stamp_events perf buffer.
ts_us	Histogram<u64>	Histogram of ts_us values seen in both net_metrics and time_stamp_events

Testing:

environment: minikube with --cni=cilium --driver= docker
metrics_image: lorenzotettamanti/cortexflow-metrics:otel-test-1

you should see a MetricsExporter in the otel-agent pod

[siddh34]

@LorenzoTettamanti

I'm unable to run metrics image here

manifest for lorenzotettamanti/cortexflow-metrics:otel-test-1

{
"schemaVersion": 2,
"mediaType": "application/vnd.oci.image.index.v1+json",
"manifests": [
{
"mediaType": "application/vnd.oci.image.manifest.v1+json",
"size": 1243,
"digest": "sha256:de99d019f806bff96b53548c8804c4e419bd77a5148afdfdeace6439c290075d",
"platform": {
"architecture": "amd64",
"os": "linux"
}
},
{
"mediaType": "application/vnd.oci.image.manifest.v1+json",
"size": 564,
"digest": "sha256:ae33194edaef7f03fea1b0ec46a84c1e5c40e8ba634e7fae1f9444d70a6833ce",
"platform": {
"architecture": "unknown",
"os": "unknown"
}
}
]
}

Manifest for lorenzotettamanti/cortexflow-identity:0.1.2

{
"schemaVersion": 2,
"mediaType": "application/vnd.docker.distribution.manifest.v2+json",
"config": {
"mediaType": "application/vnd.docker.container.image.v1+json",
"size": 2971,
"digest": "sha256:e942e039853eb1dd03c8d0509dc23df1209d060127ae74d8a19f16b59c036f39"
},
"layers": [
{
"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
"size": 30595557,
"digest": "sha256:e93fce65fb9fdbf95d95433f0a49637000c985ffab1470a3bd7dc626ac7f1c06"
},
{
"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
"size": 1050612,
"digest": "sha256:2b8f9011290366c4a49905e10bc0098f7aa915e7c0709327757d914f5fc2dba5"
},
{
"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
"size": 156,
"digest": "sha256:0e09eaf03e1de2644d952e5f95139d30db617864e5f335d2ba3f09a8def9cb29"
},
{
"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
"size": 4749693,
"digest": "sha256:2cb1c84cd6acdfcd6f70433abf78970c6c17fdf18fa7a28a6383ce86d09a85d9"
},
{
"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
"size": 3721,
"digest": "sha256:377d1f65347f243187104bf1bcf15666dee9d582d11c030d198d1f249f52709e"
}
]
}

could you please check it?

[LorenzoTettamanti]

@siddh34 I've tried to follow this discussion. I set the --provenance=false and the --sbom=false in the build command.

The new image is this: lorenzotettamanti/cortexflow-metrics:otel-test-2

[siddh34]

please replace with otel-test-2

it is working fine

[LorenzoTettamanti]

done!

[siddh34]

Awesome!

[siddh34]

@LorenzoTettamanti otel is working fine

from metrics pod

from identity pod

from otel agent pod

[LorenzoTettamanti]

@siddh34 I'm merging the changes and i'm closing issue #175