You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
description: "Learn how to configure and access long-term logs for Tekton pipelines in KubeRocketCI using OpenSearch, enhancing log management and observability."
4
-
sidebar_label: "Tekton Long-Term Log Storage"
2
+
title: "Tekton Results Integration"
3
+
description: "Tekton Results helps you logically group CI/CD workload history and separate long-term result storage from the Pipeline controller: add custom Result metadata (e.g. post-run actions), group related TaskRuns and PipelineRuns, keep result history independent of the Pipeline CRD controller to free etcd resources, and store logs so completed Runs can be cleaned."
4
+
sidebar_label: "Tekton Results Integration"
5
5
---
6
6
<!-- markdownlint-disable MD025 -->
7
7
@@ -14,151 +14,63 @@ import TabItem from '@theme/TabItem';
KubeRocketCI Portal provides the capability to view long-term logs for pipelines. The long-term logs are stored in the OpenSearch cluster and can be viewed in the KubeRocketCI Portal after pipeline cleanup or deletion. This guide describes how to configure and view long-term logs in the KubeRocketCI Portal.
17
+
Tekton Results aims to help users logically group CI/CD workload history and separate out long term result storage away from the Pipeline controller. This allows you to:
18
+
19
+
- Provide custom Result metadata about your CI/CD workflows not available in the Tekton TaskRun/PipelineRun CRDs (for example: post-run actions)
20
+
- Group related workloads together (e.g. bundle related TaskRuns and PipelineRuns into a single unit)
21
+
- Make long-term result history independent of the Pipeline CRD controller, letting you free up etcd resources for Run execution.
22
+
- Store logs produced by the TaskRuns/PipelineRuns so that completed Runs can be cleaned to save resources.
18
23
19
24
## Long-Term Log Access Workflow
20
25
21
26
The following diagram illustrates the workflow for accessing long-term logs for pipelines in the KubeRocketCI Portal:
22
27
23
-
```mermaid
24
-
sequenceDiagram
25
-
actor U as User
26
-
participant KR as KubeRocketCI Portal
27
-
participant PC as Pipeline Controller
28
-
participant RW as Fluent Bit Agent
29
-
participant KD as KrakenD API Gateway
30
-
participant RA as OpenSearch
31
-
32
-
U->>KR: Trigger or create a pipeline
33
-
KR->>PC: Start pipeline execution
34
-
RW-->>PC: Collect pipeline logs
35
-
RW->>RA: Send logs to OpenSearch
36
-
Note over KR,RA: Pipeline is deleted or cleaned up
37
-
U->>KR: View pipeline logs
38
-
KR-->>KD: Request long-term logs
39
-
KD-->>RA: Fetch long-term logs
40
-
KR->>U: Return long-term logs
41
-
```
42
-
43
-
## Configuration
44
-
45
-
To configure long-term log storage for pipelines in the KubeRocketCI Portal, follow the steps below:
46
-
47
-
1. Install and configure OpenSearch cluster:
48
-
49
-
:::note
50
-
To install OpenSearch cluster, we recommend to use the [edp-cluster-add-ons](https://github.com/epam/edp-cluster-add-ons/tree/main/clusters/core/addons/opensearch) repository, which contains already prepared configurations for OpenSearch installation.
51
-
:::
52
-
53
-
To install OpenSearch cluster using the add-ons repository, follow the steps below:
54
-
55
-
1. Clone the forked [edp-cluster-add-ons](https://github.com/epam/edp-cluster-add-ons/) repository.
56
-
57
-
2. Navigate to the `clusters/core/addons/opensearch` directory and configure the `values.yaml` file with the necessary values for OpenSearch cluster installation.
58
-
59
-
:::note
60
-
Ensure that [Fluent Bit](https://github.com/epam/edp-cluster-add-ons/blob/44ca88c079d464c826fcae38f3f03fe983d1f984/clusters/core/addons/opensearch/values.yaml#L391) is configured to send container logs to the `logstash-edp` index in the OpenSearch cluster, as the KubeRocketCI Portal uses this index to retrieve long-term logs.
61
-
:::
62
-
63
-
3. Install the OpenSearch cluster using Helm or [Argo CD](../add-ons-overview.md) methods.
64
-
65
-
2. Install and configure KrakenD API Gateway:
66
-
67
-
:::note
68
-
To install KrakenD API Gateway with OpenSearch connection, we recommend to use the [edp-cluster-add-ons](https://github.com/epam/edp-cluster-add-ons/tree/main/clusters/core/addons/krakend) repository, which contains already prepared configurations for KrakenD installation.
69
-
For more details, refer to the [KrakenD Integration](../extensions/krakend.md) guide.
70
-
:::
71
-
72
-
To install KrakenD API Gateway with OpenSearch connection, follow the steps below:
73
-
74
-
1. Clone the forked [edp-cluster-add-ons](https://github.com/epam/edp-cluster-add-ons/) repository.
75
-
76
-
2. Navigate to the `clusters/core/addons/krakend` directory and configure the `values.yaml` file with the necessary values for KrakenD installation.
77
-
78
-
:::note
79
-
Ensure that the KrakenD endpoint object for OpenSearch has the `/search/logs` endpoint name, as the KubeRocketCI Portal uses this endpoint to fetch long-term logs.
3. Ensure that the KrakenD [configuration secret](https://github.com/epam/edp-cluster-add-ons/blob/main/clusters/core/addons/krakend/templates/external-secrets/krakend.yaml) contains the OpenSearch connection variables.
Tekton Results is deployed as part of the Tekton Pipelines installation. For installing Tekton Pipelines, we recommend using the [add-ons approach](https://github.com/epam/edp-cluster-add-ons/blob/4456631feb6510ff875b5b839534968c3846da7a/clusters/core/apps/values.yaml#L286C1-L289C30). Here's an example of how to configure values.yaml before installation:
2. In the **Components** section, navigate to the appropriate component (if component is not created yet, refer to the [Add Application](../../user-guide/add-application.md) guide to create a new one).
5. In the Pull Request checks section, click the **Show all checks** button and navigate to the review pipeline in the KubeRocketCI Portal by clicking the **Details** button.
To configure long-term log storage for pipelines in the KubeRocketCI Portal, follow the steps below:
144
44
145
-
6. In the pipeline window, check the pipeline status. Ensure that the pipeline is successfully executed and the logs are available.
45
+
1. Storage sizes in the [results-pg.yaml](https://github.com/epam/edp-cluster-add-ons/blob/main/clusters/core/addons/tekton/results-pg.yaml) file define the allocated volumes for the Tekton Results PostgreSQL cluster:
2. Storage and retention settings in the results.yaml [results.yaml](https://github.com/epam/edp-cluster-add-ons/blob/main/clusters/core/addons/tekton/results.yaml) file define how Tekton Results logs and data are stored and managed:
152
56
153
-
8. Return to the Pull Request and click the **Show all checks** button. Navigate to the review pipeline again by clicking the **Details** button.
ConfigMap.data.defaultRetention in tekton-results-config-results-retention-policy
63
+
(default: "30" days) — Defines how long log files are retained before being automatically removed.
156
64
157
-
9. After the pipeline deletion, the long-term logs are displayed in the pipeline window.
65
+
- Cleanup schedule
66
+
ConfigMap.data.runAt in tekton-results-config-results-retention-policy
67
+
(default: "5 5 * * 0") — Specifies the cron expression that determines when automated log cleanup runs (every Sunday at 5:05 AM).
158
68
159
-

69
+
:::warning
70
+
The retention policy agent removes only database records from PostgreSQL. It does not delete associated log files stored on persistent volumes, S3, or GCS backends. As a result, log data may remain on disk and consume storage even after the corresponding records are expired.
160
71
161
-
This example demonstrates how to view long-term logs for review pipeline, but the same approach can be applied to other pipeline types.
72
+
Users are currently responsible for manually cleaning up unused log files from the underlying storage backends.
0 commit comments