Skip to content

Commit 38f0cc3

Browse files
author
Kostiantyn Krykun
committed
docs: Add Tekton Results to docs
1 parent d7154e8 commit 38f0cc3

8 files changed

Lines changed: 164 additions & 516 deletions

File tree

42.5 KB
Loading

docs/operator-guide/ci/tekton-long-term-storage.md

Lines changed: 41 additions & 129 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
---
2-
title: "Tekton Long-Term Log Storage"
3-
description: "Learn how to configure and access long-term logs for Tekton pipelines in KubeRocketCI using OpenSearch, enhancing log management and observability."
4-
sidebar_label: "Tekton Long-Term Log Storage"
2+
title: "Tekton Results Integration"
3+
description: "Tekton Results helps you logically group CI/CD workload history and separate long-term result storage from the Pipeline controller: add custom Result metadata (e.g. post-run actions), group related TaskRuns and PipelineRuns, keep result history independent of the Pipeline CRD controller to free etcd resources, and store logs so completed Runs can be cleaned."
4+
sidebar_label: "Tekton Results Integration"
55
---
66
<!-- markdownlint-disable MD025 -->
77

@@ -14,151 +14,63 @@ import TabItem from '@theme/TabItem';
1414
<link rel="canonical" href="https://docs.kuberocketci.io/docs/operator-guide/ci/tekton-long-term-storage" />
1515
</head>
1616

17-
KubeRocketCI Portal provides the capability to view long-term logs for pipelines. The long-term logs are stored in the OpenSearch cluster and can be viewed in the KubeRocketCI Portal after pipeline cleanup or deletion. This guide describes how to configure and view long-term logs in the KubeRocketCI Portal.
17+
Tekton Results aims to help users logically group CI/CD workload history and separate out long term result storage away from the Pipeline controller. This allows you to:
18+
19+
- Provide custom Result metadata about your CI/CD workflows not available in the Tekton TaskRun/PipelineRun CRDs (for example: post-run actions)
20+
- Group related workloads together (e.g. bundle related TaskRuns and PipelineRuns into a single unit)
21+
- Make long-term result history independent of the Pipeline CRD controller, letting you free up etcd resources for Run execution.
22+
- Store logs produced by the TaskRuns/PipelineRuns so that completed Runs can be cleaned to save resources.
1823

1924
## Long-Term Log Access Workflow
2025

2126
The following diagram illustrates the workflow for accessing long-term logs for pipelines in the KubeRocketCI Portal:
2227

23-
```mermaid
24-
sequenceDiagram
25-
actor U as User
26-
participant KR as KubeRocketCI Portal
27-
participant PC as Pipeline Controller
28-
participant RW as Fluent Bit Agent
29-
participant KD as KrakenD API Gateway
30-
participant RA as OpenSearch
31-
32-
U->>KR: Trigger or create a pipeline
33-
KR->>PC: Start pipeline execution
34-
RW-->>PC: Collect pipeline logs
35-
RW->>RA: Send logs to OpenSearch
36-
Note over KR,RA: Pipeline is deleted or cleaned up
37-
U->>KR: View pipeline logs
38-
KR-->>KD: Request long-term logs
39-
KD-->>RA: Fetch long-term logs
40-
KR->>U: Return long-term logs
41-
```
42-
43-
## Configuration
44-
45-
To configure long-term log storage for pipelines in the KubeRocketCI Portal, follow the steps below:
46-
47-
1. Install and configure OpenSearch cluster:
48-
49-
:::note
50-
To install OpenSearch cluster, we recommend to use the [edp-cluster-add-ons](https://github.com/epam/edp-cluster-add-ons/tree/main/clusters/core/addons/opensearch) repository, which contains already prepared configurations for OpenSearch installation.
51-
:::
52-
53-
To install OpenSearch cluster using the add-ons repository, follow the steps below:
54-
55-
1. Clone the forked [edp-cluster-add-ons](https://github.com/epam/edp-cluster-add-ons/) repository.
56-
57-
2. Navigate to the `clusters/core/addons/opensearch` directory and configure the `values.yaml` file with the necessary values for OpenSearch cluster installation.
58-
59-
:::note
60-
Ensure that [Fluent Bit](https://github.com/epam/edp-cluster-add-ons/blob/44ca88c079d464c826fcae38f3f03fe983d1f984/clusters/core/addons/opensearch/values.yaml#L391) is configured to send container logs to the `logstash-edp` index in the OpenSearch cluster, as the KubeRocketCI Portal uses this index to retrieve long-term logs.
61-
:::
62-
63-
3. Install the OpenSearch cluster using Helm or [Argo CD](../add-ons-overview.md) methods.
64-
65-
2. Install and configure KrakenD API Gateway:
66-
67-
:::note
68-
To install KrakenD API Gateway with OpenSearch connection, we recommend to use the [edp-cluster-add-ons](https://github.com/epam/edp-cluster-add-ons/tree/main/clusters/core/addons/krakend) repository, which contains already prepared configurations for KrakenD installation.
69-
For more details, refer to the [KrakenD Integration](../extensions/krakend.md) guide.
70-
:::
71-
72-
To install KrakenD API Gateway with OpenSearch connection, follow the steps below:
73-
74-
1. Clone the forked [edp-cluster-add-ons](https://github.com/epam/edp-cluster-add-ons/) repository.
75-
76-
2. Navigate to the `clusters/core/addons/krakend` directory and configure the `values.yaml` file with the necessary values for KrakenD installation.
77-
78-
:::note
79-
Ensure that the KrakenD endpoint object for OpenSearch has the `/search/logs` endpoint name, as the KubeRocketCI Portal uses this endpoint to fetch long-term logs.
80-
:::
28+
![KubeRocketCI Portal Diagram](../../assets/operator-guide/ci/diagram-tekton-results.png "KubeRocketCI Portal Diagram")
8129

82-
3. Ensure that the KrakenD [configuration secret](https://github.com/epam/edp-cluster-add-ons/blob/main/clusters/core/addons/krakend/templates/external-secrets/krakend.yaml) contains the OpenSearch connection variables.
30+
## Install Tekton Results
8331

84-
<Tabs
85-
defaultValue="externalsecret"
86-
values={[
87-
{label: 'Manifests', value: 'manifests'},
88-
{label: 'External Secrets Operator', value: 'externalsecret'},
89-
]}>
32+
Tekton Results is deployed as part of the Tekton Pipelines installation. For installing Tekton Pipelines, we recommend using the [add-ons approach](https://github.com/epam/edp-cluster-add-ons/blob/4456631feb6510ff875b5b839534968c3846da7a/clusters/core/apps/values.yaml#L286C1-L289C30). Here's an example of how to configure values.yaml before installation:
9033

91-
<TabItem value="manifests">
34+
```yaml
35+
tekton:
36+
createNamespace: true
37+
enable: true
38+
namespace: tekton-pipelines
39+
```
9240

93-
```yaml
94-
apiVersion: v1
95-
kind: Secret
96-
metadata:
97-
name: krakend
98-
namespace: krakend
99-
type: Opaque
100-
stringData:
101-
OPENSEARCH_URL: https://opensearch-cluster-master.logging:9200
102-
OPENSEARCH_CREDS: <base64-encoded-credentials>
103-
```
104-
105-
</TabItem>
106-
107-
<TabItem value="externalsecret">
108-
109-
```json
110-
{
111-
"OPENSEARCH_URL": "https://opensearch-cluster-master.logging:9200",
112-
"OPENSEARCH_CREDS": "<base64-encoded-credentials>"
113-
}
114-
```
115-
</TabItem>
116-
117-
</Tabs>
118-
119-
4. Install the KrakenD API Gateway using Helm or [Argo CD](../add-ons-overview.md) methods.
120-
121-
## Viewing Long-Term Logs
122-
123-
After configuring long-term log storage for Pipelines in the KubeRocketCI Portal, follow the steps below to view long-term logs:
124-
125-
1. Navigate to the KubeRocketCI Portal and sign in with appropriate credentials.
126-
127-
![KubeRocketCI Portal](../../assets/operator-guide/ci/kuberocketci-portal.png "KubeRocketCI Portal")
128-
129-
2. In the **Components** section, navigate to the appropriate component (if component is not created yet, refer to the [Add Application](../../user-guide/add-application.md) guide to create a new one).
130-
131-
![KubeRocketCI Portal Components](../../assets/operator-guide/ci/portal-components.png "KubeRocketCI Portal Components")
132-
133-
3. In the component window, in the right upper corner, click the **GIT** button to navigate to the component repository (e.g., GitHub repository).
134-
135-
![Component Git Repository](../../assets/operator-guide/ci/component-git-repository.png "Component Git Repository")
136-
137-
4. In the component repository, create a Pull Request to trigger the review pipeline execution.
138-
139-
![Create Pull Request](../../assets/operator-guide/ci/create-pull-request.png "Create Pull Request")
140-
141-
5. In the Pull Request checks section, click the **Show all checks** button and navigate to the review pipeline in the KubeRocketCI Portal by clicking the **Details** button.
41+
## Configuration
14242

143-
![PR Checks](../../assets/operator-guide/ci/pr-checks.png "PR Checks")
43+
To configure long-term log storage for pipelines in the KubeRocketCI Portal, follow the steps below:
14444

145-
6. In the pipeline window, check the pipeline status. Ensure that the pipeline is successfully executed and the logs are available.
45+
1. Storage sizes in the [results-pg.yaml](https://github.com/epam/edp-cluster-add-ons/blob/main/clusters/core/addons/tekton/results-pg.yaml) file define the allocated volumes for the Tekton Results PostgreSQL cluster:
14646

147-
![Pipeline Logs](../../assets/operator-guide/ci/pipeline-logs.png "Pipeline Logs")
47+
* Database instance storage
48+
spec.instances.dataVolumeClaimSpec.resources.requests.storage
49+
(default: 2Gi) — Specifies the size of the main PostgreSQL data volume.
14850

149-
7. Navigate to the **Pipelines** section and find the executed review pipeline. Delete the pipeline by clicking the **Delete** button.
51+
* Backup repository storage
52+
spec.backups.pgbackrest.repos.volume.volumeClaimSpec.resources.requests.storage
53+
(default: 2Gi) — Specifies the size of the pgBackRest backup repository volume.
15054

151-
![Delete Pipeline](../../assets/operator-guide/ci/delete-pipeline.png "Delete Pipeline")
55+
2. Storage and retention settings in the results.yaml [results.yaml](https://github.com/epam/edp-cluster-add-ons/blob/main/clusters/core/addons/tekton/results.yaml) file define how Tekton Results logs and data are stored and managed:
15256

153-
8. Return to the Pull Request and click the **Show all checks** button. Navigate to the review pipeline again by clicking the **Details** button.
57+
- Log storage size
58+
PersistentVolumeClaim.spec.resources.requests.storage
59+
(default: 5Gi) — Specifies the allocated disk size for storing pipeline logs.
15460

155-
![PR Checks](../../assets/operator-guide/ci/pr-checks.png "PR Checks")
61+
- Retention period
62+
ConfigMap.data.defaultRetention in tekton-results-config-results-retention-policy
63+
(default: "30" days) — Defines how long log files are retained before being automatically removed.
15664

157-
9. After the pipeline deletion, the long-term logs are displayed in the pipeline window.
65+
- Cleanup schedule
66+
ConfigMap.data.runAt in tekton-results-config-results-retention-policy
67+
(default: "5 5 * * 0") — Specifies the cron expression that determines when automated log cleanup runs (every Sunday at 5:05 AM).
15868

159-
![Long Term Logs](../../assets/operator-guide/ci/reserved-logs.png "Long Term Logs")
69+
:::warning
70+
The retention policy agent removes only database records from PostgreSQL. It does not delete associated log files stored on persistent volumes, S3, or GCS backends. As a result, log data may remain on disk and consume storage even after the corresponding records are expired.
16071

161-
This example demonstrates how to view long-term logs for review pipeline, but the same approach can be applied to other pipeline types.
72+
Users are currently responsible for manually cleaning up unused log files from the underlying storage backends.
73+
:::
16274

16375
## Related Articles
16476

42.5 KB
Loading

0 commit comments

Comments
 (0)