Skip to content

Add Tekton Pipelines guide for NERC OpenShift#324

Open
kpwrennbu wants to merge 1 commit into
nerc-project:mainfrom
kpwrennbu:tekton-pipelines-nerc-openshift
Open

Add Tekton Pipelines guide for NERC OpenShift#324
kpwrennbu wants to merge 1 commit into
nerc-project:mainfrom
kpwrennbu:tekton-pipelines-nerc-openshift

Conversation

@kpwrennbu
Copy link
Copy Markdown
Contributor

Summary

  • Adds a new page documenting how to install and use Tekton Pipelines on NERC OpenShift
  • Uses the DS-551 Data Engineering at Scale course (Boston University, Spring 2026) as a real working deployment reference — 12 student teams ran concurrent pipelines on NERC for a full semester
  • Placed under OpenShift → Pipeline Orchestration in the nav (new section, after Serverless Computing)

What's covered

  • Operator install via OperatorHub subscription
  • ServiceAccount + RBAC setup scoped to pipeline needs
  • Task, Pipeline, and PipelineRun definitions with real YAML from the course
  • CronJob-triggered recurring runs
  • Observability commands (oc get/describe/logs, Tekton dashboard port-forward)
  • Lessons learned from running 12 namespaces concurrently (SA scope, consumer group collisions, pip cold-start cost, task timeouts, PipelineRun accumulation)
  • Reference architecture diagram of the full NERC cluster layout

Test plan

  • Verify mkdocs.yaml nav entry renders correctly
  • Review YAML snippets for correctness against Tekton v1 API
  • Confirm links to tekton.dev, Strimzi, and NERC docs are valid

🤖 Generated with Claude Code

Documents how to install and use Tekton Pipelines on NERC OpenShift,
using the DS-551 Data Engineering at Scale course (Boston University,
Spring 2026) as a working reference deployment. Covers operator
install, ServiceAccount/RBAC, Task/Pipeline/PipelineRun definitions,
CronJob scheduling, observability commands, lessons learned from
running 12 concurrent student pipelines, and a reference architecture
diagram of the full cluster layout.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- [Tekton Pipelines documentation](https://tekton.dev/docs/pipelines/)
- [Strimzi Kafka Operator on OpenShift](https://strimzi.io/docs/operators/latest/overview.html)
- [NERC OpenShift documentation](https://nerc-project.github.io/nerc-docs/)
- DS-551 course repository: `github.com/langd0n-classes/data-eng-at-scale`
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good but needs to be run through the linter. You can run pre-commit run --all-files

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants