Skip to content

FINAL_SUMMARY.md

Latest commit

 

History

History
436 lines (330 loc) · 11.9 KB

FINAL_SUMMARY.md

File metadata and controls

436 lines (330 loc) · 11.9 KB

Final Summary - shift-eks Setup

Complete summary of the multi-cluster Kubernetes development environment.

What You Have

Two enterprise-grade Kubernetes clusters running locally:

1. OpenShift (CRC)

  • Type: Full OpenShift 4.x cluster
  • Provider: CodeReady Containers
  • Resources: 16 GB RAM, 6 CPUs, 100 GB disk
  • Features: OpenShift web console, operators, full API
  • CLI: oc and kubectl
  • Requirement: Red Hat pull secret (free)

2. EKS Anywhere

  • Type: AWS-curated Kubernetes distribution
  • Provider: Docker provider (local development)
  • Resources: 4 GB RAM, 2 CPUs, 5 GB disk
  • Features: AWS-curated packages, upgrades, GitOps
  • CLI: eksctl-anywhere and kubectl
  • Requirement: None (free for Docker provider)

Architecture

┌─────────────────────────────────────────────────────────┐
│ Host Machine                                            │
│                                                         │
│  ┌────────────────────────────────────────────────┐    │
│  │ Docker Daemon (Host)                           │    │
│  └────────────────────────────────────────────────┘    │
│           ↑                                             │
│           │ /var/run/docker.sock                        │
│           │                                             │
│  ┌────────┴───────────────────────────────────────┐    │
│  │ DevContainer                                   │    │
│  │                                                │    │
│  │  - Docker CLI (installed)                     │    │
│  │  - Socket mounted from host                   │    │
│  │  - Uses host Docker daemon                    │    │
│  │                                                │    │
│  │  ┌──────────────────────────────────────────┐ │    │
│  │  │ OpenShift (CRC)                          │ │    │
│  │  │ - Containers run on host Docker          │ │    │
│  │  │ - Context: openshift                     │ │    │
│  │  │ - CLI: oc, kubectl                       │ │    │
│  │  └──────────────────────────────────────────┘ │    │
│  │                                                │    │
│  │  ┌──────────────────────────────────────────┐ │    │
│  │  │ EKS Anywhere                             │ │    │
│  │  │ - Containers run on host Docker          │ │    │
│  │  │ - Kubeconfig: ~/clusters/eks-local/...   │ │    │
│  │  │ - CLI: eksctl-anywhere, kubectl          │ │    │
│  │  └──────────────────────────────────────────┘ │    │
│  └────────────────────────────────────────────────┘    │
└─────────────────────────────────────────────────────────┘

File Structure

shift-eks/
├── .devcontainer/
│   ├── devcontainer.json       # DevContainer configuration
│   ├── Dockerfile              # All tools pre-installed
│   ├── post-create.sh          # Configuration (runs once)
│   ├── start-clusters.sh       # Orchestrator
│   ├── start-crc.sh           # OpenShift startup
│   └── start-eks.sh           # EKS Anywhere startup
│
├── cluster-manager.sh          # Cluster management CLI
├── setup-crc-pull-secret.sh    # Pull secret helper
├── verify-setup.sh             # Verify installation
│
└── Documentation/
    ├── README.md               # Main documentation
    ├── QUICKSTART.md           # Getting started
    ├── ARCHITECTURE.md         # Technical details
    ├── EKS_ANYWHERE.md         # EKS-A guide
    ├── DOCKER_SETUP.md         # Docker configuration
    ├── REFACTORING.md          # Refactoring details
    ├── PULL_SECRET.md          # Pull secret guide
    ├── CRC_VS_KIND.md          # Comparison
    └── FINAL_SUMMARY.md        # This file

Tools Installed

All tools are pre-installed in the Docker image:

  • kubectl - Kubernetes CLI
  • oc - OpenShift CLI
  • crc - CodeReady Containers CLI
  • eksctl-anywhere - EKS Anywhere CLI
  • docker - Docker CLI (uses host daemon)
  • docker-compose - Docker Compose v2
  • python3 - Python 3 with pip
  • jq - JSON processor
  • curl, wget, git - Standard tools

Docker Configuration

What We Use

Docker CLI with host socket mount:

  • Docker CLI installed in Dockerfile
  • Socket mounted: /var/run/docker.sock
  • Uses host Docker daemon
  • No separate Docker daemon in container

Why This Approach

✅ Simple - Direct Docker CLI installation
✅ Reliable - No feature build issues
✅ Efficient - Single Docker daemon
✅ Explicit - Full control over installation
✅ Works - CRC and EKS-A containers run on host

What We Don't Use

❌ docker-in-docker - Separate daemon (redundant)
❌ docker-outside-of-docker feature - Had build issues

Setup Process

1. Build Time (Dockerfile)

  • Install system packages
  • Install Docker CLI
  • Install CRC, oc, eksctl-anywhere
  • Create directories
  • Configure user groups

2. Container Creation (post-create.sh)

  • Configure CRC settings (memory, CPU, disk)
  • One-time setup

3. Container Start (start-clusters.sh)

  • Start EKS Anywhere cluster (5-10 min first time)
  • Start OpenShift CRC cluster (10-15 min first time)
  • Configure kubeconfigs
  • Display cluster information

Usage

Starting Clusters

Clusters start automatically when DevContainer starts.

Manual start:

./cluster-manager.sh start-all

Checking Status

./cluster-manager.sh status

Switching Clusters

OpenShift:

./cluster-manager.sh switch-os
# or
oc config use-context openshift

EKS Anywhere:

./cluster-manager.sh switch-eks
# or
export KUBECONFIG=~/clusters/eks-local/eks-local-eks-a-cluster.kubeconfig

Managing Clusters

# Restart individual cluster
./cluster-manager.sh restart-os
./cluster-manager.sh restart-eks

# Stop all clusters
./cluster-manager.sh stop-all

# View logs
./cluster-manager.sh logs-eks

First-Time Setup

Prerequisites

  1. Red Hat Pull Secret (for OpenShift)

  2. System Resources

    • 20+ GB RAM available
    • 8+ CPU cores
    • 120+ GB disk space

Steps

  1. Rebuild DevContainer

    gitpod devcontainer rebuild

  2. Configure Pull Secret

    ./setup-crc-pull-secret.sh

  3. Wait for Clusters

    • EKS Anywhere: 5-10 minutes (first time)
    • OpenShift CRC: 10-15 minutes (first time)

  4. Verify

    ./verify-setup.sh
./cluster-manager.sh status

Common Commands

OpenShift

# Get credentials
crc console --credentials

# Access web console
crc console --url

# Check nodes
oc get nodes

# List projects
oc projects

# Create project
oc new-project my-project

EKS Anywhere

# Set kubeconfig
export KUBECONFIG=~/clusters/eks-local/eks-local-eks-a-cluster.kubeconfig

# Check nodes
kubectl get nodes

# List clusters
eksctl-anywhere get clusters

# Upgrade cluster
eksctl-anywhere upgrade cluster -f ~/clusters/eks-local-config.yaml

Docker

# View all containers
docker ps

# View CRC containers
docker ps --filter "name=crc"

# View EKS-A containers
docker ps --filter "name=eks-local"

# View logs
docker logs <container-name>

Resource Usage

Expected Usage

  • OpenShift CRC: ~16 GB RAM, 6 CPUs
  • EKS Anywhere: ~4 GB RAM, 2 CPUs
  • DevContainer: ~1 GB RAM, 1 CPU
  • Total: ~21 GB RAM, 9 CPUs

Optimization

If resources are limited, adjust in .devcontainer/post-create.sh:

# Reduce CRC resources
crc config set memory 12288  # 12 GB
crc config set cpus 4        # 4 cores

Troubleshooting

Clusters Won't Start

# Check Docker
docker ps
docker info

# Check resources
docker stats

# Restart clusters
./cluster-manager.sh stop-all
./cluster-manager.sh start-all

CRC Issues

# Check status
crc status

# View logs
crc logs

# Reconfigure pull secret
./setup-crc-pull-secret.sh

# Delete and recreate
crc delete
crc setup
crc start -p ~/.crc/pull-secret.json

EKS Anywhere Issues

# Check clusters
eksctl-anywhere get clusters

# Check containers
docker ps --filter "name=eks-local"

# View logs
cat /tmp/eksa-create.log

# Delete and recreate
eksctl-anywhere delete cluster eks-local
bash .devcontainer/start-eks.sh

Docker Issues

# Check socket
ls -la /var/run/docker.sock

# Test Docker
docker version
docker ps

# Check permissions
groups  # Should include 'docker'

Key Design Decisions

1. Docker Configuration

  • Choice: Docker CLI with host socket mount
  • Why: Simple, reliable, efficient
  • Alternative: docker-in-docker (too complex)

2. EKS Anywhere vs Alternatives

  • Choice: EKS Anywhere with Docker provider
  • Why: Official AWS distribution, free, production parity
  • Alternative: LocalStack EKS (requires Pro license)

3. Tool Installation

  • Choice: All tools in Dockerfile (build time)
  • Why: Faster startup, consistent versions
  • Alternative: Runtime installation (slower)

4. Script Organization

  • Choice: Separate scripts per cluster
  • Why: Modular, testable, maintainable
  • Alternative: Monolithic script (harder to debug)

Benefits

For Development

✅ Two production-grade clusters - OpenShift and EKS Anywhere
✅ Automatic startup - Clusters ready when container starts
✅ Pre-installed tools - Everything you need
✅ Easy management - Simple CLI for common operations
✅ Good documentation - Comprehensive guides

For Learning

✅ Real distributions - Not simulations
✅ Production parity - Same as production environments
✅ Multiple platforms - Learn both OpenShift and EKS
✅ Full features - All capabilities available

For Testing

✅ Multi-cluster - Test cross-cluster scenarios
✅ Isolated - Separate from production
✅ Reproducible - Consistent environment
✅ Fast iteration - Quick cluster recreation

Next Steps

  1. Rebuild the DevContainer to apply all changes
  2. Configure pull secret for OpenShift
  3. Wait for clusters to start (first time is slow)
  4. Explore both clusters and their features
  5. Deploy test workloads to both clusters

Support

  • Documentation: See all .md files in repository
  • Issues: Check troubleshooting sections
  • Logs: Use cluster-manager.sh logs-eks
  • Status: Use cluster-manager.sh status

Conclusion

You now have a complete multi-cluster Kubernetes development environment with:

  • ✅ OpenShift 4.x (CRC)
  • ✅ EKS Anywhere (Docker provider)
  • ✅ All tools pre-installed
  • ✅ Automatic cluster startup
  • ✅ Easy management
  • ✅ Comprehensive documentation

Ready to rebuild and start using it!