Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
105 changes: 105 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1 +1,106 @@
/openshift-router

# Created by https://www.gitignore.io/api/go,vim,emacs,visualstudiocode

### Emacs ###
# -*- mode: gitignore; -*-
*~
\#*\#
/.emacs.desktop
/.emacs.desktop.lock
*.elc
auto-save-list
tramp
.\#*

# Org-mode
.org-id-locations
*_archive

# flymake-mode
*_flymake.*

# eshell files
/eshell/history
/eshell/lastdir

# elpa packages
/elpa/

# reftex files
*.rel

# AUCTeX auto folder
/auto/

# cask packages
.cask/
dist/

# Flycheck
flycheck_*.el

# server auth directory
/server/

# projectiles files
.projectile
projectile-bookmarks.eld

# directory configuration
.dir-locals.el

# saveplace
places

# url cache
url/cache/

# cedet
ede-projects.el

# smex
smex-items

# company-statistics
company-statistics-cache.el

# anaconda-mode
anaconda-mode/

### Go ###
# Binaries for programs and plugins
*.exe
*.exe~
*.dll
*.so
*.dylib

# Test binary, build with 'go test -c'
*.test

# Output of the go coverage tool, specifically when used with LiteIDE
*.out

### Vim ###
# swap
.sw[a-p]
.*.sw[a-p]
# session
Session.vim
# temporary
.netrwhist
# auto-generated tag files
tags

### VisualStudioCode ###
.vscode/*
!.vscode/settings.json
!.vscode/tasks.json
!.vscode/extensions.json
.history

### GoLand ###
.idea/*

# End of https://www.gitignore.io/api/go,vim,emacs,visualstudiocode
79 changes: 79 additions & 0 deletions HACKING.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,3 +51,82 @@ Run unit tests:
```
$ make check
```

## Local Run and Debugging

The steps below configures the local development environment to run OpenShift router.

> Note that a local router cannot properly proxy requests, since it cannot reach the overlay network in the
Kubernetes cluster. This mode is mostly useful for debugging controller behavior and configuration builder,
improving the "code + run + test + debug" workflow on these scenarios. Once the router seems to be building
proper configurations, a fully working OpenShift environment should be used to complete the exploratory e2e
tests.

### Install HAProxy

HAProxy needs to be installed, either from the package manager or compiled from the sources. The steps below
describe how to compile HAProxy from sources.

```bash
sudo yum install -y pcre2-devel
mkdir -p ~/ws/haproxy
cd ~/ws/haproxy
# Choose a version from https://www.haproxy.org, column "Latest version" points to the source tarball.
curl -LO https://www.haproxy.org/download/2.8/src/haproxy-2.8.18.tar.gz
tar xzf haproxy-2.8.18.tar.gz
cd haproxy-2.8.18/
make TARGET=linux-glibc USE_OPENSSL=1 USE_PROMEX=1 USE_PCRE2=1 USE_PCRE2_JIT=1
./haproxy -vv
sudo cp haproxy /usr/sbin/
```

### Run OpenShift router locally

Prepare the local environment and start router with default options:

```bash
make local-run
```

Prepare means create and clean `/var/lib/haproxy` and sub-directories, which router uses to configure HAProxy.

### Debug locally

Here is the VSCode launch configuration that starts OpenShift router in debug mode.

> Run `make local-prepare` before the very first run, and whenever a clean environment is needed.

`.vscode/launch.json` content, merge its content with any previous one:

```json
{
"version": "0.2.0",
"configurations": [
{
"name": "Launch OpenShift router",
"type": "go",
"request": "launch",
"mode": "debug",
"cwd": ".",
"program": "cmd/openshift-router",
"env": {
"KUBECONFIG": "/PATH/TO/KUBECONFIG.yaml",
"ROUTER_SERVICE_HTTP_PORT": "9090",
"ROUTER_SERVICE_HTTPS_PORT": "9443",
"STATS_USERNAME": "admin",
"STATS_PASSWORD": "admin",
"STATS_PORT": "1936",
},
"args": [
"--template=images/router/haproxy/conf/haproxy-config.template",
"--reload=images/router/haproxy/reload-haproxy",
],
}
]
}
```

Caveats:

1. On debug mode, the binary name is not `openshift-router`, making the bootstrap code to fail. Temporarily patch `cmd/openshift-router/main.go` and hardcode `openshift-router` in the `CommandFor()` call.
1. A haproxy instance should be left behind depending on how the router is stopped. Run `killall haproxy` in case the router complains when trying to listen to sockets.
6 changes: 6 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,12 @@ images/router/*/Dockerfile.ocp: images/router/base/Dockerfile.ocp
check:
CGO_ENABLED=1 $(GO) test -race ./...

local-prepare:
hack/local-router.sh prepare

local-run:
hack/local-router.sh run

.PHONY: verify
verify:
hack/verify-gofmt.sh
Expand Down
56 changes: 56 additions & 0 deletions hack/local-router.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
#!/bin/bash

set -eo pipefail

die() { echo "$*" >&2; exit 1; }

help() {
echo "Usage: $(basename "$0") <command>"
echo
echo " Commands:"
echo
echo " prepare: Prepares the environment without starting the router."
echo " Note that it kills any running HAProxy and cleans its"
echo " configuration directory."
echo " run: Prepares the environment and starts the router locally."
echo " help: Shows this help message."
exit 0
}

router_prepare() {
# dropping any leaked haproxy instance that router did not have a chance to kill
killall haproxy 2>/dev/null || :

# copying mandatory files to HAProxy's configuration dir
readonly haproxydir="/var/lib/haproxy"
if ! [ -d "$haproxydir" ] || ! [ -w "$haproxydir" ]; then
echo "We're going to ask your sudo password in order to create and fix permission on $haproxydir"
sudo mkdir -p "$haproxydir"
sudo chown "${USER}:${USER}" "$haproxydir"
fi
rm -rf "${haproxydir}/{conf,run}"
mkdir -p "${haproxydir}/{conf,run}"
cp images/router/haproxy/conf/* "${haproxydir}/conf/"

echo "$haproxydir prepared"
}

router_run() {
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be nice to have a check to make sure that the KUBECONFIG is set properly before running?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see, I'm not sure, I though about this and ended up choosing the simplicity, since router itself will check this for us - although its message isn't that clear:

make local-run 
hack/local-router.sh run
/var/lib/haproxy prepared
github.com/openshift/router/pkg/router/template
github.com/openshift/router/pkg/router/metrics
github.com/openshift/router/pkg/cmd/infra/router
I0210 11:56:03.528934  399293 template.go:561] "msg"="starting router" "logger"="router" "version"="majorFromGit: \nminorFromGit: \ncommitFromGit: \nversionFromGit: unknown\ngitTreeState: \nbuildDate: \n"
error: Unable to configure Kubernetes client: invalid configuration: no configuration has been provided, try setting KUBERNETES_MASTER environment variable
exit status 1
make: *** [Makefile:46: local-run] Error 1

What do you think from the user's perspective? I'm fine for both, either checking or letting router check for me.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah okay then if router checks then fine

# changing listening ports, we usually don't have permission to bind to the default ones 80/443
ROUTER_SERVICE_HTTP_PORT=9090 ROUTER_SERVICE_HTTPS_PORT=9443 STATS_USERNAME=admin STATS_PASSWORD=admin STATS_PORT=1936 \
go run -v ./cmd/openshift-router \
--template images/router/haproxy/conf/haproxy-config.template \
--reload images/router/haproxy/reload-haproxy
}

[ $# -ne 1 ] && help

# goes to repository root, we use relative paths
cd "$(dirname $0)/.."

case "$1" in
prepare) router_prepare;;
run) router_prepare; router_run;;
help) help;;
*) die "invalid command: $1";;
esac