Add HTTPS Backend Support

[aaydin-tr]

Add HTTPS Backend Support

🚨 Priority: CRITICAL

Labels: enhancement, critical, backend, tls
Estimated Effort: 2 days
Assignee: @aaydin-tr

Problem Description

Currently, Divisor only supports HTTP backends, making it unusable in production environments where backends use HTTPS. This is a critical limitation as most modern services require HTTPS communication.

Current Issues:

1. Hardcoded HTTP scheme in internal/proxy/proxy.go:77:

   req.URI().SetSchemeBytes(httpB) // Always sets "http"

2. Protocol stripping in pkg/config/config.go:156:

   b.Url = protocolRegex.ReplaceAllString(b.Url, "")

3. No TLS configuration for backend connections

Proposed Solution

Add basic HTTPS support with optional TLS verification for backend connections.

Implementation Plan

1. Update Backend Configuration Structure

File: pkg/config/config.go

type Backend struct {
    Url                       string        `yaml:"url"`
    HealthCheckPath           string        `yaml:"health_check_path"`
    UseHTTPS                  bool          `yaml:"use_https"`          // NEW
    SkipTLSVerify            bool          `yaml:"skip_tls_verify"`    // NEW
    Weight                    uint          `yaml:"weight,omitempty"`
    MaxConnection             int           `yaml:"max_conn"`
    MaxConnWaitTimeout        time.Duration `yaml:"max_conn_timeout"`
    MaxConnDuration           time.Duration `yaml:"max_conn_duration"`
    MaxIdleConnDuration       time.Duration `yaml:"max_idle_conn_duration"`
    MaxIdemponentCallAttempts int           `yaml:"max_idemponent_call_attempts"`
}

2. Update Health Check URL Generation

func (b *Backend) GetHealthCheckURL() string {
    scheme := "http"
    if b.UseHTTPS {
        scheme = "https"
    }
    return scheme + "://" + b.Url + b.HealthCheckPath
}

3. Update Proxy Client Implementation

File: internal/proxy/proxy.go

Add HTTPS support to ProxyClient:

type ProxyClient struct {
    proxy             *fasthttp.HostClient
    totalRequestCount *uint64
    totalResTime      *uint64
    customHeaders     map[string]string
    Addr              string
    addrB             []byte
    useHTTPS          bool    // NEW
}

func (h *ProxyClient) preReq(req *fasthttp.Request, clientIP []byte) {
    for _, h := range hopHeaders {
        req.Header.DelBytes(h)
    }

    // Set scheme based on backend configuration
    if h.useHTTPS {
        req.URI().SetSchemeBytes([]byte("https"))
    } else {
        req.URI().SetSchemeBytes(httpB)
    }
    
    req.SetHostBytes(h.addrB)
    req.Header.SetBytesKV(XForwardedFor, clientIP)
    h.setCustomHeaders(req, clientIP)
}

func NewProxyClient(backend config.Backend, customHeaders map[string]string) IProxyClient {
    var tlsConfig *tls.Config
    if backend.UseHTTPS {
        tlsConfig = &tls.Config{
            InsecureSkipVerify: backend.SkipTLSVerify,
        }
    }

    proxyClient := &fasthttp.HostClient{
        Addr:                      backend.Url,
        IsTLS:                     backend.UseHTTPS,
        TLSConfig:                 tlsConfig,
        MaxConns:                  backend.MaxConnection,
        MaxConnDuration:           backend.MaxConnDuration,
        MaxIdleConnDuration:       backend.MaxIdleConnDuration,
        MaxIdemponentCallAttempts: backend.MaxIdemponentCallAttempts,
        MaxConnWaitTimeout:        backend.MaxConnWaitTimeout,
    }

    return &ProxyClient{
        proxy:             proxyClient,
        Addr:              backend.Url,
        addrB:             helper.S2b(backend.Url),
        totalRequestCount: new(uint64),
        totalResTime:      new(uint64),
        customHeaders:     customHeaders,
        useHTTPS:          backend.UseHTTPS,
    }
}

Configuration Example

After implementation, users should be able to configure HTTPS backends:

type: "round-robin"
host: "localhost"
port: "8080"

backends:
  # HTTPS backend with TLS verification
  - url: "api.example.com:443"
    health_check_path: "/health"
    use_https: true
    skip_tls_verify: false
    max_conn: 100

  # HTTPS backend without TLS verification (dev/test only)
  - url: "internal-api.local:8443"
    health_check_path: "/ping"
    use_https: true
    skip_tls_verify: true
    max_conn: 50

  # HTTP backend (backward compatibility)
  - url: "legacy-api.internal:8080"
    health_check_path: "/status"
    use_https: false
    max_conn: 100

Acceptance Criteria

• Configuration supports use_https and skip_tls_verify fields
• HTTPS backends can be successfully proxied to
• Health checks work with HTTPS backends
• TLS verification can be disabled for development/testing
• Backward compatibility maintained (existing HTTP configs work)
• Error handling for TLS connection failures
• Documentation updated with HTTPS examples

Files to Modify

1. pkg/config/config.go - Add HTTPS fields to Backend struct
2. internal/proxy/proxy.go - Update scheme handling and TLS config
3. Examples/ - Add HTTPS configuration examples

---

Related Issues: None
Blocks: All production deployments requiring HTTPS backends
Documentation: Update README with HTTPS examples after implementation