Skip to content

Commit b9648d9

Browse files
piewaredpiewared
authored
Merge pull request #2 from piewared/feature/ADDING_INGRESS
Feature/adding ingress
2 parents ac04bec + e26d5f4 commit b9648d9

20 files changed

Lines changed: 3232 additions & 319 deletions

File tree

‎docs/fastapi-flyio-kubernetes.md‎

Lines changed: 309 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,309 @@
1+
# FastAPI Fly.io Kubernetes (FKS) Deployment
2+
3+
This document covers deploying API Forge to Fly.io's Kubernetes Service (FKS), including compatibility analysis with our existing Helm-based deployment and future CLI design considerations.
4+
5+
## Overview
6+
7+
Fly.io Kubernetes Service (FKS) is Fly.io's managed Kubernetes offering that integrates with their global Anycast network. It provides a different deployment model than traditional Kubernetes clusters (GKE, EKS, AKS), with some unique advantages and constraints.
8+
9+
> **Status:** FKS support is planned but not yet implemented. This document captures research and design decisions for future development.
10+
11+
## FKS Key Characteristics
12+
13+
### How FKS Differs from Standard Kubernetes
14+
15+
| Aspect | Standard K8s (GKE, EKS, AKS) | Fly.io FKS |
16+
|--------|------------------------------|------------|
17+
| **Ingress** | Ingress resource + Ingress Controller (nginx) | Not used - Fly's Anycast proxy handles routing |
18+
| **LoadBalancer** | Cloud provider provisions external IP | Maps directly to Fly.io's edge network |
19+
| **TLS Certificates** | cert-manager + Let's Encrypt | Automatic - Fly provisions and renews certs |
20+
| **DNS** | Manual - point domain to LB IP | Automatic `*.fly.dev` + custom domain support |
21+
| **Global Distribution** | Multi-region requires complex setup | Built-in - deploy to 30+ regions easily |
22+
| **Scaling** | HPA, node autoscaling | Fly Machines with automatic scaling |
23+
24+
### What FKS Provides Automatically
25+
26+
1. **Automatic TLS** - No cert-manager, no Let's Encrypt configuration needed
27+
2. **Global Anycast** - Traffic routed to nearest region automatically
28+
3. **DDoS Protection** - Built into Fly's edge network
29+
4. **Automatic DNS** - `yourapp.fly.dev` domains provisioned automatically
30+
5. **Custom Domains** - Simple CNAME setup with automatic certificate issuance
31+
32+
### What FKS Does NOT Use
33+
34+
- ❌ Ingress resources
35+
- ❌ Ingress Controllers (nginx, traefik, etc.)
36+
- ❌ cert-manager
37+
- ❌ External DNS controllers
38+
- ❌ Cloud provider load balancer integrations
39+
40+
## Exposing Services on FKS
41+
42+
### LoadBalancer Service (Recommended)
43+
44+
On FKS, you expose services using `type: LoadBalancer` which Fly.io intercepts:
45+
46+
```yaml
47+
apiVersion: v1
48+
kind: Service
49+
metadata:
50+
name: app
51+
annotations:
52+
# Fly-specific annotations
53+
fly.io/app: my-fastapi-app
54+
spec:
55+
type: LoadBalancer
56+
ports:
57+
- port: 443
58+
targetPort: 8000
59+
selector:
60+
app.kubernetes.io/name: app
61+
```
62+
63+
Fly.io then:
64+
1. Creates a Fly App if it doesn't exist
65+
2. Provisions `my-fastapi-app.fly.dev` domain
66+
3. Issues TLS certificate automatically
67+
4. Routes global traffic through Anycast
68+
69+
### Custom Domains
70+
71+
```yaml
72+
metadata:
73+
annotations:
74+
fly.io/app: my-fastapi-app
75+
fly.io/domains: "api.example.com,api.mycompany.io"
76+
```
77+
78+
Then add a CNAME record:
79+
```
80+
api.example.com CNAME my-fastapi-app.fly.dev
81+
```
82+
83+
Fly automatically issues certificates for custom domains.
84+
85+
## Compatibility Analysis
86+
87+
### Current API Forge Features vs FKS
88+
89+
| API Forge Feature | Standard K8s | FKS Compatibility |
90+
|-------------------|--------------|-------------------|
91+
| Helm chart deployment | ✅ Works | ✅ Works (FKS is standard K8s) |
92+
| PostgreSQL StatefulSet | ✅ Works | ⚠️ Consider Fly Postgres instead |
93+
| Redis Deployment | ✅ Works | ⚠️ Consider Fly Redis (Upstash) |
94+
| Temporal | ✅ Works | ✅ Works |
95+
| `--ingress` flag | Creates Ingress | ❌ Should create LoadBalancer instead |
96+
| `--ingress-host` | Sets Ingress host | Should set Fly app name |
97+
| `--ingress-tls-secret` | References K8s Secret | ❌ Not needed |
98+
| `--ingress-tls auto` | Uses cert-manager | ❌ Not needed |
99+
| NetworkPolicies | ✅ Works | ✅ Works |
100+
| PersistentVolumeClaims | ✅ Works | ✅ Works (Fly Volumes) |
101+
102+
### Components That Need Adaptation
103+
104+
1. **Ingress → LoadBalancer Service**
105+
- Replace Ingress resource with LoadBalancer Service
106+
- Add Fly-specific annotations
107+
- Remove Ingress Controller dependency
108+
109+
2. **TLS Configuration**
110+
- Remove cert-manager setup
111+
- Remove TLS secret references
112+
- Fly handles all certificate management
113+
114+
3. **Database Considerations**
115+
- Could use in-cluster PostgreSQL (works but not recommended)
116+
- Better: Use Fly Postgres (managed, with replicas)
117+
- Fly Postgres uses their own clustering solution
118+
119+
4. **Redis Considerations**
120+
- Could use in-cluster Redis (works)
121+
- Alternative: Upstash Redis (Fly partnership, serverless)
122+
123+
## Proposed CLI Design
124+
125+
### Option A: Unified Command with Detection (Complex)
126+
127+
```bash
128+
# CLI detects cluster type and adapts
129+
uv run api-forge-cli deploy up k8s --ingress --ingress-host myapp
130+
131+
# On FKS: Creates LoadBalancer Service with Fly annotations
132+
# On standard K8s: Creates Ingress + optional cert-manager
133+
```
134+
135+
**Pros:** Single command, automatic adaptation
136+
**Cons:** Complex logic, harder to debug, surprises users
137+
138+
### Option B: Separate Target (Recommended)
139+
140+
```bash
141+
# Explicit Fly.io target
142+
uv run api-forge-cli deploy up fly --app myapp --region ord
143+
144+
# Standard Kubernetes
145+
uv run api-forge-cli deploy up k8s --ingress --ingress-host api.example.com
146+
```
147+
148+
**Pros:** Clear intent, simpler implementation, Fly-specific optimizations
149+
**Cons:** Another target to maintain
150+
151+
### Recommended: Option B
152+
153+
Separate targets are better because:
154+
155+
1. **Fly has unique features** - Machines, regions, Fly Postgres are Fly-specific
156+
2. **Simpler Helm chart** - No conditionals for "is this FKS?"
157+
3. **Better UX** - Users explicitly choose their target
158+
4. **Fly CLI integration** - Can leverage `flyctl` where appropriate
159+
5. **Different defaults** - FKS might skip in-cluster Postgres entirely
160+
161+
### Proposed CLI Commands
162+
163+
```bash
164+
# Setup Fly.io (one-time)
165+
uv run api-forge-cli deploy setup fly
166+
# - Verifies flyctl is installed
167+
# - Authenticates with Fly.io
168+
# - Creates Fly organization if needed
169+
170+
# Deploy to Fly Kubernetes
171+
uv run api-forge-cli deploy up fly \
172+
--app my-fastapi-app \
173+
--region ord \
174+
--postgres fly # Use Fly Postgres (recommended)
175+
--redis upstash # Use Upstash Redis (optional)
176+
177+
# Or with in-cluster databases (not recommended for production)
178+
uv run api-forge-cli deploy up fly \
179+
--app my-fastapi-app \
180+
--postgres in-cluster \
181+
--redis in-cluster
182+
183+
# Status
184+
uv run api-forge-cli deploy status fly --app my-fastapi-app
185+
186+
# Teardown
187+
uv run api-forge-cli deploy down fly --app my-fastapi-app
188+
```
189+
190+
### Implementation Phases
191+
192+
**Phase 1: Basic FKS Support**
193+
- Deploy app and worker to FKS
194+
- Use LoadBalancer Service for external access
195+
- In-cluster PostgreSQL and Redis (same as standard K8s)
196+
197+
**Phase 2: Fly-Native Services**
198+
- Fly Postgres integration
199+
- Upstash Redis integration
200+
- Fly Volumes for persistent storage
201+
202+
**Phase 3: Advanced Features**
203+
- Multi-region deployment
204+
- Fly Machines autoscaling
205+
- Fly.io metrics integration
206+
207+
## Helm Chart Modifications for FKS
208+
209+
### Conditional Ingress vs LoadBalancer
210+
211+
```yaml
212+
# values.yaml
213+
app:
214+
# Standard K8s ingress (existing)
215+
ingress:
216+
enabled: false
217+
# ... existing config
218+
219+
# Fly.io specific (new)
220+
fly:
221+
enabled: false
222+
app: ""
223+
regions: ["ord"]
224+
domains: []
225+
```
226+
227+
### FKS-Specific Service Template
228+
229+
```yaml
230+
# templates/services/app-fly.yaml
231+
{{- if .Values.app.fly.enabled }}
232+
apiVersion: v1
233+
kind: Service
234+
metadata:
235+
name: {{ .Values.app.name | default "app" }}
236+
namespace: {{ .Values.global.namespace }}
237+
annotations:
238+
fly.io/app: {{ .Values.app.fly.app | required "app.fly.app is required" }}
239+
{{- if .Values.app.fly.domains }}
240+
fly.io/domains: {{ .Values.app.fly.domains | join "," | quote }}
241+
{{- end }}
242+
labels:
243+
{{- include "api-forge.labels" . | nindent 4 }}
244+
spec:
245+
type: LoadBalancer
246+
ports:
247+
- port: 443
248+
targetPort: {{ .Values.app.service.port | default 8000 }}
249+
name: https
250+
selector:
251+
app.kubernetes.io/name: {{ .Values.app.name | default "app" }}
252+
{{- end }}
253+
```
254+
255+
## TLS Strategy by Platform
256+
257+
| Platform | TLS Strategy | CLI Flag |
258+
|----------|--------------|----------|
259+
| **Minikube** | None (HTTP) or self-signed | `--ingress` (no TLS) |
260+
| **Standard K8s** | cert-manager + Let's Encrypt | `--ingress --ingress-tls auto` |
261+
| **Standard K8s** | Manual certificate | `--ingress --ingress-tls-secret name` |
262+
| **AWS EKS** | ACM certificate | `--ingress` + ACM annotation |
263+
| **GKE** | Google-managed cert | `--ingress` + ManagedCertificate |
264+
| **Fly.io FKS** | Automatic (Fly-managed) | `deploy up fly` (TLS automatic) |
265+
266+
## Database Strategy by Platform
267+
268+
| Platform | Recommended PostgreSQL | Recommended Redis |
269+
|----------|----------------------|-------------------|
270+
| **Development** | Docker Compose (local) | Docker Compose (local) |
271+
| **Minikube** | In-cluster StatefulSet | In-cluster Deployment |
272+
| **Standard K8s** | In-cluster or managed (RDS, Cloud SQL) | In-cluster or managed |
273+
| **Fly.io FKS** | Fly Postgres (managed) | Upstash Redis or in-cluster |
274+
275+
## Migration Path
276+
277+
### From Docker Compose to FKS
278+
279+
1. **Test locally** with `deploy up dev`
280+
2. **Test on Minikube** with `deploy up k8s`
281+
3. **Deploy to FKS** with `deploy up fly`
282+
283+
### From Standard K8s to FKS
284+
285+
1. **Export data** from existing PostgreSQL
286+
2. **Create Fly Postgres** cluster
287+
3. **Import data** to Fly Postgres
288+
4. **Deploy app** to FKS with `--postgres fly`
289+
5. **Update DNS** to point to Fly
290+
291+
## Current Limitations
292+
293+
1. **FKS is relatively new** - Some features may change
294+
2. **Fly Postgres clustering** - Different from standard PostgreSQL HA
295+
3. **Temporal on Fly** - May need special consideration for workflows
296+
4. **Cost model** - Fly charges differently than traditional cloud
297+
298+
## Related Documentation
299+
300+
- [Kubernetes Deployment Guide](./fastapi-kubernetes-deployment.md) - Standard K8s deployment
301+
- [Ingress Configuration](./fastapi-kubernetes-ingress.md) - Ingress and TLS for standard K8s
302+
- [Docker Dev Environment](./fastapi-docker-dev-environment.md) - Local development
303+
304+
## External Resources
305+
306+
- [Fly.io Kubernetes Documentation](https://fly.io/docs/kubernetes/)
307+
- [Fly.io FKS Quickstart](https://fly.io/docs/kubernetes/fks-quickstart/)
308+
- [Fly Postgres](https://fly.io/docs/postgres/)
309+
- [Upstash Redis on Fly](https://fly.io/docs/reference/redis/)

‎docs/fastapi-kubernetes-deployment.md‎

Lines changed: 10 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -863,35 +863,18 @@ spec:
863863

864864
### Ingress
865865

866-
Expose your FastAPI application via Ingress:
866+
API Forge includes built-in Ingress support via CLI flags. Enable external access with:
867867

868-
```yaml
869-
apiVersion: networking.k8s.io/v1
870-
kind: Ingress
871-
metadata:
872-
name: app-ingress
873-
annotations:
874-
cert-manager.io/cluster-issuer: letsencrypt-prod
875-
nginx.ingress.kubernetes.io/ssl-redirect: "true"
876-
spec:
877-
ingressClassName: nginx
878-
tls:
879-
- hosts:
880-
- api.example.com
881-
secretName: app-tls
882-
rules:
883-
- host: api.example.com
884-
http:
885-
paths:
886-
- path: /
887-
pathType: Prefix
888-
backend:
889-
service:
890-
name: app
891-
port:
892-
number: 8000
868+
```bash
869+
# Basic ingress (HTTP)
870+
uv run api-forge-cli deploy up k8s --ingress
871+
872+
# Custom hostname with TLS
873+
uv run api-forge-cli deploy up k8s --ingress --ingress-host api.example.com --ingress-tls-secret api-tls
893874
```
894875

876+
For comprehensive Ingress documentation including TLS setup, cloud provider configurations, and troubleshooting, see the **[Ingress Configuration Guide](./fastapi-kubernetes-ingress.md)**.
877+
895878
### NetworkPolicies
896879

897880
Restrict pod-to-pod communication:
@@ -1350,6 +1333,7 @@ kubectl apply -f argocd-application.yaml
13501333

13511334
## Related Documentation
13521335

1336+
- [Ingress Configuration](./fastapi-kubernetes-ingress.md) - External access, TLS, and routing
13531337
- [Docker Dev Environment](./fastapi-docker-dev-environment.md) - Local testing before deployment
13541338
- [Docker Compose Production](./fastapi-production-deployment-docker-compose.md) - Alternative deployment
13551339
- [Testing Strategy](./fastapi-testing-strategy.md) - Test before deploying

0 commit comments

Comments
 (0)