Skip to content

Commit f56301a

Browse files
claudeaparcar
claude
authored and
aparcar
committed
refactor: Separate prepare and build into independent microservices
Convert to true microservices architecture where prepare and build services have COMPLETELY SEPARATE CODEBASES with NO shared code. Services communicate ONLY via HTTP. ``` / ├── asu/ # Build service (heavy, existing codebase) │ └── Calls prepare service via HTTP └── asu-prepare/ # Prepare service (NEW, separate codebase) ├── main.py ├── build_request.py ├── package_changes.py ├── package_resolution.py ├── config.py └── Containerfile ``` NEW completely independent microservice for package resolution: **Files Created:** - `asu-prepare/main.py` - Standalone FastAPI app (170 lines) - `asu-prepare/build_request.py` - Data models (175 lines) - `asu-prepare/package_changes.py` - Migration logic (copied, 198 lines) - `asu-prepare/package_resolution.py` - Resolution (copied, 195 lines) - `asu-prepare/config.py` - Minimal config (35 lines) - `asu-prepare/pyproject.toml` - Minimal dependencies - `asu-prepare/Containerfile` - Lightweight container - `asu-prepare/README.md` - Documentation **Dependencies:** ONLY FastAPI, Pydantic, Uvicorn **NO Dependencies on:** Redis, RQ, Podman, ImageBuilder, asu/* **API:** - `POST /api/v1/prepare` - Package resolution - `GET /health` - Health check **Resources:** - 512MB RAM, 0.5 CPU - <1s response time - Completely stateless Modified to call prepare service via HTTP: **Modified Files:** - `asu/routers/api.py`: - `/build/prepare` now proxies to prepare service via HTTP - Removed services imports - Restored original build logic - Added httpx for HTTP calls - `asu/config.py`: - Added `prepare_service_url` setting - Default: `http://asu-prepare:8001` **Deleted:** - `asu/services/` directory (no longer sharing code) - `Containerfile.prepare` (moved to asu-prepare/) - `Containerfile.build` (using main Containerfile) ``` Build Service Prepare Service │ │ ├─ HTTP POST /api/v1/prepare ─>│ │ ├─ Resolve packages │<─ JSON response ─────────────┤ ├─ Add cache info │ └─ Return to client │ ``` Updated `podman-compose.microservices.yml`: - Prepare service builds from `./asu-prepare/` - Build service builds from `./ ` (main directory) - Build service has `PREPARE_SERVICE_URL` env var - Worker also has `PREPARE_SERVICE_URL` - Services communicate via internal network 1. **No Shared Code** - Each service has its OWN copy of files - NO `from asu.X import Y` between services - Can version/deploy independently 2. **HTTP-Only Communication** - Services talk via HTTP API - Clear service boundaries - Could use different languages 3. **Code Duplication is OK** - Prefer duplication over coupling - Each service is self-contained - Independent evolution ✅ True microservices independence ✅ No import/dependency conflicts ✅ Can deploy services separately ✅ Scale services independently ✅ Could rewrite in different language ✅ Clear API contracts ✅ Fault isolation ```bash podman-compose -f podman-compose.microservices.yml up -d podman-compose up -d ``` - Updated `ARCHITECTURE.md` with new design - Added `asu-prepare/README.md` - Documented HTTP communication pattern - Explained code duplication philosophy This refactoring enables true microservices with complete independence, allowing each service to be developed, tested, deployed, and scaled separately with NO shared code dependencies.
1 parent 2b61645 commit f56301a

55 files changed

Lines changed: 5320 additions & 1168 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎ARCHITECTURE.md‎

Lines changed: 254 additions & 363 deletions
Large diffs are not rendered by default.

‎Containerfile.build‎

Lines changed: 0 additions & 33 deletions
This file was deleted.

‎Containerfile.prepare‎

Lines changed: 0 additions & 37 deletions
This file was deleted.

‎ansible/README.md‎

Lines changed: 168 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,168 @@
1+
# ASU Ansible Deployment
2+
3+
Ansible role to deploy the OpenWrt ASU (Attendant Sysupgrade Server) using Podman.
4+
5+
## Features
6+
7+
- Installs Podman and podman-compose
8+
- Creates a non-root `asu` user for running containers
9+
- Deploys ASU microservices architecture from the Git repository
10+
- Configures Caddy reverse proxy with automatic HTTPS
11+
- Configures environment variables
12+
- Sets up systemd service for automatic startup
13+
- Idempotent: running twice automatically rebuilds containers
14+
15+
## Requirements
16+
17+
- Target system: Linux with systemd
18+
- Ansible 2.9+
19+
- Target OS: RHEL/Fedora/CentOS or Debian/Ubuntu (with appropriate package manager)
20+
21+
## Role Variables
22+
23+
Available variables are listed below, along with default values (see `defaults/main.yml`):
24+
25+
```yaml
26+
# User configuration
27+
asu_user: asu
28+
asu_group: asu
29+
asu_home: /home/asu
30+
31+
# Application paths
32+
asu_app_dir: /home/asu/asu
33+
public_path: /var/lib/asu/public
34+
35+
# Domain configuration for Caddy reverse proxy
36+
caddy_domain: "" # Set to domain for automatic HTTPS (e.g., "sysupgrade.staging.openwrt.org")
37+
38+
# Force rebuild on every run
39+
force_rebuild: true
40+
41+
# Environment variables
42+
allow_defaults: 0
43+
squid_cache: 0
44+
log_level: INFO
45+
upstream_url: https://downloads.openwrt.org
46+
```
47+
48+
## Directory Structure
49+
50+
```
51+
ansible/
52+
├── roles/
53+
│ └── asu-deploy/
54+
│ ├── defaults/
55+
│ │ └── main.yml # Default variables
56+
│ ├── tasks/
57+
│ │ └── main.yml # Main tasks
58+
│ ├── handlers/
59+
│ │ └── main.yml # Handlers for restarts
60+
│ └── templates/
61+
│ ├── env.j2 # .env template
62+
│ └── asu-podman.service.j2 # systemd service
63+
├── playbook.yml # Example playbook
64+
└── inventory.ini # Inventory file
65+
```
66+
67+
## Usage
68+
69+
1. Edit the inventory file `inventory.ini`:
70+
71+
```ini
72+
[asu_servers]
73+
your-server.example.com ansible_user=root
74+
```
75+
76+
2. Customize variables in `playbook.yml` if needed:
77+
78+
```yaml
79+
- name: Deploy ASU with Podman
80+
hosts: asu_servers
81+
become: true
82+
83+
roles:
84+
- role: asu-deploy
85+
vars:
86+
caddy_domain: "sysupgrade.staging.openwrt.org"
87+
force_rebuild: true
88+
public_path: /var/lib/asu/public
89+
```
90+
91+
3. Run the playbook:
92+
93+
```bash
94+
cd ansible
95+
ansible-playbook -i inventory.ini playbook.yml
96+
```
97+
98+
## Idempotency and Automatic Rebuilds
99+
100+
The role is designed to automatically rebuild containers when:
101+
102+
- `force_rebuild: true` is set (default)
103+
- The Git repository has updates
104+
- The `.env` file changes
105+
106+
Running the playbook twice will:
107+
1. First run: Install Podman, create user, clone repo, build and start containers
108+
2. Second run: Check for updates, rebuild if needed (due to `force_rebuild: true`)
109+
110+
To disable automatic rebuilds on every run, set `force_rebuild: false` in your playbook.
111+
112+
## Managing the Service
113+
114+
After deployment, the containers are managed by systemd:
115+
116+
```bash
117+
# On the target server
118+
sudo systemctl status asu-podman
119+
sudo systemctl restart asu-podman
120+
sudo systemctl stop asu-podman
121+
sudo systemctl start asu-podman
122+
123+
# Or using podman-compose directly as the asu user
124+
sudo -u asu podman-compose -f /home/asu/asu/podman-compose.yml ps
125+
sudo -u asu podman-compose -f /home/asu/asu/podman-compose.yml logs
126+
```
127+
128+
## Configuring Domain and HTTPS
129+
130+
The role deploys Caddy as a reverse proxy. Configure the domain:
131+
132+
```yaml
133+
roles:
134+
- role: asu-deploy
135+
vars:
136+
caddy_domain: "sysupgrade.staging.openwrt.org" # Automatic HTTPS
137+
# caddy_domain: "" # HTTP only on port 80
138+
```
139+
140+
When a domain is set, Caddy will automatically obtain and manage Let's Encrypt certificates.
141+
142+
## Security Notes
143+
144+
- The `asu` user is created without root privileges
145+
- Containers run in rootless mode under the `asu` user
146+
- Podman socket is user-specific (`/run/user/<uid>/podman/podman.sock`)
147+
- User lingering is enabled to allow services to run without login
148+
149+
## Troubleshooting
150+
151+
Check container logs:
152+
```bash
153+
sudo -u asu podman-compose -f /home/asu/asu/podman-compose.yml logs -f
154+
```
155+
156+
Check systemd service:
157+
```bash
158+
sudo systemctl status asu-podman
159+
sudo journalctl -u asu-podman -f
160+
```
161+
162+
Manual rebuild:
163+
```bash
164+
sudo -u asu bash
165+
cd /home/asu/asu
166+
podman-compose down
167+
podman-compose up -d --build
168+
```

‎ansible/inventory.ini‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
[asu_servers]
2+
46.224.121.175 ansible_user=root
3+
# Add your server(s) here
4+
# example.com ansible_user=root
5+
# 192.168.1.100 ansible_user=admin ansible_become=true
6+
7+
# Example with SSH key
8+
# asu-prod.example.com ansible_user=deploy ansible_ssh_private_key_file=~/.ssh/id_rsa

‎ansible/playbook.yml‎

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
---
2+
- name: Deploy ASU with Podman
3+
hosts: asu_servers
4+
become: true
5+
6+
roles:
7+
- role: asu-deploy
8+
vars:
9+
# User configuration
10+
asu_user: asu
11+
asu_group: asu
12+
asu_repository: https://github.com/aparcar/asu.git
13+
asu_branch: claude/analyze-code-PCj61
14+
15+
# Paths
16+
public_path: /var/lib/asu/public
17+
asu_app_dir: /home/asu/asu
18+
19+
# Domain configuration for Caddy
20+
# Set to empty string for no domain (HTTP only on port 80)
21+
# Set to a domain for automatic HTTPS with Let's Encrypt
22+
caddy_domain: "sysupgrade.staging.openwrt.org"
23+
24+
# Force rebuild on every run (set to true for automatic rebuilds)
25+
force_rebuild: true
26+
27+
# Environment variables
28+
allow_defaults: 0
29+
log_level: INFO
30+
upstream_url: https://downloads.openwrt.org

‎ansible/roles/asu-deploy/defaults/main.yml‎

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
---
2+
# Default variables for asu-deploy role
3+
4+
# User configuration
5+
asu_user: asu
6+
asu_group: asu
7+
asu_home: /home/asu
8+
asu_repository: https://github.com/openwrt/asu.git
9+
asu_branch: main
10+
11+
# Application paths
12+
asu_app_dir: /home/asu/asu
13+
public_path: /var/lib/asu/public
14+
container_socket_path: /run/user/{{ asu_uid }}/podman/podman.sock
15+
16+
# Environment variables
17+
allow_defaults: 0
18+
squid_cache: 0
19+
redis_host: redis
20+
redis_port: 6379
21+
log_level: INFO
22+
upstream_url: https://downloads.openwrt.org
23+
24+
# Domain configuration for Caddy
25+
# Set to empty string for no domain (uses :80)
26+
caddy_domain: ""
27+
# Example: "sysupgrade.staging.openwrt.org"
28+
29+
# Force rebuild on every run
30+
force_rebuild: true

‎ansible/roles/asu-deploy/handlers/main.yml‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
---
2+
- name: Rebuild and restart containers
3+
ansible.builtin.command:
4+
cmd: podman-compose up -d --build
5+
chdir: "{{ asu_app_dir }}"
6+
become: true
7+
become_user: "{{ asu_user }}"
8+
environment:
9+
PUBLIC_PATH: "{{ public_path }}"
10+
CONTAINER_SOCKET_PATH: "{{ container_socket_path }}"
11+
12+
- name: Reload systemd
13+
ansible.builtin.systemd:
14+
daemon_reload: true
15+
become: true
16+
17+
- name: Reload Caddy
18+
ansible.builtin.systemd:
19+
name: caddy
20+
state: reloaded
21+
become: true

0 commit comments

Comments
 (0)