A distributed system for tracking VM running time across multiple Proxmox nodes using real-time state monitoring.
┌─────────────────────────────────────────────────────────────────┐
│ Proxmox Machines (Clients) │
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │
│ │ pve-01 │ │ pve-02 │ │ pve-03 │ │
│ │ (Client) │ │ (Client) │ │ (Client) │ │
│ └──────┬──────┘ └──────┬──────┘ └──────┬──────┘ │
└─────────┼────────────────┼────────────────┼─────────────────────┘
│ │ │
│ Proxmox API │ HTTP/API │
└────────────────┼────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────┐
│ Manager Server (Ubuntu 24.04) │
│ ┌─────────────────────────────────────────────────────────┐ │
│ │ Manager API │ │
│ │ (FastAPI + Web Dashboard) │ │
│ └────────────────────────┬────────────────────────────────┘ │
│ │ │
│ ┌────────────────────────▼────────────────────────────────┐ │
│ │ MySQL Database │ │
│ │ (Sessions, Rentals, Nodes, VMs) │ │
│ └─────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────┘
- Client polls Proxmox API every 30 seconds for VM status
- Detects state changes (VM started/stopped)
- Reports to Manager immediately via HTTP API
- Manager tracks sessions with start/end times and duration
- Force Sync button reconciles all VM states on demand
On each Proxmox node, create an API token for the client:
- Go to Datacenter → Permissions → API Tokens
- Click Add
- User:
root@pam - Token ID:
tracker - Privilege Separation: ❌ Uncheck (or add VM.Audit permission)
- Click Add and copy the token value
pveum user token add root@pam tracker --privsep=0Output:
┌──────────────┬──────────────────────────────────────┐
│ key │ value │
├──────────────┼──────────────────────────────────────┤
│ full-tokenid │ root@pam!tracker │
│ info │ {"privsep":"0"} │
│ value │ xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx │
└──────────────┴──────────────────────────────────────┘
Save the token value - you'll need it for client_config.yaml.
# Clone repository
git clone <repo-url> /opt/proxmox-tracker
cd /opt/proxmox-tracker
# Create virtual environment
python3 -m venv venv
source venv/bin/activate
# Install dependencies
pip install -r requirements-manager.txt
# Set up MySQL
mysql -u root -p << 'EOF'
CREATE DATABASE vm_tracking CHARACTER SET utf8mb4;
CREATE USER 'vm_tracker'@'localhost' IDENTIFIED BY 'your-secure-password';
GRANT ALL PRIVILEGES ON vm_tracking.* TO 'vm_tracker'@'localhost';
FLUSH PRIVILEGES;
EOF
# Configure
nano config.yaml # Edit MySQL credentials and API key
# Test run (HTTP)
python -m uvicorn manager.main:app --host 0.0.0.0 --port 8000
# Install as service
sudo cp deploy/proxmox-tracker-manager.service /etc/systemd/system/
sudo systemctl daemon-reload
sudo systemctl enable proxmox-tracker-manager
sudo systemctl start proxmox-tracker-manager# Create SSL directory
mkdir -p /opt/proxmox-tracker/ssl
cd /opt/proxmox-tracker/ssl
# Generate self-signed certificate (valid for 365 days)
openssl req -x509 -newkey rsa:4096 -keyout key.pem -out cert.pem -days 365 -nodes \
-subj "/C=US/ST=State/L=City/O=Organization/CN=pve-tracker"
# Set permissions
chmod 600 key.pem
chmod 644 cert.pem
# Update service file to use HTTPS (already configured in deploy/proxmox-tracker-manager.service)
sudo systemctl daemon-reload
sudo systemctl restart proxmox-tracker-manager# Copy client files to Proxmox
scp -r client/ requirements-client.txt client_config.yaml root@pve-node:/opt/proxmox-tracker/
# SSH to Proxmox node
ssh root@pve-node
# Install dependencies
cd /opt/proxmox-tracker
pip3 install -r requirements-client.txt
# Configure - ADD YOUR API TOKEN HERE
nano client_config.yamlclient_config.yaml:
node:
name: "StormWorking" # Your node name
proxmox:
host: "127.0.0.1"
port: 8006
user: "root@pam"
token_name: "tracker"
token_value: "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" # Your token
verify_ssl: false
manager:
url: "https://YOUR-MANAGER-IP:8000" # Use https:// for secure connection
api_key: "shared-secret-key"
verify_ssl: false # Set to false for self-signed certificates
polling:
interval_seconds: 30
track_qemu: true
track_lxc: true# Test Proxmox API connection
python3 -m client.main --test
# Test single sync
python3 -m client.main --once
# Install as service
cp deploy/proxmox-tracker-client.service /etc/systemd/system/
systemctl daemon-reload
systemctl enable proxmox-tracker-client
systemctl start proxmox-tracker-clientOpen: http://your-manager-ip:8000
Features:
- 🌙/☀️ Dark/Light mode toggle
- 📊 VM status overview
- ⏱️ Running time tracking
- 🔄 Force Sync button
- 📅 Rental management
database:
type: mysql
host: localhost
port: 3306
user: vm_tracker
password: "your-password"
database: vm_tracking
security:
api_key: "shared-secret-key" # Must match clients
server:
host: "0.0.0.0"
port: 8000node:
name: "pve-01" # Unique per node
proxmox:
host: "127.0.0.1"
port: 8006
user: "root@pam"
token_name: "tracker"
token_value: "your-api-token"
verify_ssl: false
manager:
url: "http://manager-ip:8000"
api_key: "shared-secret-key"
polling:
interval_seconds: 30
track_qemu: true # Track QEMU VMs
track_lxc: true # Track LXC containers| Method | Endpoint | Description |
|---|---|---|
| POST | /api/ingest/register |
Register a new node |
| POST | /api/ingest/vm-start |
Report VM started |
| POST | /api/ingest/vm-stop |
Report VM stopped |
| POST | /api/ingest/vm-states |
Send full state snapshot |
| POST | /api/ingest/heartbeat |
Node heartbeat |
| POST | /api/ingest/force-sync |
Request sync from all nodes |
| Method | Endpoint | Description |
|---|---|---|
| GET | /api/nodes |
List registered nodes |
| GET | /api/vms |
List all VMs |
| GET | /api/sessions |
List sessions |
| GET/POST/PUT/DELETE | /api/rentals |
Manage rentals |
| GET | /api/rentals/{id}/report |
Usage report |
proxmox-tracker/
├── manager/ # Manager server (Ubuntu)
│ ├── main.py # FastAPI app + dashboard
│ ├── config.py # MySQL configuration
│ ├── models/ # SQLAlchemy models
│ │ ├── database.py # VMSession, TrackedVM, Rental, ProxmoxNode
│ │ └── schemas.py # Pydantic models
│ ├── services/ # Business logic
│ │ └── ingest_service.py
│ └── routes/ # API endpoints
│ ├── ingest.py # Client data ingestion
│ └── vms.py, sessions.py, rentals.py, nodes.py
│
├── client/ # Client (Proxmox machines)
│ ├── main.py # State polling daemon
│ ├── config.py # Client configuration
│ ├── proxmox_api.py # Proxmox API client
│ └── api_client.py # Manager HTTP client
│
├── deploy/ # Deployment files
│ ├── proxmox-tracker-manager.service
│ └── proxmox-tracker-client.service
│
├── config.yaml # Manager config
├── client_config.yaml # Client config
├── requirements-manager.txt # Manager dependencies
└── requirements-client.txt # Client dependencies
- 📊 Multi-Node Tracking - Track VMs across multiple Proxmox clusters
- ⏱️ Real-Time State Monitoring - Uses Proxmox API, not logs
- 🔄 Force Sync - Reconcile all VM states on demand
- 📅 Rental Management - Set billing start months, generate usage reports
- 🔒 API Key Auth - Secure client-manager communication
- 🌐 Web Dashboard - Beautiful dark/light mode dashboard
- 📈 MySQL Storage - Production-ready database
python3 -m client.main --testExpected output:
✓ Proxmox API connection successful
Node: StormWorking
VMs found: 5
- 104: vGPU-1 (running)
- 105: vGPU-2 (stopped)
# Check client logs
journalctl -u proxmox-tracker-client -f
# Test manager is reachable
curl http://manager-ip:8000/health# Add VM.Audit permission to token
pveum aclmod / -user root@pam -token tracker -role PVEVMUser- Check
track_qemuandtrack_lxcsettings in client_config.yaml - Run
python3 -m client.main --onceto force sync - Click "Force Sync" button in dashboard
MIT