Skip to content

LOCAL_DEV_PLAN.md

Latest commit

 

History

History
214 lines (163 loc) · 6.89 KB

LOCAL_DEV_PLAN.md

File metadata and controls

214 lines (163 loc) · 6.89 KB

Local Development and Testing Setup Plan

Objectives

  1. Shut down GCP Cloud Run service to stop burning money during debugging
  2. Create local development environment with console-based Discord interaction
  3. Download production database for realistic testing
  4. Enhanced logging to monitor long-term operation
  5. Debug monitoring loop issues in controlled environment

Implementation Steps

Phase 1: Infrastructure Setup

1.1 Shut Down Cloud Run Service

# Scale down to 0 instances to stop costs
gcloud run services update dispinmap-bot --region=us-central1 --min-instances=0 --max-instances=0

1.2 Download Production Database

  • Create script scripts/download_production_db.py
  • Download latest backup from dispinmap-bot-sqlite-backups GCS bucket
  • Use Litestream to restore to local file local_db/pinball_bot.db
  • Verify database integrity and content

1.3 Local Environment Configuration

  • Create .env.local file with:
    • DISCORD_TOKEN (from user's Discord bot)
    • DATABASE_PATH=local_db/pinball_bot.db
    • LOCAL_DEV_MODE=true
    • LOG_LEVEL=DEBUG

Phase 2: Console Discord Interface

2.1 Create Console Discord Simulator

  • New file: src/console_discord.py
  • Implements a fake Discord channel that:
    • Accepts commands via stdin (like !add location "Ground Kontrol")
    • Sends responses to stdout
    • Simulates a single user and single channel
    • Integrates with existing command handlers

2.2 Local Development Entry Point

  • New file: src/local_dev.py
  • Entry point that:
    • Loads local environment variables
    • Starts both real Discord connection AND console interface
    • Runs monitoring loop normally
    • Provides graceful shutdown

2.3 Console Interface Features

  • Command input: > !check (user types commands)
  • Bot responses: Immediate output to console
  • Background monitoring: Shows monitoring loop activity
  • Status display: Current targets, last check times, etc.
  • Manual triggers: Force monitoring checks with special commands

Phase 3: Enhanced Logging

3.1 Single Log File with Rotation

  • Use Python's RotatingFileHandler
  • Single file: logs/bot.log (max 10MB, keep 5 files)
  • All console output also goes to log file
  • Timestamps and log levels for all entries

3.2 Monitoring Loop Debugging

  • Enhanced logging in runner.py to track:
    • Every monitoring loop iteration
    • Channel processing details
    • API call results and timing
    • Database operations
    • Error conditions with full context

Phase 4: Testing and Debugging

4.1 Long-term Monitoring Test

  • Run locally for 24+ hours
  • Monitor log file for:
    • Monitoring loop stability
    • Memory usage patterns
    • API rate limiting issues
    • Database performance
    • Error patterns

4.2 Interactive Testing

  • Test all commands through console interface
  • Verify monitoring targets work correctly
  • Test error conditions and recovery
  • Validate notification logic

File Structure

DisPinMap-Main/
├── .env.local                    # Local environment config
├── LOCAL_DEV_PLAN.md            # This file
├── logs/
│   └── bot.log                  # Single rotating log file
├── local_db/
│   └── pinball_bot.db          # Downloaded production database
├── scripts/
│   ├── download_production_db.py # Database download script
│   └── local_setup.sh          # Setup script
└── src/
    ├── console_discord.py       # Console Discord simulator
    ├── local_dev.py            # Local development entry point
    └── log_config.py           # Enhanced logging configuration

Console Interface Design

Input Format

> !add location "Ground Kontrol"
> !list
> !check
> !monitor_health
> .quit                         # Special command to exit
> .status                       # Show current monitoring status
> .trigger                      # Force immediate monitoring check

Output Format

[2025-07-04 10:15:32] [CONSOLE] > !add location "Ground Kontrol"
[2025-07-04 10:15:33] [BOT] ✅ Successfully added location monitoring for "Ground Kontrol"
[2025-07-04 10:15:33] [BOT] 📋 **Last 5 submissions across all monitored targets:**
[2025-07-04 10:15:33] [MONITOR] 🔄 Monitor loop iteration #145 starting
[2025-07-04 10:15:34] [MONITOR] ✅ Channel 999999: Ready to poll (61.2 min >= 60 min)

Implementation Order

  1. Shut down Cloud Run (immediate cost savings)
  2. Create database download script and get production data
  3. Set up basic local environment with .env.local
  4. Create console Discord interface for basic interaction
  5. Enhance logging system for better monitoring
  6. Create local_dev.py entry point that ties everything together
  7. Test and debug monitoring loop issues
  8. Document findings and prepare Cloud Run fixes

Success Criteria

  • Cloud Run service scaled to 0 (costs stopped)
  • Local environment runs with production database
  • Console interface accepts and processes commands
  • Monitoring loop runs continuously without crashes
  • All activity logged to rotating log file
  • Can run for 24+ hours without issues
  • Monitoring loop actually sends notifications when appropriate
  • Ready to fix Cloud Run configuration based on local findings

Implementation Status

✅ COMPLETED - All core functionality is working!

What's Working:

  • Cloud Run scaled to 0 instances (costs stopped)
  • Production database successfully downloaded and restored using Litestream
  • Local environment with .env.local configuration
  • Enhanced logging with rotation (logs/bot.log, 10MB max, 5 backups)
  • Console Discord interface with stdin/stdout interaction
  • Monitoring loop starts and makes successful API calls
  • Bot connects to Discord and processes real channels

Test Results:

  • Bot starts up in ~3 seconds
  • Monitoring loop begins immediately and polls PinballMap API
  • Database queries work correctly (10 monitoring targets, 5 active channels)
  • Graceful shutdown handling works
  • All logs captured to both console and rotating file

Next Steps:

  1. Run extended testing (24+ hours) to identify stability issues
  2. Test console commands interactively
  3. Investigate Cloud Run health check configuration
  4. Document findings for Cloud Run fixes

Cloud Run Issue Investigation (Parallel)

While testing locally, also investigate:

  • Missing health check configuration in terraform
  • Startup probe and liveness probe settings
  • Container resource limits and timeout values
  • Litestream backup path issue (db vs db-v2)
  • Process startup timing in startup.sh script

Next Steps After Local Testing

  1. Fix identified issues in Cloud Run configuration
  2. Add proper health checks to terraform
  3. Test fixes locally first
  4. Deploy improved version to Cloud Run
  5. Monitor deployment for stability
  6. Scale back up once confirmed working