Skip to content

ARCHITECTURE.md

Latest commit

 

History

History
324 lines (296 loc) · 12.6 KB

ARCHITECTURE.md

File metadata and controls

324 lines (296 loc) · 12.6 KB

🏗️ OMNI System Architecture Documentation

📊 Complete System Overview

The OMNI Super Intelligent Trading System is built on a multi-layered architecture that combines quantum computing, AI agents, and real-time data processing to create an autonomous trading platform.

🌐 Infrastructure Layer

☁️ AWS Lightsail Deployment

  • Public IP: 3.111.22.56
  • Instance: 2GB RAM, 1 vCPU, 60GB SSD
  • OS: Ubuntu 20.04 LTS
  • Uptime: 99.9% with auto-restart capabilities

🔧 NGINX Reverse Proxy Configuration

# Location: /etc/nginx/sites-available/omni
server {
    listen 80;
    server_name 3.111.22.56;
    
    # Frontend routing
    location / {
        proxy_pass http://localhost:10001;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
    }
    
    # API routing
    location /api/ {
        proxy_pass http://localhost:10002;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
    }
    
    # WebSocket routing
    location /socket.io/ {
        proxy_pass http://localhost:10003;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
    }
    
    # gRPC routing
    location /grpc/ {
        grpc_pass grpc://localhost:10004;
    }
    
    # Bybit API proxy
    location /bybit/ {
        proxy_pass https://api-demo.bybit.com/;
        proxy_ssl_verify off;
    }
}

⚙️ PM2 Process Management

// ecosystem.config.js
module.exports = {
  apps: [
    {
      name: 'omni-dashboard-frontend',
      script: 'serve',
      args: '-s build -l 10001',
      cwd: './ui/dashboard',
      instances: 1,
      autorestart: true,
      watch: false,
      max_memory_restart: '1G'
    },
    {
      name: 'omni-api',
      script: './src/server.js',
      cwd: './ui/dashboard-backend',
      instances: 1,
      autorestart: true,
      watch: false,
      max_memory_restart: '1G',
      env: {
        PORT: 10002,
        NODE_ENV: 'production'
      }
    },
    {
      name: 'omni-websocket',
      script: './src/websocket-server.js',
      cwd: './ui/dashboard-backend',
      instances: 1,
      autorestart: true,
      watch: false,
      max_memory_restart: '512M',
      env: {
        PORT: 10003,
        NODE_ENV: 'production'
      }
    },
    {
      name: 'omni-grpc',
      script: './src/grpc-server.js',
      cwd: './ui/dashboard-backend',
      instances: 1,
      autorestart: true,
      watch: false,
      max_memory_restart: '512M',
      env: {
        PORT: 10004,
        NODE_ENV: 'production'
      }
    }
  ]
};

🎯 Application Layer

⚛️ React Frontend (Port 10001)

ui/dashboard/
├── src/
│   ├── components/
│   │   ├── Layout/
│   │   │   ├── Sidebar.js          # Navigation sidebar
│   │   │   ├── TopBar.js           # Header navigation
│   │   │   └── Layout.js           # Main layout wrapper
│   │   ├── Charts/
│   │   │   ├── CandlestickChart.js # Financial charts
│   │   │   ├── QuantumChart.js     # Quantum visualizations
│   │   │   └── HyperdimensionalChart.js # 3D patterns
│   │   ├── Cards/
│   │   │   ├── MetricCard.js       # Performance metrics
│   │   │   ├── AgentCard.js        # AI agent status
│   │   │   └── AssetCard.js        # Asset information
│   │   └── Common/
│   │       ├── LoadingSpinner.js   # Loading animations
│   │       └── ErrorBoundary.js    # Error handling
│   ├── pages/
│   │   ├── Dashboard.js            # Main overview
│   │   ├── Trades.js               # Trading interface
│   │   ├── Agents.js               # AI agent management
│   │   ├── Assets.js               # Asset analysis
│   │   ├── SocialMediaInsights.js  # Sentiment analysis
│   │   ├── GeminiIntelligence.js   # AI market analysis
│   │   ├── Leaderboard.js          # Performance rankings
│   │   ├── Metrics.js              # System analytics
│   │   ├── Quantum.js              # Quantum computing
│   │   ├── Hyperdimensional.js     # Pattern recognition
│   │   ├── StrategyBuilder.js      # Strategy creation
│   │   ├── Reports.js              # Performance reports
│   │   ├── Settings.js             # Configuration
│   │   └── Documentation.js        # Help and docs
│   ├── services/
│   │   ├── api.js                  # API client
│   │   ├── websocket.js            # WebSocket client
│   │   └── grpc.js                 # gRPC client
│   ├── utils/
│   │   ├── formatters.js           # Data formatting
│   │   ├── validators.js           # Input validation
│   │   └── constants.js            # Application constants
│   └── styles/
│       ├── theme.js                # Material-UI theme
│       ├── global.css              # Global styles
│       └── animations.css          # Framer Motion styles

🔌 Express.js Backend (Port 10002)

ui/dashboard-backend/
├── src/
│   ├── routes/
│   │   ├── system.js               # System status endpoints
│   │   ├── trades.js               # Trading endpoints
│   │   ├── agents.js               # AI agent endpoints
│   │   ├── assets.js               # Asset data endpoints
│   │   ├── sentiment.js            # Sentiment analysis
│   │   ├── gemini.js               # Gemini AI integration
│   │   ├── metrics.js              # Performance metrics
│   │   ├── quantum.js              # Quantum computing
│   │   └── hyperdimensional.js     # Pattern recognition
│   ├── middleware/
│   │   ├── auth.js                 # Authentication
│   │   ├── cors.js                 # CORS configuration
│   │   ├── rateLimit.js            # Rate limiting
│   │   └── logging.js              # Request logging
│   ├── services/
│   │   ├── bybitService.js         # Bybit API integration
│   │   ├── mcpService.js           # MCP orchestration
│   │   ├── geminiService.js        # Gemini AI service
│   │   └── sentimentService.js     # Sentiment analysis
│   ├── utils/
│   │   ├── cache.js                # NodeCache implementation
│   │   ├── logger.js               # Winston logging
│   │   └── validators.js           # Input validation
│   └── config/
│       ├── database.js             # Database configuration
│       ├── redis.js                # Redis configuration
│       └── environment.js          # Environment variables

🦀 Rust Trading Engine

🏗️ Core Architecture

src/
├── main.rs                         # Application entry point
├── trading_system.rs               # Main trading logic
├── neural_interface.rs             # AI neural networks
├── agents/
│   ├── mod.rs                      # Agent module exports
│   ├── zero_loss_enforcer.rs       # Risk management
│   ├── quantum_predictor.rs        # Quantum algorithms
│   ├── ghost_trader.rs             # Trade simulation
│   ├── god_kernel.rs               # Agent coordination
│   ├── memory_node.rs              # Data persistence
│   ├── compound_controller.rs      # Capital management
│   ├── asset_scanner.rs            # Market scanning
│   ├── high_frequency_trader.rs    # HFT execution
│   ├── risk_manager.rs             # Risk assessment
│   ├── spectral_tree_engine.rs     # Pattern analysis
│   ├── hyperdimensional_computer.rs # Advanced math
│   ├── quantum_entanglement.rs     # Correlation analysis
│   ├── multi_factor_strategy.rs    # Strategy development
│   ├── trade_executor.rs           # Order execution
│   ├── position_manager.rs         # Portfolio management
│   ├── strategy_evolution.rs       # Self-improvement
│   ├── sentiment_analyzer.rs       # Market sentiment
│   └── momentum_detector.rs        # Trend analysis
├── quantum/
│   ├── mod.rs                      # Quantum module exports
│   ├── algorithms.rs               # Quantum algorithms
│   ├── entanglement.rs             # Quantum entanglement
│   ├── superposition.rs            # Quantum superposition
│   └── interference.rs             # Quantum interference
├── exchange/
│   ├── mod.rs                      # Exchange module exports
│   ├── bybit.rs                    # Bybit integration
│   ├── websocket.rs                # WebSocket handling
│   └── rest_client.rs              # REST API client
├── strategy/
│   ├── mod.rs                      # Strategy module exports
│   ├── base.rs                     # Base strategy trait
│   ├── quantum_strategy.rs         # Quantum strategies
│   └── hyperdimensional_strategy.rs # HD strategies
├── execution/
│   ├── mod.rs                      # Execution module exports
│   ├── order_manager.rs            # Order management
│   ├── position_tracker.rs         # Position tracking
│   └── risk_calculator.rs          # Risk calculations
├── position/
│   ├── mod.rs                      # Position module exports
│   ├── manager.rs                  # Position management
│   ├── tracker.rs                  # Position tracking
│   └── calculator.rs               # P&L calculations
└── market_data/
    ├── mod.rs                      # Market data exports
    ├── processor.rs                # Data processing
    ├── aggregator.rs               # Data aggregation
    └── analyzer.rs                 # Technical analysis

🔗 External Integrations

💹 Bybit Demo Trading API

  • Base URL: https://api-demo.bybit.com
  • WebSocket: wss://stream-demo.bybit.com
  • Authentication: API Key + Secret
  • Rate Limits: 100 requests/second
  • Supported Markets: USDT Perpetual, Spot, Options

🧠 MCP Service Integrations

  1. Exa Search MCP: Web search and intelligence
  2. Browserbase MCP: Cloud browser automation
  3. ScrapeGraph MCP: AI-powered data extraction
  4. Cloudflare Playwright: Browser automation
  5. Browser Use MCP: Browser interaction
  6. Firecrawl MCP: Web crawling and extraction
  7. Twikit Twitter MCP: Social media intelligence

🤖 AI Service Integrations

  • Gemini AI: Comprehensive market analysis
  • Sentiment Analysis: Social media monitoring
  • News API: Real-time financial news
  • Technical Analysis: TA-Lib integration

📊 Performance Specifications

⚡ Response Time Targets

  • API Endpoints: <700ms (currently 450ms average)
  • WebSocket Updates: <100ms latency
  • gRPC Calls: Microsecond-level response
  • Trading Execution: Sub-second order placement
  • Data Processing: Real-time stream processing

💾 Resource Utilization

  • API Server: 581.4MB memory, 0% CPU (idle)
  • WebSocket Server: 219.4MB memory, 35.3% CPU
  • gRPC Server: 219.1MB memory, 29.4% CPU
  • Frontend: 60.7MB memory, minimal CPU
  • Total System: ~1.1GB memory footprint

🔄 Scalability Features

  • Horizontal Scaling: PM2 cluster mode support
  • Load Balancing: NGINX upstream configuration
  • Caching: NodeCache + Redis integration
  • Database: Scalable database architecture
  • CDN: Static asset optimization

🔒 Security Architecture

🛡️ Security Layers

  1. Network Security: NGINX security headers
  2. Application Security: Input validation, sanitization
  3. API Security: Rate limiting, authentication
  4. Data Security: Encryption at rest and in transit
  5. Infrastructure Security: AWS security groups

🔐 Authentication & Authorization

  • API Keys: Secure credential management
  • Environment Variables: Sensitive data protection
  • CORS: Cross-origin request control
  • Rate Limiting: DDoS protection
  • SSL/TLS: HTTPS encryption ready

This architecture documentation provides a comprehensive overview of the OMNI system's technical implementation, from infrastructure to application layers, ensuring maintainability and scalability for future development.