Skip to content

Add worker system and event bus protocols for backend services#443

Merged
hotlong merged 3 commits into
mainfrom
copilot/generate-rest-api-endpoints
Jan 31, 2026
Merged

Add worker system and event bus protocols for backend services#443
hotlong merged 3 commits into
mainfrom
copilot/generate-rest-api-endpoints

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Jan 31, 2026

Implements backend service infrastructure protocols for task processing and event-driven architectures. REST/GraphQL server protocols already existed and required no changes.

Worker System (worker.zod.ts)

Priority-based task queue system with retry logic and batch processing:

const task: Task = {
  type: 'send_email',
  payload: { to: 'user@example.com', template: 'welcome' },
  priority: 'high',  // critical | high | normal | low | background
  retryPolicy: {
    maxRetries: 3,
    backoffStrategy: 'exponential',
    initialDelayMs: 1000,
  },
  queue: 'notifications',
};

const batch: BatchTask = {
  type: 'import_records',
  items: customers,  // Array of items
  batchSize: 100,
  parallel: true,
  stopOnError: false,
};
  • Task priority system (0-4) with queue management
  • Retry policies: exponential, linear, fixed backoff
  • Batch processing with progress callbacks
  • Queue configuration: concurrency, rate limiting, auto-scaling
  • Worker stats and monitoring

Event Bus Enhancements (events.zod.ts)

Extended event system with sourcing, replay, and integration patterns:

const config: EventBusConfig = {
  eventSourcing: {
    enabled: true,
    snapshotInterval: 100,
    retention: 365,
  },
  queue: {
    concurrency: 20,
    priorityEnabled: true,
    deadLetterQueue: 'failed_events',
  },
  webhooks: [{
    eventPattern: 'order.*',
    url: 'https://api.example.com/webhook',
    retryPolicy: { maxRetries: 3 },
  }],
  messageQueue: {
    provider: 'kafka',
    topic: 'events',
    partitionKey: 'metadata.tenantId',
  },
  realtime: {
    protocol: 'websocket',
    channels: [{ name: 'notifications', eventPattern: 'notification.*' }],
  },
};
  • Event priority levels and filtering
  • Event sourcing with snapshots
  • Event replay with timestamp ranges
  • Dead letter queue for failed handlers
  • Webhook integration with auth and retry
  • Message queue adapters (Kafka, RabbitMQ, SQS, Pub/Sub)
  • Real-time broadcasting via WebSocket/SSE
  • Event logging with handler execution tracking

Test Coverage

  • 39 tests for worker system
  • 53 tests for event bus (enhanced from 13)
  • All 2,955 tests passing
Original prompt

REST API 服务器

RESTful 路由自动生成
CRUD 端点
批量操作端点
元数据端点
GraphQL 服务器

Schema 自动生成
Resolver 自动生成
订阅支持
DataLoader 集成
P0: 工作器 (Worker)

后台任务系统

任务队列
任务调度
任务重试
任务优先级
定时任务 (Cron Jobs)

Cron 表达式解析
任务调度器
任务日志
错误处理
批处理作业

P0: 事件总线 (Event Bus)

事件系统增强

事件类型定义
事件优先级
事件过滤
事件持久化
异步事件处理

事件队列
事件重放
死信队列
事件源 (Event Sourcing)
事件集成

Webhook 支持
消息队列集成 (Kafka, RabbitMQ)
实时通知
事件日志


💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

@vercel
Copy link
Copy Markdown

vercel Bot commented Jan 31, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
spec Ready Ready Preview, Comment Jan 31, 2026 9:19am

Request Review

Copilot AI and others added 2 commits January 31, 2026 09:15
- Created worker.zod.ts with task queues, priorities, retry logic, and batch processing
- Enhanced events.zod.ts with event priorities, filtering, queuing, sourcing, and integrations
- Added comprehensive test coverage for all new features
- All 2955 tests passing including 39 worker tests and 53 event tests

Co-authored-by: hotlong <50353452+hotlong@users.noreply.github.com>
- Add detailed documentation for onProgress callback in BatchTaskSchema
- Fix naming collision by converting EventBusConfig helper to function
- All 2955 tests passing

Co-authored-by: hotlong <50353452+hotlong@users.noreply.github.com>
Copilot AI changed the title [WIP] Add RESTful route generation for API server Add worker system and event bus protocols for backend services Jan 31, 2026
Copilot AI requested a review from hotlong January 31, 2026 09:20
@github-actions
Copy link
Copy Markdown
Contributor

This PR is very large. Consider breaking it into smaller PRs for easier review.

@hotlong hotlong marked this pull request as ready for review January 31, 2026 09:36
Copilot AI review requested due to automatic review settings January 31, 2026 09:36
@hotlong hotlong merged commit bfb4e4d into main Jan 31, 2026
13 checks passed
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR implements comprehensive backend service protocols for task processing and event-driven architectures, adding a worker system for background job processing and significantly enhancing the event bus with enterprise-grade features.

Changes:

  • Introduces a complete worker/task queue system (worker.zod.ts) with priority-based processing, retry policies, batch operations, and monitoring capabilities similar to industry-standard systems like Sidekiq, Celery, and BullMQ
  • Extends the event bus protocol (events.zod.ts) with event sourcing, replay capabilities, dead letter queues, webhook integrations, message queue adapters, and real-time broadcasting
  • Provides comprehensive test coverage with 39 tests for the worker system and 40+ new tests for event bus enhancements

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated no comments.

Show a summary per file
File Description
packages/spec/src/system/worker.zod.ts New worker system protocol with task queuing, priorities, retry logic, batch processing, and monitoring schemas
packages/spec/src/system/worker.test.ts Comprehensive test suite for worker system covering all schemas, edge cases, and integration scenarios
packages/spec/src/system/events.zod.ts Enhanced event bus with event sourcing, replay, webhooks, message queue integration, real-time notifications, and event logging
packages/spec/src/system/events.test.ts Extended test suite with 40+ new tests covering all event bus enhancements
packages/spec/src/system/index.ts Export addition for new worker system

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants