[Submission] temporal-workdir — Remote-backed workspace sync for Temporal activities

[saeedseyfi]

Project link

https://github.com/saeedseyfi/temporal-workdir

Language

Python

Short description (max 256 chars)

Syncs a local directory with remote storage (GCS, S3, Azure) before and after Temporal activity execution. Enables file-based activities to work across distributed workers where disk is not shared.

Long Description

Temporal activities that read/write files on local disk can't scale to multiple worker instances. Each worker has its own disk with no shared state. The workaround is manually downloading from and uploading to object storage at the start and end of every activity. Every team reimplements this boilerplate.

Other orchestrators solved this years ago. Flyte has FlyteDirectory for automatic directory sync between tasks. Argo has artifact passing with declarative tarballing between steps. Temporal had no equivalent.

temporal-workdir fills that gap. It provides a Workspace that pulls a remote archive to a local temp directory before the activity runs, and pushes changes back after. The activity works with real files on real disk. No FUSE, no custom APIs.

Context manager (generic):

async with Workspace("gs://bucket/state/job-123") as ws:
    data = json.loads((ws.path / "config.json").read_text())
    (ws.path / "result.csv").write_text(compute(data))

Temporal decorator:

@workspace("gs://bucket/{workflow_id}/{activity_type}")
@activity.defn
async def process(input: ProcessInput) -> ProcessOutput:
    ws = get_workspace_path()
    ...

Storage backend auto-detected from URL scheme via fsspec (GCS, S3, Azure, local, memory). Published on PyPI as temporal-workdir.

Author(s)

Saeed Seyfi — https://github.com/saeedseyfi

[github-actions]

Hi, I'm ZiggyBot! 🤖 Here's my pre-evaluation of this submission:

---

Criterion	Assessment
Useful to Temporal users	✅ Yes — Solves a real pain point for teams running file-based activities across distributed workers, eliminating repetitive boilerplate for remote storage sync.
Benefits clearly explained	✅ Yes — The README clearly articulates the problem (no shared disk across workers), the solution (automatic pull/push with remote storage), and provides concrete usage examples.
OSI Approved License	✅ Yes — MIT License, which is OSI approved and community-friendly.
README with working instructions	✅ Yes — Includes installation commands with optional cloud backend packages, two usage patterns (context manager and decorator), and a clear explanation of the pull/execute/push lifecycle.

Overall: Looks good ✅

Notes:

• The project addresses a genuine gap in the Temporal ecosystem by providing artifact/directory sync functionality that other orchestrators (Flyte, Argo) have built-in. The comparison to FlyteDirectory and Argo artifacts in the submission helps contextualize the need.
• Clean separation of concerns: archive utilities handle tar/untar, workspace handles sync logic, and the Temporal decorator integrates activity context.
• Good security consideration in _archive.py with path traversal prevention during unpacking.
• The fsspec integration is clever — it means broad storage backend support without reinventing wheels.
• Zero stars suggests this is a fresh submission, but code quality looks solid for an early-stage project.
• Published on PyPI (temporal-workdir) which lowers the barrier to adoption.

Suggested questions for submitter:

• How does the library handle concurrent activity executions writing to the same remote path? Is there any locking mechanism, or is conflict resolution left to the user?
• Have you considered adding support for incremental sync (only changed files) for large workspaces where full tar/untar might be expensive?
• What happens if the push fails after activity completion — is there any retry logic or compensation mechanism?

Teaching moment:

The @workspace decorator demonstrates a powerful pattern for extending activity behavior without modifying activity logic. By combining Python's decorator pattern with activity.info() for runtime context (workflow_id, activity_type), the library shows how to build reusable infrastructure that integrates cleanly with Temporal's execution model. The template variable approach ("gs://bucket/{workflow_id}/{activity_type}") is particularly elegant — it automatically scopes workspace paths to the current execution context, preventing accidental state collisions across workflows. This is a great example of building "invisible infrastructure" that Temporal activity authors can adopt with minimal code changes.
View in source ↗

---

---

ZiggyBot is an AI pre-screener based on Temporal's community mascot Ziggy. Final decisions are made by the community team.

Reviewer: Check the box below to generate a Contentful-ready long description for this submission.

• Generate long description if needed

[github-actions]

Hi, I'm ZiggyBot! 🤖 Here's my pre-evaluation of this submission:

---

Criterion	Assessment
Useful to Temporal users	✅ Yes — Addresses a real pain point where file-based activities can't share disk state across distributed workers, providing a clean abstraction that other orchestrators already have.
Benefits clearly explained	✅ Yes — The README clearly explains the problem (workers with separate disks), the solution (sync to/from remote storage), and provides concrete usage examples with both context manager and decorator patterns.
OSI Approved License	✅ Yes — Licensed under MIT, which is OSI-approved and the recommended license.
README with working instructions	✅ Yes — Installation instructions cover pip with optional cloud backend extras, and usage examples show both generic and Temporal-specific patterns with clear code samples.

Overall: Looks good ✅

Notes:

• The project solves a legitimate gap in the Temporal ecosystem — while Flyte has FlyteDirectory and Argo has artifact passing, Temporal users have had to roll their own file sync logic.
• Clean separation of concerns: _workspace.py handles the generic sync logic, _temporal.py provides the Temporal-specific decorator integration, and _archive.py handles tar/untar operations.
• Good security awareness with path traversal protection in the archive unpacking code.
• The fsspec integration is clever — it automatically supports multiple cloud backends without the library needing to implement each one.
• Zero stars is expected for a new submission, not a concern.
• The transactional semantics (no upload on exception) align well with Temporal's retry model — failed activities won't corrupt remote state.

Suggested questions for submitter:

• How does this interact with Activity retries? If an activity fails mid-execution, are there any edge cases around partial state that users should be aware of?
• Have you considered adding heartbeat integration for long-running file operations to prevent activity timeouts during large archive uploads/downloads?

Teaching moment:

The @workspace decorator pattern demonstrates an elegant way to inject cross-cutting concerns into Temporal activities while preserving activity semantics. The template variable resolution using activity.info() (e.g., {workflow_id}/{activity_type}) is particularly clever — it lets users create deterministic, workflow-scoped storage paths without manual wiring. This pattern of "decorator that enriches activity context" is reusable for other infrastructure concerns like metrics, tracing, or resource management.
View in source ↗

---

---

ZiggyBot is an AI pre-screener based on Temporal's community mascot Ziggy. Final decisions are made by the community team.

Reviewer: Check the box below to generate a Contentful-ready long description for this submission.

• Generate long description if needed