Skip to content

Unify file creation and signed upload sessions - #6264

Merged
TheodoreSpeaks merged 4 commits into
improvement/v2-endpointsfrom
feat/v2-knowledge-small-file-put
Aug 5, 2026
Merged

Unify file creation and signed upload sessions#6264
TheodoreSpeaks merged 4 commits into
improvement/v2-endpointsfrom
feat/v2-knowledge-small-file-put

Conversation

@TheodoreSpeaks

@TheodoreSpeaks TheodoreSpeaks commented Aug 4, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • replace legacy buffered, presigned, fallback, and multipart upload paths with stateless signed upload sessions
  • use a single signed PUT for workspace files from zero bytes through 50 MiB, with multipart transfer above that boundary
  • add immutable staging, provider-native create-only promotion, replay-safe completion, conditional cleanup, and bounded local-storage cleanup
  • migrate workspace files, knowledge documents, table imports, profile pictures, workspace logos, mothership attachments, and execution attachments
  • add authored file creation to the workspace UI and public v2 API, with optional UTF-8 or base64 initial content
  • harden create and content-replace APIs with authorization-before-body parsing, canonical base64 validation, bounded JSON bodies, and canonical v2 errors
  • return the committed workspace-file row directly so successful creates cannot fail on a follow-up read
  • bound multi-file admission, transfer concurrency, and table-import memory usage
  • update OpenAPI and self-hosting CORS/lifecycle documentation
  • align the v2 service-account credential contract with the existing dataCenter field required by its shared field discriminator

Test plan

  • bun run lint
  • bun run type-check
  • bun run check:boundaries
  • bun run check:api-validation:strict
  • bun run check:openapi
  • bun run check:client-boundary
  • bun run check:utils
  • bun run check:zustand-v5
  • bun run check:react-query
  • bun run check:realtime-prune
  • 17 upload/create/content-focused Vitest files, 186 tests
  • git diff --check

@vercel

vercel Bot commented Aug 4, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
docs Skipped Skipped Aug 4, 2026 11:34pm

Request Review

@cursor

cursor Bot commented Aug 4, 2026

Copy link
Copy Markdown

PR Summary

High Risk
Touches authentication, workspace permissions, storage quota accounting, and public v2 file/upload contracts across many upload purposes; regressions could break uploads, billing, or API clients.

Overview
Replaces buffered multipart/form-data uploads, standalone presigned URL minting, and the legacy /api/files/multipart action router with purpose-scoped signed upload sessions under /api/files/uploads (create, part URLs, complete, abort). Clients get a signed control token plus either a single PUT (empty through 50 MiB) or multipart transfer instructions for larger files, with domain-specific finalization in finalizers and purposes.

v2 API and clients shift workspace file creation to JSON POST /api/v2/files for small inline content; larger or streamed bytes use upload-session endpoints. Knowledge documents, table imports, and related OpenAPI schemas now describe PUT vs multipart completion and separate session envelopes from transfer metadata.

Self-hosting docs add Azure Blob CORS for direct uploads, extra GCS exposed metadata headers, and lifecycle rules to expire staged upload-sessions/ objects (and incomplete multipart where supported).

Also documents optional dataCenter on v2 service-account credential create/update schemas.

Reviewed by Cursor Bugbot for commit c03fc3d. Bugbot is set up for automated code reviews on this repo. Configure here.

@TheodoreSpeaks

Copy link
Copy Markdown
Collaborator Author

@greptile-apps

Comment thread apps/sim/app/api/files/uploads/finalizers.ts
Comment thread apps/sim/lib/uploads/upload-session/service.ts
@greptile-apps

greptile-apps Bot commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

The PR replaces legacy buffered and presigned upload paths with a unified stateless signed-session model.

  • Uses one signed PUT through 50 MiB and multipart transfer above that boundary.
  • Adds immutable staging, create-only promotion, replay-safe finalization, abort handling, and bounded cleanup.
  • Migrates workspace files, knowledge documents, tables, profile assets, and workflow attachments to the shared flow.
  • Updates upload contracts, provider implementations, OpenAPI specifications, and self-hosting guidance.

Confidence Score: 5/5

The PR appears safe to merge based on the eligible follow-up review scope.

No blocking failure remains in the available scoring findings.

Important Files Changed

Filename Overview
apps/sim/lib/uploads/upload-session/service.ts Implements the central stateless session lifecycle, including validation, promotion, replay handling, finalization, abort, and cleanup.
apps/sim/lib/uploads/upload-session/provider.ts Unifies local, S3, GCS, and Azure staging, multipart assembly, create-only promotion, and conditional deletion operations.
apps/sim/lib/uploads/client/upload-session.ts Implements the shared browser PUT and multipart transfer engine with retries, cancellation, progress, and bounded concurrency.
apps/sim/app/api/files/uploads/finalizers.ts Adds purpose-specific, conflict-aware finalization for workspace files and internal attachment or profile assets.
apps/sim/lib/table/orchestration/import-resource.ts Migrates table imports to uploaded resources while bounding import processing and preserving retryable orchestration.
apps/sim/lib/api/contracts/upload-sessions.ts Defines the shared signed-session request, transfer, completion, and purpose-result contracts.
apps/docs/content/docs/en/platform/self-hosting/object-storage.mdx Documents browser CORS requirements and lifecycle cleanup for staged upload objects and abandoned multipart data.

Sequence Diagram

sequenceDiagram
  participant Client
  participant API as Upload Session API
  participant Store as Object Storage
  participant Finalizer as Purpose Finalizer
  Client->>API: Create session with file metadata
  API-->>Client: Signed token and PUT or multipart transfer
  Client->>Store: Upload into immutable staging key
  Client->>API: Complete session
  API->>Store: Verify staged object or assemble parts
  API->>Store: Promote to create-only final key
  API->>Finalizer: Register file, document, or import
  API->>Store: Conditionally remove staging object
  API-->>Client: Purpose-specific result
Loading

Reviews (2): Last reviewed commit: "fix(uploads): preserve attachment storag..." | Re-trigger Greptile

@TheodoreSpeaks

Copy link
Copy Markdown
Collaborator Author

@greptile-apps

@TheodoreSpeaks

Copy link
Copy Markdown
Collaborator Author

@cursor review

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit b1d4d63. Configure here.

@TheodoreSpeaks TheodoreSpeaks changed the title Unify signed file upload sessions Unify file creation and signed upload sessions Aug 4, 2026

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 2 potential issues.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit c03fc3d. Configure here.


const { workspaceId, name, contentType, folderId, content, encoding } = parsed.data.body
const access = await resolveWorkspaceAccess(rateLimit, userId, workspaceId, 'write')
if (access) return v2WorkspaceAccessError(access)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

V2 create parses before authz

Medium Severity

The new POST /api/v2/files handler runs parseRequest with a 70 MiB JSON cap before resolveWorkspaceAccess verifies workspace write permission. Callers with a valid API key but without write access can still force the server to read and validate large bodies for arbitrary workspaceId values, unlike the workspace JSON create route in the same change, which checks permissions before parsing.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit c03fc3d. Configure here.

if (!params.contentType.trim()) {
throw new UploadSessionError('validation', 'contentType must not be empty')
}
const minimum = params.purpose === 'workspace_file' ? 0 : 1

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Zero-byte session quota block

Medium Severity

Allowing workspace_file upload sessions with fileSize: 0 still runs checkStorageQuotaForBillingContext with zero additional bytes. When recorded usage is already above the payer limit, that check fails even though completion would not increase storage, so signed-session empty uploads are rejected while inline POST create paths can still add zero-byte files.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit c03fc3d. Configure here.

@TheodoreSpeaks
TheodoreSpeaks merged commit f01c4be into improvement/v2-endpoints Aug 5, 2026
4 checks passed
@TheodoreSpeaks
TheodoreSpeaks deleted the feat/v2-knowledge-small-file-put branch August 5, 2026 00:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant