Skip to content

Seed buckets: skip OS junk files (.DS_Store) in objects_path, or support an ignore list #5931

Description

@borzos6

Existing issues

  • I have searched the existing issues.

Affected area

Storage

Problem to solve

When a bucket declares objects_path together with an image-only allowed_mime_types, seeding fails as soon as macOS Finder has dropped a .DS_Store into the seed directory:

[storage.buckets.app_content]
public = true
file_size_limit = "10MiB"
allowed_mime_types = ["image/png", "image/jpeg", "image/gif", "image/webp"]
objects_path = "./seed/app_content"

Creating Storage bucket: app_content
Uploading: supabase/seed/app_content/.DS_Store => app_content/.DS_Store
...
Error status 400: {"statusCode":"415","error":"invalid_mime_type","message":"mime type application/octet-stream is not supported"}

Two failure modes compound:

  1. The junk file itself is rejected (415), which aborts the whole seed run — remaining legitimate objects never upload.
  2. On buckets without a MIME allowlist it's arguably worse: the junk uploads silently and .DS_Store becomes a public object.

.DS_Store appears any time someone merely opens the seed folder in Finder, so this recurs for every macOS contributor. Windows has the same issue with Thumbs.db / desktop.ini.

Proposed solution

In UpsertObjects (pkg/storage/batch.go), where fs.WalkDir currently uploads every regular file and isUploadableEntry only filters non-regular files, either:

  • (a) minimal: skip well-known OS metadata files by default (.DS_Store, Thumbs.db, desktop.ini) — matching what most deploy tools (e.g. gcloud, Netlify, Vercel CLIs) do implicitly; or
  • (b) general: skip hidden files/directories (dot-prefixed) by default, with an opt-in to include them; or
  • (c) configurable: an objects_ignore = [".DS_Store", "*.tmp"] glob list per bucket in config.toml.

Even just (a) would eliminate the common failure. A skipped file could log Skipping OS metadata file: … alongside the existing Skipping non-regular file: message.

Alternatives considered

  • Deleting junk pre-seed via a wrapper script (our current workaround — an extra cross-platform cleanup script wired in front of every supabase seed buckets invocation).
  • Widening allowed_mime_types to admit application/octet-stream — defeats the purpose of the allowlist.
  • defaults write com.apple.desktopservices DSDontWriteNetworkStores — only affects network volumes, not local directories.

Additional context

Observed on CLI 2.109.0; the walk in current main (apps/cli-go/pkg/storage/batch.go, UpsertObjects) still uploads unconditionally. Related: #12390 (storage seeding feature thread).

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions